Skip to content

CDN_ENDPOINT used for both server-side and presigned URL, browser cannot access in k8s #1267

@theothersideofgod

Description

@theothersideofgod

Problem

CDN_ENDPOINT environment variable is shared by two libraries:

  • bucket-provisioner-resolver — Server-side S3 operations
  • presigned-url-plugin — Generates presigned URL for browser upload

In k8s environment, these two need different endpoints, causing configuration conflict.

Execution Flow

Container needs to access MinIO
    ↓
Set CDN_ENDPOINT = "minio:9000"
    ↓
presigned-url-plugin also reads CDN_ENDPOINT
    ↓
Generated presigned URL = "http://minio:9000/bucket/key?signature=..."
    ↓
Browser receives this URL, tries to PUT
    ↓
❌ Fails! Browser doesn't recognize "minio" hostname

Root Cause

Location: pgpm/env/src/env.ts

cdn: {
  ...(CDN_ENDPOINT && { endpoint: CDN_ENDPOINT }),
}

Both libraries read from getEnvOptions().cdn.endpoint:

// bucket-provisioner-resolver.ts (server-side S3)
const { endpoint } = cdn;  // Used to create S3Client

// presigned-url-plugin (generates URL for browser)
// Also reads cdn.endpoint or storage_module.endpoint
Operation Executor Required Address
Server-side S3 Inside k8s pod minio:9000 (k8s internal DNS)
Presigned URL Browser localhost:9000 or external domain

Evidence

docker-compose (Solved ✅)

Using extra_hosts workaround:

extra_hosts:
  - "localhost:host-gateway"

CDN_ENDPOINT: "http://localhost:9000"

k8s local-simple (Unsolved ❌)

# constructive-server.yaml
- name: MINIO_ENDPOINT  # Note: code doesn't read this variable name!
  value: "http://minio:9000"

k8s dev/staging (Unsolved ❌)

No MinIO ingress configuration.

Suggested Fix

Option A: Ingress Unified Entry (Recommended)

Add Ingress for MinIO, providing a unified domain:

# k8s/overlays/dev/ingress-minio.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: minio-ingress
  annotations:
    nginx.ingress.kubernetes.io/proxy-body-size: "100m"
spec:
  rules:
    - host: minio.launchql.dev
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: minio
                port:
                  number: 9000

Then configure:

CDN_ENDPOINT: "https://minio.launchql.dev"

Option B: Separate Two Endpoints

Add new S3_ENDPOINT variable for server-side use:

S3_ENDPOINT: "http://minio:9000"       # server-side S3 operations
CDN_ENDPOINT: "http://localhost:9000"  # presigned URL for browser

Changes needed:

  • pgpm/env/src/env.ts — add S3_ENDPOINT mapping
  • bucket-provisioner-resolver.ts — prefer reading S3_ENDPOINT

Current Workaround

docker-compose uses extra_hosts: localhost:host-gateway to solve this.

No workaround for k8s environment.

Files

  • constructive/pgpm/env/src/env.ts — CDN_ENDPOINT mapping
  • constructive/graphile/graphile-settings/src/bucket-provisioner-resolver.ts — server-side S3
  • constructive/graphile/graphile-presigned-url-plugin/src/* — presigned URL generation
  • constructive-functions/k8s/overlays/local-simple/constructive-server.yaml — wrong variable name (MINIO_ENDPOINT)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions