Skip to content

refactor: harden outbound clients and generated media retrieval - #570

Merged
chenyme merged 3 commits into
devfrom
enhancement
Aug 4, 2026
Merged

refactor: harden outbound clients and generated media retrieval#570
chenyme merged 3 commits into
devfrom
enhancement

Conversation

@chenyme

@chenyme chenyme commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Summary

统一收敛身份源、Turnstile、Stripe、模型、MCP、Embedding、模型生成媒体等外部 HTTP 调用,建立清晰、可复用的基础设施适配器边界,并在不破坏现有上游兼容性的前提下应用一致的出站安全策略。

本次变更包括:

  • 将 OIDC/OAuth2、Discovery 和 Turnstile 请求迁移到身份源基础设施适配器。
  • 将 Stripe Checkout 请求迁移到独立支付基础设施适配器。
  • 将模型、MCP、Embedding 出站请求接入统一的有界 HTTP 客户端池。
  • 支持管理员配置的模型、MCP、Embedding、身份源和 Turnstile 私网 endpoint 按精确 origin 局部信任。
  • 初始请求必须匹配管理员配置的 scheme + host + port,避免局部信任被其他请求复用。
  • 保留现有重定向兼容性:
    • 同 origin 重定向正常跟随;
    • 跨 origin 公网重定向正常跟随;
    • 跨 origin 私网重定向命中 SSRF_ALLOWED_HOSTSSSRF_ALLOWED_CIDRS 时正常跟随;
    • 关闭 SSRF 防护时保持原有重定向行为。
  • 保持 metadata、link-local、multicast、unspecified 等危险目标不可授权。
  • 新增图片和视频生成结果的统一 artifact 下载适配器。
  • 支持与当前模型 endpoint 同 origin 的私网媒体制品下载,并在校验后转存到平台存储。
  • 统一处理连接复用、超时、重定向、响应大小限制、MIME 校验、Gemini Files 轮询和错误分类。
  • 避免签名 URL、认证信息和上游响应内容进入日志或用户可见错误。
  • 增加 media.artifact_unavailable 稳定错误码及中英文前端文案。
  • 完善 OpenRouter 官方定价持久化缓存、并发刷新和过期缓存降级能力。
  • 在应用关闭时统一释放各出站适配器持有的空闲连接。
  • 更新中英文部署文档和示例配置,明确管理员 endpoint 局部信任与全局白名单的职责边界。

Change type

  • Bug fix
  • Feature
  • Documentation
  • Refactor
  • Configuration / deployment
  • Security hardening
  • Other

Affected areas

  • Frontend / UI
  • Backend / API
  • Authentication / authorization
  • Conversations / streaming
  • Files / RAG / extraction
  • Model routing / providers
  • MCP / tools
  • Billing / payments
  • Admin console
  • Deployment / Docker / configuration
  • Documentation

Verification

  • GOCACHE=/private/tmp/deeix-chat-origin-compat-full-cache go test ./...
  • GOCACHE=/private/tmp/deeix-chat-origin-compat-race-cache go test -race ./internal/infra/outboundhttp ./internal/infra/identityprovider ./internal/infra/llm ./internal/infra/mcp ./internal/infra/embedding ./internal/infra/mediaartifact ./internal/application/conversation
  • GOCACHE=/private/tmp/deeix-chat-origin-compat-workspace-cache pnpm exec turbo run check --env-mode=loose
  • GOCACHE=/private/tmp/deeix-chat-origin-compat-api-cache pnpm api:check
  • GOCACHE=/private/tmp/deeix-chat-origin-compat-build-cache go build -o /private/tmp/deeix-chat-origin-compat-check ./cmd/server
  • git diff --check
  • Not run; reason:

Screenshots, API examples, or logs

No layout changes requiring screenshots.

The frontend change is limited to the stable generated-media error code and its localized user-facing messages.

Automated coverage includes:

  • exact-origin endpoint isolation;
  • bounded LRU client reuse and connection cleanup;
  • concurrent client-pool access;
  • configured private model, MCP, Embedding and identity-provider endpoints;
  • public cross-origin redirects;
  • globally allowlisted private redirects;
  • rejection of non-allowlisted private redirects;
  • redirect compatibility when SSRF protection is disabled;
  • generated image and video download limits;
  • same-origin private media artifact downloads;
  • MIME validation and Gemini Files polling;
  • cross-origin Gemini credential handling;
  • sanitized operational and user-facing errors;
  • OpenRouter pricing cache persistence, concurrent refresh and stale-cache fallback.

Configuration, migration, and compatibility notes

  • Administrator-saved model, MCP, Embedding, OIDC/OAuth2, Discovery and custom Turnstile endpoints are locally authorized by exact origin and do not need global SSRF allowlist entries for direct access.
  • Existing SSRF_ALLOWED_HOSTS and SSRF_ALLOWED_CIDRS configuration remains compatible.
  • Public cross-origin redirects remain supported.
  • Private cross-origin redirects remain supported when covered by the global hostname or CIDR allowlist.
  • OIDC/OAuth2 and Turnstile retain their stricter identity-provider redirect boundary.
  • Existing private model, MCP and Embedding endpoints continue to work; they no longer require duplicate global allowlist configuration for direct access.
  • Generated public media URLs remain supported.
  • Generated private media URLs are supported when they share the selected model endpoint’s exact origin.
  • Base64 and Data URL media behavior is unchanged.
  • Metadata, link-local, multicast, unspecified and credential-embedded URLs cannot be authorized.
  • Invalid global allowlist configuration prevents backend startup.
  • Global allowlist changes require a backend restart.
  • No database migration is required.
  • No new public URL is introduced.
  • No API contract or Swagger change is required.
  • Generated TypeScript API contracts remain up to date.
  • Existing OIDC, Turnstile, Stripe, model routing, MCP, Embedding and generated-media configurations do not require migration.

Documentation

  • Documentation is not needed for this change.
  • Documentation was updated.
  • Documentation still needs to be updated.

Security and privacy

  • No secrets, tokens, credentials, local config, or personal data are included.
  • User data access remains scoped by authenticated user context unless an admin-only path explicitly requires broader access.
  • Security-sensitive behavior was reviewed, including authentication, authorization, provider routing, file processing, billing, generated media, redirects, and admin-configured endpoints.

Checklist

  • I searched existing issues and pull requests.
  • Changes are focused and do not include unrelated refactors.
  • Tests or static verification were run where practical.
  • User-facing behavior, deployment steps, API contracts, and configuration changes are documented.
  • Generated artifacts are included only when this project explicitly requires them.
  • Caches, build output, .pyc files, .env files, and local storage data are not committed.

@chenyme
chenyme merged commit 839411e into dev Aug 4, 2026
11 checks passed
@chenyme chenyme self-assigned this Aug 4, 2026
@chenyme
chenyme deleted the enhancement branch August 4, 2026 12:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant