Skip to content

fix: route image attachments through configured processors - #568

Merged
chenyme merged 2 commits into
devfrom
mcp
Aug 4, 2026
Merged

fix: route image attachments through configured processors#568
chenyme merged 2 commits into
devfrom
mcp

Conversation

@chenyme

@chenyme chenyme commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes #555.

Allow administrators to configure an MCP tool as an image attachment processor. When users select that tool and upload images, the backend calls the MCP processor first and sends only its textual analysis to the main model.

This change also:

  • Prevents raw images from silently falling back to the main model when processor resolution fails.
  • Enforces one image processor across manual selection, global defaults, project defaults, cached state, and backend validation.
  • Supports local $ref, anyOf, oneOf, and allOf when resolving string arguments from tool schemas.
  • Revalidates attachment mappings during MCP synchronization, preserving compatible mappings and clearing invalid ones.
  • Counts attachment processing against the existing tool-call limit.
  • Persists image analysis for subsequent conversation context.
  • Removes Data URL and Base64 payloads from tool audit output.
  • Records processor responses without textual analysis as failures.

Change type

  • Bug fix
  • Feature
  • Documentation
  • Refactor
  • Configuration / deployment
  • Security hardening
  • Other

Affected areas

  • Frontend / UI
  • Backend / API
  • Authentication / authorization
  • Conversations / streaming
  • Files / RAG / extraction
  • Model routing / providers
  • MCP / tools
  • Billing / payments
  • Admin console
  • Deployment / Docker / configuration
  • Documentation

Verification

  • pnpm verify
  • go test ./...
  • go vet ./...
  • go build ./...
  • pnpm --filter @deeix/web check
  • pnpm --filter @deeix/web test
  • API contract and generated Swagger consistency checks
  • Next.js production build
  • git diff --check
  • Not run; reason: Not applicable.

Screenshots, API examples, or logs

No new logs or request samples contain image payloads.

Browser-based screenshots were not captured. The UI was verified through linting, TypeScript checks, and the production build.

Configuration, migration, and compatibility notes

  • Adds attachment-processing metadata to MCP tool records:
    • attachment_input_mode
    • attachment_argument
    • attachment_encoding
    • attachment_prompt_argument
  • Existing MCP tools default to standard behavior with attachment_input_mode = none.
  • No manual data migration is required.
  • No new environment variables or deployment configuration are required.
  • Existing MCP tools remain unchanged unless an administrator explicitly enables image attachment processing.
  • The API contract and generated Swagger artifacts were updated.
  • Schema synchronization automatically clears attachment mappings that are no longer compatible with the remote tool schema.

Documentation

  • Documentation is not needed for this change.
  • Documentation was updated.
  • Documentation still needs to be updated.

The administrator-facing configuration includes inline descriptions and tooltips.

Security and privacy

  • No secrets, tokens, credentials, local config, or personal data are included.
  • User data access remains scoped by authenticated user context unless an admin-only path explicitly requires broader access.
  • Security-sensitive behavior was reviewed, including authentication, authorization, provider routing, file processing, billing, and admin APIs where relevant.

Image bytes are read through the existing user-scoped attachment flow. Raw image payloads are not persisted in tool-call input or output audit records, and the main model receives only the processor’s textual analysis after successful routing.

Checklist

  • I searched existing issues and pull requests.
  • Changes are focused and do not include unrelated refactors.
  • Tests or static verification were run where practical.
  • User-facing behavior, deployment steps, API contracts, or configuration changes are documented.
  • Generated artifacts are included only when this project explicitly requires them.
  • Caches, build output, .pyc files, .env files, and local storage data are not committed.

@chenyme chenyme self-assigned this Aug 4, 2026
@chenyme
chenyme merged commit 9b1227f into dev Aug 4, 2026
11 checks passed
@chenyme
chenyme deleted the mcp branch August 4, 2026 05:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: 图片附件直接内嵌主模型,勾选了 vision MCP 工具也不调用,非视觉模型无法识图

1 participant