fix(translation): encode Responses image URLs

[alam0rt]

Hey! Just writing here to show that a human is behind this PR. I required this patch (patching it using a Nix overlay) to avoid issues with mixing the different provider APIs via switchyard.

What

Fix OpenAI Responses request encoding for normalized image content.

The buffered Responses codec previously serialized the internal tagged ImageSource enum directly into input_image.image_url. That produces an object such as { "type": "raw", "data": ... }, while the Responses API requires image_url to be the scalar URL/data URL.

This change maps normalized image sources to the wire form Responses accepts:

• URL sources become scalar URLs.
• Base64 sources become data:<media-type>;base64,<payload> data URLs.
• Common raw Anthropic/OpenAI image shapes are recovered to their URL or data URL.
• Unmappable raw image shapes go through the existing lossy-translation diagnostic path.

Regression tests cover Anthropic URL and base64 image blocks translated into OpenAI Responses.

Why

Image-bearing Anthropic requests routed through Switchyard to an OpenAI Responses backend currently emit a malformed input_image block. The route can then fail with an upstream schema error instead of preserving the image.

How tested

TDD: added the two regression tests first and confirmed both failed before the codec change with the internal tagged ImageSource object in image_url; both pass after the fix.

• uv run ruff check . clean — not run; no Python files changed
• uv run mypy switchyard clean — not run; no Python files changed
• uv run pytest tests/ green — not run; no Python tests changed
• Manual smoke (describe what was run)

Rust checks run:

cargo fmt --all -- --check
cargo clippy -p switchyard-translation --all-targets -- -D warnings
cargo test -p switchyard-translation
cargo test -p switchyard-translation --test request_translation

Results: formatting clean; clippy clean; translation suite green, including all 59 request-translation tests.

Checklist

• One class per file; filename = snake_case of the primary class. — N/A; Rust codec change only
• New public symbols exported from switchyard/__init__.py.__all__ if intended for downstream use. — N/A; no public Python symbols
• Unit tests added for new components / bug fixes.
• README / --help updated if customer-facing surface changed. — N/A; bug fix only
• Commits signed off (Signed-off-by: Your Name <email>) per the DCO.

Notes for reviewers

This intentionally preserves the current scalar image_url output shape and does not add support for a Responses detail field. Unmappable raw image sources report lossy conversion through the existing policy hook rather than serializing a malformed internal enum.

Summary by CodeRabbit

• New Features

  • Improved image handling in Responses translations, including URL, base64, and raw image sources.
  • Base64-encoded images are converted into compatible data URLs with media types.

• Bug Fixes

  • Image sources that cannot be converted now produce a diagnostic instead of failing direct serialization.

• Tests

  • Added coverage for translating Anthropic image blocks into OpenAI Responses image inputs.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 97cba755-af17-4fa2-af66-7be428e50d0d

📥 Commits

Reviewing files that changed from the base of the PR and between 8dc8911 and 38e5a59.

📒 Files selected for processing (2)

• crates/switchyard-translation/src/codecs/responses/buffered.rs
• crates/switchyard-translation/tests/request_translation.rs

Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.

---

Walkthrough

The Responses codec now converts image sources into scalar URLs or data URLs. Unsupported image sources are omitted with a lossy diagnostic. Tests cover Anthropic base64 and URL image translation.

Changes

Responses image translation

Layer / File(s)	Summary
Image source mapping and encoding crates/switchyard-translation/src/codecs/responses/buffered.rs	The codec converts URL, base64, and raw image sources into response image URLs or data URLs. Image encoding uses this conversion and reports unmappable images through the lossy-diagnostic path.
Anthropic image translation coverage crates/switchyard-translation/tests/request_translation.rs	Tests verify base64 images become data URLs and URL images remain unchanged in OpenAI Responses input_image values.

Estimated code review effort: 2 (Simple) | ~15 minutes

Merge Risk: ⚪ Minimal · up to 38e5a

Responses image requests now encode URLs and base64 images in the required scalar format, with unsupported shapes handled by the existing lossy policy. The covered translation paths are ready to merge.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 8 functions across 2 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly identifies the main change: fixing image URL encoding in the OpenAI Responses translation codec. It is concise and related to the implementation and tests.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

• Fix all pre-merge checks with AI

---

_{A rabbit maps the images bright
From base64 clouds to data light
URLs hop through unchanged
Lost pictures leave a note arranged
Tests guard the path just right}

_{Comment @coderabbitai help to get the list of available commands.}