Skip to content

release: v1.3.0 — fix broken auto-upgrade, streaming corruption, and Claude Code compatibility - #28

Merged
MartinForReal merged 1 commit into
mainfrom
release/v1.3.0
Jul 27, 2026
Merged

release: v1.3.0 — fix broken auto-upgrade, streaming corruption, and Claude Code compatibility#28
MartinForReal merged 1 commit into
mainfrom
release/v1.3.0

Conversation

@MartinForReal

@MartinForReal MartinForReal commented Jul 27, 2026

Copy link
Copy Markdown
Owner

Description

Releases v1.3.0. The headline items are that auto_upgrade never worked (it panicked the process at startup), and that streaming responses could be silently corrupted or truncated in ways that then poisoned the conversation history on the next request.

Much of this came from diffing behaviour against ericc-ch/copilot-api and mining its issue tracker for defects this proxy shares.

Auto-upgrade was completely broken

maybe_auto_upgrade was called synchronously from the async main. self_update (reqwest backend) builds its own blocking client backed by a private Tokio runtime, and dropping that runtime inside an async context panics:

thread 'main' panicked at tokio/src/runtime/blocking/shutdown.rs:51:21:
Cannot drop a runtime in a context where blocking is not allowed.

So enabling auto_upgrade crashed the proxy on every start. It now runs on a blocking thread.

Separately, v1.2.3 was tagged with Cargo.toml still at 1.2.2. Binaries from that release report 1.2.2, so bump_is_greater(1.2.2, 1.2.3) stays true and they re-download the same release forever. Verified by running a binary stamped 1.2.0: it fetched and replaced itself with v1.2.3, and the replacement reported 1.2.2. The release workflow now fails when the tag and Cargo.toml disagree.

Streaming integrity

Every SSE chunk was decoded with String::from_utf8_lossy on arrival, but chunks do not respect UTF-8 character boundaries. A multi-byte character (CJK, emoji, ) split across two chunks became U+FFFD; the damaged payload then failed to parse as JSON and was dropped entirely, deleting text from the middle of a response with no error — and that corrupted/missing text was replayed into the next request as history. Parsing now buffers raw bytes and decodes only complete lines, the same shape TextDecoderStream + TextLineStream gives copilot-api.

Truncated streams were also delivered as if complete: a transport error mid-stream only did break, leaving the client with a partial answer, no terminator, and a 200 in the request log. All five streaming paths now emit a protocol-appropriate terminator and record 502. Anthropic streams that end without a finish_reason are closed with message_stop so Claude Code neither blocks nor keeps a partial turn as complete.

A : keepalive comment is emitted after 15s of silence — injected at event boundaries only, so the verbatim passthrough paths are never spliced mid-event — to survive the ~60s idle timeout that kills long thinking turns.

Claude Code / MCP compatibility

  • context_management was silently stripped, disabling Claude Code's context editing. It is forwarded now, with the context-management-2025-06-27 beta derived automatically. Verified end to end: the upstream response echoes "context_management":{"applied_edits":[]}.
  • The client's anthropic-beta header was overwritten by the 1M-context flag; flags are merged now.
  • tool_result content arrays were forwarded raw, so every MCP tool returning blocks rather than a bare string was rejected upstream. Text blocks are joined, image blocks become image_url data URLs.
  • Images nested in a tool_result did not set Copilot-Vision-Request.
  • Missing max_tokens is filled from the model catalog; models requiring max_completion_tokens get one automatic retry.
  • The local token estimate ignored per-message framing and tool schemas, so Claude Code compacted too late and then hit a hard limit failure. Same request with one tool definition: 15 → 67 tokens.

Other

Stopped scraping the AUR for the VS Code version — the Arch maintainers asked projects to stop, and this repo inherited the exact same URL. Now uses Microsoft's own release API.

Cost estimates were computed from the client-supplied model name rather than the model actually served, so the alias opus priced at the fallback rate (~38x low); the gpt-4o rate was also unreachable behind the broader gpt-4 arm.

Dashboard pagination could overflow usize and panic the handler, and an unbounded per_page forced a full clone of the request store.

Added

GET /health, GET /v1/models/{model}, graceful shutdown on Ctrl-C/SIGTERM, ghc_proxy_uptime_seconds, and a model= filter on /api/audit.

Related issues

None filed in this repo. Defects were cross-referenced against copilot-api issues #95 (tool_result arrays), #217 (context_management), #223 (stream idle timeout), #210 (token undercount), #6 (missing max_tokens), #222 (max_completion_tokens), and #262 (AUR).

Type of change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that changes existing behavior)
  • Documentation update
  • Other (please describe):

Checklist

  • cargo build succeeds
  • cargo test passes — 95 tests, 25 of them new
  • cargo clippy --all-targets is clean (-D warnings)
  • I have updated documentation where relevant — CHANGELOG.md, README.md, docs/api.md

Verification against live upstream

Beyond unit tests, the following were exercised against the real Copilot API:

  • Multi-byte payload (你好世界🎉こんにちは안녕하세요Ωμέγα) round-trips through both the OpenAI and Anthropic streaming paths with zero U+FFFD and correct termination.
  • context_management, missing max_tokens, tool-schema token accounting, and MCP block-array tool_result all confirmed working.
  • Auto-upgrade path template checked against all 7 published v1.2.3 assets, plus a real download-and-replace run.

One caveat worth recording: images inside tool_result still return image media type not supported. I isolated this by sending a native OpenAI image_url body straight through with no translation and getting the identical error, so it is an upstream Copilot limitation, not a translation defect — it is not claimed as fixed.

…lth/model APIs

Fixes auto-upgrade, which panicked on startup, and several defects that could
silently corrupt or truncate model output.

Streaming integrity
- Decode SSE by buffering raw bytes and splitting complete lines only. Chunks
  do not respect UTF-8 boundaries, so per-chunk from_utf8_lossy turned any
  multi-byte character split across chunks into U+FFFD; the damaged payload
  then failed to parse as JSON and was dropped, deleting text mid-response and
  replaying the corruption into the next request as history.
- Detect interrupted streams and emit a protocol-appropriate terminator
  (OpenAI error chunk + [DONE], Anthropic/Responses `event: error`, Gemini
  finishReason OTHER), recording the request as 502 instead of 200.
- Close Anthropic streams that end without a finish_reason so clients neither
  block on a missing message_stop nor keep a partial turn as complete.
- Flush a trailing event that arrives without a newline; forward unparsable
  data payloads instead of dropping them.
- Emit a `: keepalive` comment after 15s of silence, at event boundaries only,
  so extended thinking does not trip the ~60s upstream idle timeout.

Auto-upgrade
- Run self_update on a blocking thread. It owns a private Tokio runtime, and
  dropping that from the async main panicked the process, which made
  auto_upgrade: true unusable.
- Fail the release workflow when the tag does not match Cargo.toml. v1.2.3
  shipped as 1.2.2, so its binaries re-downloaded the same release forever.

Claude Code / MCP compatibility
- Forward context_management plus the derived context-management beta; it was
  stripped, silently disabling context editing.
- Merge the client's anthropic-beta header instead of overwriting it.
- Normalize tool_result content arrays; MCP tools returning blocks rather than
  a bare string were rejected upstream.
- Detect images nested in tool_result for the vision header, and translate
  url-source images.
- Fill missing max_tokens from the model catalog; retry once with
  max_completion_tokens for models that require it.
- Count per-message and tool-schema overhead in the local token estimate.
- Serve /v1/messages/count_tokens from upstream when available, else a local
  estimate, instead of always failing with 400.

Other fixes
- Stop scraping the AUR for the VS Code version; use Microsoft's release API.
- Price requests by the served model, and unshadow the gpt-4o rate.
- Clamp pagination and use saturating arithmetic; aggregate metrics and audit
  queries in place rather than cloning the whole request store.
- Add a connect timeout, rotate error.log, and drop a duplicate startup model
  fetch.

Added
- GET /health, GET /v1/models/{model}, graceful shutdown,
  ghc_proxy_uptime_seconds, and a model= filter on /api/audit.
@MartinForReal
MartinForReal merged commit 50b3ac9 into main Jul 27, 2026
1 check passed
@MartinForReal
MartinForReal deleted the release/v1.3.0 branch July 27, 2026 11:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant