release: v1.3.0 — fix broken auto-upgrade, streaming corruption, and Claude Code compatibility - #28
Merged
Merged
Conversation
…lth/model APIs
Fixes auto-upgrade, which panicked on startup, and several defects that could
silently corrupt or truncate model output.
Streaming integrity
- Decode SSE by buffering raw bytes and splitting complete lines only. Chunks
do not respect UTF-8 boundaries, so per-chunk from_utf8_lossy turned any
multi-byte character split across chunks into U+FFFD; the damaged payload
then failed to parse as JSON and was dropped, deleting text mid-response and
replaying the corruption into the next request as history.
- Detect interrupted streams and emit a protocol-appropriate terminator
(OpenAI error chunk + [DONE], Anthropic/Responses `event: error`, Gemini
finishReason OTHER), recording the request as 502 instead of 200.
- Close Anthropic streams that end without a finish_reason so clients neither
block on a missing message_stop nor keep a partial turn as complete.
- Flush a trailing event that arrives without a newline; forward unparsable
data payloads instead of dropping them.
- Emit a `: keepalive` comment after 15s of silence, at event boundaries only,
so extended thinking does not trip the ~60s upstream idle timeout.
Auto-upgrade
- Run self_update on a blocking thread. It owns a private Tokio runtime, and
dropping that from the async main panicked the process, which made
auto_upgrade: true unusable.
- Fail the release workflow when the tag does not match Cargo.toml. v1.2.3
shipped as 1.2.2, so its binaries re-downloaded the same release forever.
Claude Code / MCP compatibility
- Forward context_management plus the derived context-management beta; it was
stripped, silently disabling context editing.
- Merge the client's anthropic-beta header instead of overwriting it.
- Normalize tool_result content arrays; MCP tools returning blocks rather than
a bare string were rejected upstream.
- Detect images nested in tool_result for the vision header, and translate
url-source images.
- Fill missing max_tokens from the model catalog; retry once with
max_completion_tokens for models that require it.
- Count per-message and tool-schema overhead in the local token estimate.
- Serve /v1/messages/count_tokens from upstream when available, else a local
estimate, instead of always failing with 400.
Other fixes
- Stop scraping the AUR for the VS Code version; use Microsoft's release API.
- Price requests by the served model, and unshadow the gpt-4o rate.
- Clamp pagination and use saturating arithmetic; aggregate metrics and audit
queries in place rather than cloning the whole request store.
- Add a connect timeout, rotate error.log, and drop a duplicate startup model
fetch.
Added
- GET /health, GET /v1/models/{model}, graceful shutdown,
ghc_proxy_uptime_seconds, and a model= filter on /api/audit.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Releases v1.3.0. The headline items are that
auto_upgradenever worked (it panicked the process at startup), and that streaming responses could be silently corrupted or truncated in ways that then poisoned the conversation history on the next request.Much of this came from diffing behaviour against
ericc-ch/copilot-apiand mining its issue tracker for defects this proxy shares.Auto-upgrade was completely broken
maybe_auto_upgradewas called synchronously from the asyncmain.self_update(reqwest backend) builds its own blocking client backed by a private Tokio runtime, and dropping that runtime inside an async context panics:So enabling
auto_upgradecrashed the proxy on every start. It now runs on a blocking thread.Separately,
v1.2.3was tagged withCargo.tomlstill at1.2.2. Binaries from that release report1.2.2, sobump_is_greater(1.2.2, 1.2.3)stays true and they re-download the same release forever. Verified by running a binary stamped1.2.0: it fetched and replaced itself with v1.2.3, and the replacement reported1.2.2. The release workflow now fails when the tag andCargo.tomldisagree.Streaming integrity
Every SSE chunk was decoded with
String::from_utf8_lossyon arrival, but chunks do not respect UTF-8 character boundaries. A multi-byte character (CJK, emoji,—) split across two chunks becameU+FFFD; the damaged payload then failed to parse as JSON and was dropped entirely, deleting text from the middle of a response with no error — and that corrupted/missing text was replayed into the next request as history. Parsing now buffers raw bytes and decodes only complete lines, the same shapeTextDecoderStream+TextLineStreamgives copilot-api.Truncated streams were also delivered as if complete: a transport error mid-stream only did
break, leaving the client with a partial answer, no terminator, and a200in the request log. All five streaming paths now emit a protocol-appropriate terminator and record502. Anthropic streams that end without afinish_reasonare closed withmessage_stopso Claude Code neither blocks nor keeps a partial turn as complete.A
: keepalivecomment is emitted after 15s of silence — injected at event boundaries only, so the verbatim passthrough paths are never spliced mid-event — to survive the ~60s idle timeout that kills long thinking turns.Claude Code / MCP compatibility
context_managementwas silently stripped, disabling Claude Code's context editing. It is forwarded now, with thecontext-management-2025-06-27beta derived automatically. Verified end to end: the upstream response echoes"context_management":{"applied_edits":[]}.anthropic-betaheader was overwritten by the 1M-context flag; flags are merged now.tool_resultcontent arrays were forwarded raw, so every MCP tool returning blocks rather than a bare string was rejected upstream. Text blocks are joined, image blocks becomeimage_urldata URLs.tool_resultdid not setCopilot-Vision-Request.max_tokensis filled from the model catalog; models requiringmax_completion_tokensget one automatic retry.Other
Stopped scraping the AUR for the VS Code version — the Arch maintainers asked projects to stop, and this repo inherited the exact same URL. Now uses Microsoft's own release API.
Cost estimates were computed from the client-supplied model name rather than the model actually served, so the alias
opuspriced at the fallback rate (~38x low); thegpt-4orate was also unreachable behind the broadergpt-4arm.Dashboard pagination could overflow
usizeand panic the handler, and an unboundedper_pageforced a full clone of the request store.Added
GET /health,GET /v1/models/{model}, graceful shutdown on Ctrl-C/SIGTERM,ghc_proxy_uptime_seconds, and amodel=filter on/api/audit.Related issues
None filed in this repo. Defects were cross-referenced against copilot-api issues #95 (tool_result arrays), #217 (context_management), #223 (stream idle timeout), #210 (token undercount), #6 (missing max_tokens), #222 (max_completion_tokens), and #262 (AUR).
Type of change
Checklist
cargo buildsucceedscargo testpasses — 95 tests, 25 of them newcargo clippy --all-targetsis clean (-D warnings)CHANGELOG.md,README.md,docs/api.mdVerification against live upstream
Beyond unit tests, the following were exercised against the real Copilot API:
你好世界🎉こんにちは안녕하세요Ωμέγα) round-trips through both the OpenAI and Anthropic streaming paths with zeroU+FFFDand correct termination.context_management, missingmax_tokens, tool-schema token accounting, and MCP block-arraytool_resultall confirmed working.One caveat worth recording: images inside
tool_resultstill returnimage media type not supported. I isolated this by sending a native OpenAIimage_urlbody straight through with no translation and getting the identical error, so it is an upstream Copilot limitation, not a translation defect — it is not claimed as fixed.