feat: WebSocket transport, real billing figures, consumption-first dashboard, live quota, read timeout

[MartinForReal]

Description

Four commits. The first was the original scope of this PR — live quota, read timeout, SSE buffer, auto_upgrade. The rest came out of trying to answer "what is this costing me" from the dashboard and finding that it could not, for several separate reasons.

---

1. feat: live quota from response headers, upstream read timeout, SSE buffer fixes

Live quota with no extra API call

Copilot attaches per-SKU quota to every response — dumped off a live streaming request to confirm:

x-quota-snapshot-chat:                  ent=-1&ov=0.0&ovPerm=false&rem=100.0&rst=2026-08-01T00%3A00%3A00Z
x-quota-snapshot-completions:           ent=-1&ov=0.0&ovPerm=false&rem=100.0&rst=2026-08-01T00%3A00%3A00Z
x-quota-snapshot-premium_interactions:  ent=-1&ov=0.0&ovPerm=true&rem=100.0&rst=2026-08-01T00%3A00%3A00Z

The same figures summarize_usage extracts from /copilot_internal/user, except these ride along for free. Quota previously required /usage, which costs a separate API call, so /health and /metrics — the endpoints that actually get polled — had none.

Now parsed on every proxied response and surfaced on /health (quota), /metrics (ghc_proxy_quota_* gauges labelled by sku) and /usage (live, alongside the existing authoritative response — not replacing it, since the headers carry no plan name or per-category breakdown).

These headers are undocumented, so the parser fails quiet: a value that stops parsing is ignored and the previous reading left in place. Reporting a confident 0 would read as "quota exhausted", which is worse than reporting nothing.

Upstream read timeout

The stream-interrupted handling added in 1.3.0 only fires when the body stream yields an error. A half-open connection yields no data, no error and no end-of-stream — so the request hangs forever and none of that handling runs.

upstream_read_timeout_seconds bounds the silence between reads rather than total duration, so long streaming answers are unaffected.

The default is 900s, not the 120s originally written. That figure came from reasoning about the ~60s idle window the load balancer enforces. #30 measured the real thing: the upstream buffers input_json_delta until a tool call's argument JSON is complete, then flushes it in one burst — 329.5 seconds of silence for a 35,899-token answer, scaling at ~9.5ms/token. 120s would have aborted perfectly healthy large tool calls. The reasoning and the measured number are in the code comment and asserted in the test (> 330).

SSE line buffer

• Reassembling one large event was quadratic. The buffer rescanned everything retained on every chunk. Measured: a 4 MB event in 4 KB chunks took 7.9 seconds of pure CPU. The search now resumes where the previous one stopped; the whole util test module, including that reassembly, runs in 0.06s.
• Unbounded growth. An upstream that never emits a newline could grow the buffer until the process dies. A single line is capped at 64 MB.

auto_upgrade defaults to true

Including for config files written before the key existed — plain #[serde(default)] would have left those at false, silently missing every existing user. config_version bumped to 3.

Also

• Stopped scraping the Arch User Repository for the latest VS Code version. The AUR maintainers asked projects to stop; this repo had inherited the same URL. Uses Microsoft's own update.code.visualstudio.com release API.
• Removed scripts/__pycache__/replay.cpython-313.pyc from version control.

---

2. fix(gemini): map the generation parameters that have counterparts

gemini_to_openai forwarded four fields out of generationConfig and dropped the rest silently. A client asking for a fixed seed got a non-deterministic answer; one asking for JSON output got prose. Neither with any sign the request had been altered.

Adds the mappings that exist one-for-one: candidateCount → n, seed, presencePenalty, frequencyPenalty, and responseMimeType: "application/json" → response_format, carrying responseSchema across as a JSON schema.

topK and safetySettings stay dropped on purpose — neither has an OpenAI counterpart, and approximating one would be worse than not honouring them. A test asserts they are not invented.

---

3. feat: responses over websocket, real billing figures, and a dashboard built around them

Responses over WebSocket

The model catalogue advertises ws:/responses on ten models and the proxy did not speak it. GitHub publishes no inference API documentation — confirmed by search; docs.github.com covers Copilot seats and billing only — so the protocol was established by probing, and is now written down in docs/api.md:

• Same host as endpoints.api from the token response, path /responses. /v1/responses also upgrades; /ws/responses and /realtime are 404.
• Auth and headers identical to the HTTP path.
• The client sends one flat JSON text frame, {"type":"response.create","model":"...", ...}. model must be top level — nesting the request under "response" is rejected with [ObjectParam] [input] [invalid_type] ... expected a string, but got an object, and omitting type with unsupported message type:.
• The server replies with the same response.* events as SSE, one per frame, each carrying a sequence_number.

GET /v1/responses with an Upgrade header relays to it, recording the exchange like any other request. A model that does not advertise the transport is refused with an error frame naming the endpoint that would work, rather than being proxied into an upstream rejection.

Body capture on a WebSocket reads the debug flag once, when the socket opens, and uses that reading for both halves. Re-reading it at record time — as the HTTP paths can safely do — would let a long-lived socket produce a record with a captured request and no response, or the reverse.

Figures that were wrong

• Output tokens were not comparable across surfaces. The Responses API counts reasoning inside output_tokens; a translated Gemini response reports it as a disjoint reasoning_tokens. Summing them produced a reasoning share of 262% on the dashboard. openai_usage now normalises on field position — nested under completion_tokens_details means already included, flat means additive.
• Cost was guessed from a price list. Copilot states its own per-token rate for every token type it charges, in copilot_usage.token_details, and reports the total as total_nano_aiu (verified against the rates, 5 of 5). That is now the headline figure.
• Cache savings used one hard-coded discount for every model. The entries differ: claude-haiku-4.5 prices cache writes above its input rate, gemini-3.5-flash at zero, gpt-5.5 not at all, and gpt-4o-mini prices nothing because Copilot includes it. The old maths claimed a saving on models that are free. Savings are now computed per request from the rates that request reported.
• Failed attempts counted as consumption. A burst of rejected calls inflated request_count, added an empty-named row to the cache table, and dragged every cache disposition toward "uncached" — a rate computed over requests that consumed nothing. Statistics now cover requests that produced an answer, with failed_requests reported separately. Token and billing totals still count either way: a stream cut off partway consumed what it consumed.
• SSE reassembly covered one and a half protocols. OpenAI delta.tool_calls[] fragments and the entire Responses response.* event family rendered as "(stream produced no content)", and reasoning_text was discarded outright.

Dashboard

Split into Overview, Requests and Metrics over a shared stylesheet, with the landing page ordered by what it costs rather than by what is easy to count. The recent-requests table moved to its own tab, where it can afford the width to be useful.

Each request renders as a conversation, with raw frames and protocol-specific parameters behind tabs, and a summary of what the response actually reported — different per protocol, because the protocols report different things.

Two columns were removed rather than fixed. Probes was measured on one of five code paths, so its — read as "zero probes sent" on the other four; it is now in the Idle tooltip. Session and New are protocol-specific and moved to the detail panel.

The prompt-cache panel states what it does not know. A hit rate of zero is shown as "nothing was cacheable yet" rather than as a measurement. The per-model table carries a Distribution bar, because one aggregate bar cannot say which model stopped matching. Written is hidden entirely on a workload that never writes — only the Anthropic surface with an explicit cache_control marker ever bills a cache write, measured across twelve calls with a 27k-token prefix, exactly one reported one.

Body capture is togglable from the dashboard (POST /api/config/debug), deliberately not persisted: a config file that quietly starts recording request bodies after a restart is a liability.

---

4. perf(anthropic): measure bodies instead of re-serialising them

Both /v1/messages paths serialised whole JSON trees purely to obtain byte counts, inside their four-attempt retry loops:

let req_size = serde_json::to_vec(&current).map(|v| v.len()).unwrap_or(0);

Every other endpoint in the file — including the translated /v1beta/models path — already reads resp.text() once, measures it, and parses from that string. These two were the outliers.

The figures were also wrong. req_size measured the proxy's version of the request, after system prompt injection, the tool-result suffix and the model rename, while every other endpoint records what the client sent — so the dashboard's "Sent" total was adding two definitions together. response_size measured a re-serialisation of the parsed tree, which differs from the wire bytes in whitespace, key order and number formatting.

Verified: a 103-byte request returning 865 bytes now records exactly 103 and 865.

---

Type of change

• Bug fix (non-breaking change that fixes an issue)
• New feature (non-breaking change that adds functionality)
• Breaking change (fix or feature that changes existing behavior)
• Documentation update

Checklist

• cargo build succeeds
• cargo test passes — 149 tests
• cargo clippy --all-targets is clean (-D warnings)
• Documentation updated — CHANGELOG.md, README.md, docs/api.md, docs/configuration.md

Verification against the live upstream

• scripts/protocol_check.py — 72/72 across the four HTTP surfaces.
• scripts/ws_check.py — 85/85 across 8 models, discovering the capable ones from the catalogue and checking both that they work and that the rest are refused cleanly.
• Billing and reasoning figures confirmed on all six recording paths (four protocols × streamed/not, plus WebSocket).
• Cache savings hand-checked against token_details: gpt-5.5 3840 × (500000−50000) = 1728000000; gemini-3.5-flash 4058 × (150000−15000) = 547830000.
• Minimum cacheable prefix measured while investigating empty rows: claude-haiku-4.5 caches a 6902-token prefix but not a 4082-token one.
• /health quota is {} before any request and populates after one streaming call; /metrics emits 9 gauges (3 SKUs × 3 metrics); /usage keeps plan: enterprise and gains live.
• Read timeout: a purpose-built TCP server that sends SSE headers plus half an event then stalls forever produces err.is_timeout() with the timeout set, and hangs indefinitely with it at 0.
• auto_upgrade: verified in an isolated config dir that a legacy file lacking the key runs the update check, that an explicit false suppresses it, and that --config renders the new defaults.

Relationship to #30

This branch was rebased onto #30 after it merged. Three things written locally were dropped because #30's versions are better: my keepalive implementation (a : keepalive SSE comment, which the SSE parser discards and which never resets a client's idle watchdog), stream_openai verbatim passthrough, and the poison wiring for the 64 MB cap. The cap itself stays, which is the part that prevents the OOM.