diff --git a/.gitignore b/.gitignore index 1b59c51..9069a41 100644 --- a/.gitignore +++ b/.gitignore @@ -1,3 +1,7 @@ /target **/*.rs.bk Cargo.lock.orig + +# Python bytecode from scripts/ +__pycache__/ +*.py[cod] diff --git a/CHANGELOG.md b/CHANGELOG.md index aad37cc..ef5b653 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,7 +4,269 @@ All notable changes to this project will be documented in this file. ## [Unreleased] +### Added +- **Live quota with no extra API call.** Copilot attaches per-SKU quota to every + response (`x-quota-snapshot-chat`, `-completions`, `-premium_interactions`), + carrying entitlement, overage, overage-permitted, percent remaining and reset + date. Those are now parsed on every proxied response and surfaced on: + - `GET /health` under `quota` + - `GET /metrics` as `ghc_proxy_quota_percent_remaining`, + `ghc_proxy_quota_entitlement` and `ghc_proxy_quota_overage`, labelled by `sku` + - `GET /usage` under `live`, alongside the existing authoritative response + + Previously quota was only available from `/usage`, which costs a separate + `/copilot_internal/user` request. The headers are undocumented, so a value + that stops parsing is ignored rather than reported as zero, and the previous + reading is left in place. +- **Observability for streams and failures.** Records now carry + `upstream_idle_max_ms` (longest silence between upstream chunks, which is what + assigns blame for a stall), `keepalive_probes`, `failure_kind`, `session_id` + parsed from `metadata.user_id`, and `output_tokens_final` so a count that is + still the `message_start` placeholder renders as `—` instead of being + presented as fact. The dashboard gained the matching columns, a failures-only + filter, and premium-request/cache-hit stats. +- **The Responses API over WebSocket.** Eight models advertise `ws:/responses` + in the catalog — six of them alongside `/responses` and nothing else — and the + proxy had no way to reach it. `GET /v1/responses` with an upgrade now exposes + that transport, relayed to the matching upstream socket. + + The surface is undocumented — GitHub publishes no reference for the inference + API at all, and the token response carries no websocket URL — so the protocol + was established by probing: it lives on the same host as the HTTP API, speaks + the same `response.*` events one per text frame, and takes a *flat* request + frame with `model` at the top level. Nesting the body under `response` is + rejected, as is omitting `type`. `scripts/ws_probe.py` and + `scripts/ws_explore.py` are the tools that established this, kept for when it + changes. + + A model that does not advertise the transport is refused with an error frame + naming the alternative, rather than left waiting on the socket. Refusals and + malformed frames are recorded with a `failure_kind` like every other path, so + a failed attempt leaves something to diagnose from. WebSocket turns appear in + the dashboard under `ws:/responses`, and their transcript renders through the + same reassembly as SSE — only the transport differs. +- `scripts/ws_check.py` sweeps every model the catalog says supports the + transport, discovering the list rather than hard-coding it, and asserts the + event vocabulary, monotonic sequence numbers, termination, text and usage. +- `scripts/replay.py` replays a captured request at the upstream or back through + the proxy and times the SSE stream event by event, with no stall watchdog. +- `scripts/protocol_check.py` asserts protocol conformance against a running + proxy. The property that matters for a translating gateway is not "did it + return 200" but "did it return the shape the *client* asked for", so every + check is an assertion about the client-facing response: a Gemini caller must + receive `candidates`, never the `choices` the upstream actually sent. Covers + all four surfaces in both streaming and non-streaming modes, tool calls and + tool-result round trips, stream terminators (`message_stop`, `[DONE]`, + `response.completed`), token counting, and the error paths — an unknown model + must not arrive as a `200` SSE stream that never produces an event. +- **What Copilot actually billed, per request.** Every response carries + `copilot_usage.total_nano_aiu` — the AI units the turn cost — alongside a + `token_details` breakdown giving the per-token rate for each token type + charged. The total was verified against those rates on five responses. It is + now recorded on every request across all six paths (four protocols × + streamed/not, plus WebSocket), totalled in `/api/stats` as `total_nano_aiu`, + and is the dashboard's headline figure, replacing an estimate derived from + published list prices that could not know a model is included at no charge. + Reasoning tokens are recorded beside it. +- **Prompt-cache statistics** at `GET /api/cache` and on the dashboard: where + input tokens came from (served from cache / written to cache / fresh), the + disposition of recent requests, and a per-model hit rate with its own + distribution bar. A single global number cannot tell you *which* conversation + stopped matching its prefix, which is the failure that quietly multiplies the + bill. What the cache was worth is reported as `saved_nano_aiu`, net of the + write premium, because caching is not free — see below for where the rates + come from. +- **Body capture can be toggled at runtime**, from the request browser or via + `POST /api/config/debug`. It previously required restarting the proxy with + `--debug`, by which point the request you wanted to inspect was gone; the flag + is read live, so it applies from the next call. It is not written back to + `config.yaml`, since capture puts prompts and any credentials they carry into + memory and the log and should lapse on restart. `GET /health` now reports it. + The `/api/config/` routes are guarded by the API key when one is configured — + read-only dashboard endpoints stay open, but turning on body capture must not + be something an unauthenticated caller can do. +- `upstream_read_timeout_seconds` (default `900`, `0` disables, env + `GHC_PROXY_UPSTREAM_READ_TIMEOUT`). Bounds the silence *between* reads from an + upstream response rather than the total duration, so long streaming answers + are unaffected. Without it a half-open connection yields no data, no error and + no end-of-stream: the request hangs forever and the stream-interrupted + handling never runs. The default clears the longest silence measured against + the real upstream (329.5s, while a tool call's argument JSON was buffered) with + room to spare. + ### Changed +- **The cache panel is per model, and says why a cell is empty.** The panel + opened with one aggregate stacked bar, which cannot say *which* model stopped + matching — the question a collapsing hit rate raises. The bar is now a + Distribution column, one per model, and what remains at the top is the colour + key those bars need. Below the table, a footnote separates the two reasons a + row comes back empty: a prompt too short to be eligible at all, and one long + enough but never sent before, since nothing can be read back on first sight. + + **`Written` only appears when something was written.** Only the Anthropic + surface with an explicit `cache_control` marker ever bills a cache write. + Every other surface caches implicitly: the first call reads nothing, the next + reads the whole prefix back, and no write is billed. Measured across twelve + calls with a 27k-token prefix, exactly one reported a write. The column and + its swatch are hidden outright on a workload that never writes, rather than + showing a wall of zeros for an event that cannot happen. + + Measured while investigating: `claude-haiku-4.5` cached a 6902-token prefix + but not a 4082-token one, so a prompt in the low thousands legitimately + produces an empty row. Every surface — chat, messages, responses, Gemini and + WebSocket, streamed and not — was confirmed to carry + `copilot_usage.token_details`, so an empty row is never the proxy failing to + read what upstream reported. +- **Cache savings come from the model's own rates, not a price list.** The + per-model saving was `list price × (1 − 0.1)` for reads and + `× (1.25 − 1)` for writes — one hard-coded discount applied to every model. + Copilot states its actual per-token rate for every token type it charges, in + `copilot_usage.token_details`, and the entries are not the same on every + model: `claude-haiku-4.5` prices cache writes above its input rate, + `gemini-3.5-flash` prices them at zero, `gpt-5.5` does not price them at all, + and `gpt-4o-mini` prices nothing because Copilot includes it. The old maths + claimed a saving on models that are free. + + `/api/cache` now reports `saved_nano_aiu` in the same AI units the rest of + the dashboard bills in, computed per request from the rates that request + reported, and drops `saved_usd` / `write_premium_usd` / `net_saved_usd`. + + A `null` `saved_nano_aiu` now means no response reported rates to compute one + from, distinct from `0`, which is a real figure: nothing was cached, so + nothing was saved. +- **Failed attempts no longer count as consumption.** `request_count` counted + every record, so a burst of rejected calls inflated the request total, added + an empty-named row to the per-model cache table, and dragged every cache + disposition toward "uncached" — a rate computed over requests that consumed + nothing. Statistics now cover requests that produced an answer, `/api/stats` + reports `failed_requests` separately, and the overview links to them. Token + and billing totals still count either way: a stream cut off partway consumed + what it consumed, and hiding that would understate the bill. + + The predicate — non-2xx, or a `failure_kind` on an otherwise successful + status — now has one definition shared by the statistics, the failures-only + filter and the dashboard, instead of three. +- **Dashboard restructured around consumption.** The landing page opened with + eight identically-weighted stat cards followed by the full 78-row model + catalogue, so the number that tracks spend had no more prominence than + `bytes_received`, and reference data dominated a page you open to check usage. + It now leads with what Copilot billed in AI units, input tokens (with the + cache-hit share) and output tokens (with the reasoning share); quota bars per + SKU sit directly beneath, since that is the same quantity seen from the other + end. Traffic, proxy health and the prompt-cache panel follow, with the model + catalogue folded away and the request list on its own tab. + - The three pages share one stylesheet at `/app.css` instead of three drifting + inline copies, and carry the same persistent nav with a live readiness and + version indicator — previously each page offered only a lone + `← Dashboard` link. + - **Wide screens are used.** The old 1180px cap left a 2560px monitor half + empty while the twelve-column request table still scrolled sideways. Width + is now per-page: the request table gets 2000px and the metric list 1600px, + since both are data a wide screen genuinely helps you read, while the + overview keeps a 1280px measure because a handful of large numbers only + drift apart when stretched. Message text keeps a 100ch measure so prose + stays readable at full width; tool payloads are code and stay unconstrained. + - `/health` data (per-SKU quota, Copilot token expiry, uptime, readiness, + models loaded) was reachable but shown nowhere; it is now on the overview. + The request list stays on its own tab — the overview only reports how many + requests failed and links across. + - **Debug bodies render as a conversation.** `--debug` stores request and + response bodies verbatim, which is the right thing to store and the wrong + thing to read: finding what the model actually said meant scrolling past + tool schemas, content-filter blocks and, for streams, every SSE frame. The + detail view now reconstructs the exchange — system/user/assistant turns, + tool calls and results as collapsible cards, token and stop-reason chips — + and reassembles streamed fragments into the message they describe. All + three wire formats are handled, and none of them agree: Anthropic numbers + content blocks and tags every delta, chat completions hides tool arguments + inside `choices[].delta.tool_calls[]`, and the Responses API uses a flat + `response.*` vocabulary where the delta is a bare string. Reasoning is + picked up under every name the families use (`reasoning_text`, + `reasoning_content`, `response.reasoning_summary_text.delta`) rather than + dropped, and a turn that produced no visible output says whether it ran out + of token budget instead of reporting an empty stream. The raw bodies remain + one click away on a `Raw` tab. + - A third tab shows what the wire actually carried, which is the question when + a stream stalls, repeats, or ends somewhere unexpected. For a stream it + lists **every SSE frame** — sequence number, event name and a one-line gist, + each expandable to the full payload — so the `[DONE]` sentinel, a missing + `message_stop` or a duplicated index is visible instead of buried in a + scroll of raw text. For a non-streaming completion it lists **every + top-level field**, including the ones the conversation view has no place for + (`content_filter_results`, `prompt_filter_results`, `system_fingerprint`, + `service_tier`, `copilot_usage`). + - The request table's full nanosecond ISO timestamp consumed a third of the + row width; it renders as local wall-clock time with the exact value in the + tooltip, and the table scrolls horizontally rather than letting the panel + clip columns off the right edge, with the expanded detail pinned to the left + so it stays readable while the table scrolls. Empty results say so instead + of showing a header row with nothing under it. + - **Each request now says what happened, not just what was said.** The + response section opened with a row of bare chips: the numbers were there, + but not their meaning. It now leads with the outcome, normalized across + surfaces and explained — `length`, `max_tokens`, `max_output_tokens` and + `MAX_TOKENS` are the same event on four different APIs, and all four mean + the answer on screen is not the whole answer. Endings are colour-coded by + whether they are normal (`end_turn`, `stop`, `completed`), truncated, or a + refusal or filter block. + + Beneath it sit the facts each surface actually reports, omitted when + absent: reasoning effort, service tier, prompt-cache retention, response + verbosity and output item types on the Responses API; the backend build + fingerprint on chat completions, which is worth noticing because a change + there can shift results for identical input; cache-write TTL split and + inference region on Anthropic; rejected speculative tokens, which are + billed. Content-filter verdicts appear only when something was actually + flagged — an all-clear on four categories on every request is noise. + - The `Probes` column is gone. Keepalive only runs on the Anthropic streaming + path, so four of the five request paths rendered a dash that read as "zero + probes sent" when it meant "not measured here". The count now rides in the + `Idle` tooltip, which is where it was useful anyway — the pair is what + assigns blame for a stall. + - **Protocol-specific parameters moved out of the list and into the detail.** + The three surfaces agree on very little, so a table column for any of them + is structurally empty for the other two: `New` (cache writes) only exists on + Anthropic, and `Session` only when the client encodes one into + `metadata.user_id` the way Claude Code does. Both columns are gone; the + detail now carries a panel naming the surface and listing what that request + actually set — `top_k`, `stop_sequences`, `cache_control` marks and + `thinking` for Anthropic; `seed`, penalties, `response_format` and + `logprobs` for chat completions; `reasoning.effort`, `max_output_tokens`, + `store` and `truncation` for the Responses API; `generationConfig` and + `safetySettings` for Gemini. Absent parameters are omitted rather than shown + as blanks. A Gemini request is labelled `Gemini → OpenAI Chat Completions + (translated)`, because the proxy rewrites it before forwarding and the + recorded body is the rewritten one — `topK` and `safetySettings` have no + counterpart and are dropped, which the panel says instead of leaving the + reader to assume they took effect. Gemini's `contents`/`parts`, + `functionCall` and `functionResponse` shapes render like every other + surface's. +- **`auto_upgrade` now defaults to `true`**, including for config files written + before the setting existed. The proxy checks GitHub releases on startup and + replaces its own binary when a newer version is published; the replacement + takes effect on the next start. Disable with `auto_upgrade: false`, + `--no-auto-upgrade`, or `GHC_PROXY_AUTO_UPGRADE=0` — worth doing when the + binary is managed by a package manager, or lives in a build output directory + that `cargo build`/`cargo clean` also writes to. +- **Default model mappings point at Opus 5 and Sonnet 5.** The catalog now + carries `claude-opus-5` and `claude-sonnet-5`; both are generally available and + match `claude-opus-4.8` on every published capability — 1M context, 64k + output, billing multiplier 1, vision. `claude-opus-5` is the new built-in + target, and the table gained the `opus5` / `5[1m]` aliases along with the + `claude-opus-5*`, `claude-sonnet-5*` and `claude-sonnet-4.6` prefixes. + + **Existing config files are only added to, never rewritten.** A mapping + already on disk keeps pointing where it was told to, even when it holds what + used to be the built-in default. A value equal to an old default is + indistinguishable from a version pinned on purpose, and pinning is common — + a config in the wild carried `claude-opus-4-7: claude-opus-4.7` beside + `haiku: claude-opus-4.7`, both of which a "lift stale defaults" pass would + silently retarget. Run `--setup`, or `--default`, to adopt the new defaults. +- `config_version` bumped to `4`, so existing `config.yaml` files gain the new + aliases on the next start. +- `config_version` bumped to `3`, so existing `config.yaml` files gain + `upstream_read_timeout_seconds` and the new `auto_upgrade` default on the next + start. - **Config schema upgrades apply automatically.** When a release introduces new `config.yaml` properties (signalled by a `config_version` bump), the missing keys are now filled with their defaults and written back to `config.yaml` on @@ -13,6 +275,83 @@ All notable changes to this project will be documented in this file. Opus 4.8 alias backfill is now scoped to pre-v2 files so it cannot overwrite customized mappings in current ones +### Fixed +- **Recorded request and response sizes on `/v1/messages` measured the wrong + bytes, and cost a full serialisation each to get.** Both paths built a byte + count by serialising the whole JSON tree — + `serde_json::to_vec(¤t).map(|v| v.len())` — inside their four-attempt + retry loops, and the response was serialised twice more: once for a debug log + that discards it unless logging is on, once to measure it. + + The numbers were also not the ones every other endpoint records. + `request_size` measured the proxy's version of the request, after system + prompt injection, the tool-result suffix and the model rename, so the + dashboard's "Sent" total was adding two definitions together. `response_size` + measured a re-serialisation of the parsed tree, which differs from the wire + bytes in whitespace, key order and number formatting. + + Both now follow the pattern the rest of the file already used: the client's + `body.len()` taken once, and the response read as text once, then measured, + logged and parsed from that single copy. A 103-byte request returning 865 + bytes now records exactly 103 and 865. +- **Output token totals were not comparable across surfaces.** The two + conventions for reasoning tokens disagree about whether they are already + counted: a Responses turn reports `input 11 + output 17 == total 28` with 10 + of that output being reasoning, while the translated Gemini surface reports + `prompt 5 + completion 1 + reasoning 97 == total 103`. Taken at face value the + dashboard showed a reasoning share of 262%. `output_tokens` is now normalized + to the true total on both, the way `input_tokens` already was. +- **Gemini requests lost parameters that had exact counterparts.** The + translation to chat completions carried `temperature`, `topP`, + `maxOutputTokens` and `stopSequences` and dropped the rest of + `generationConfig` in silence, so a client's `seed`, `candidateCount`, + `presencePenalty`, `frequencyPenalty` or `responseMimeType` had no effect and + nothing said why. They now map to `seed`, `n`, `presence_penalty`, + `frequency_penalty` and `response_format` (with `responseSchema` carried over + as a `json_schema` format). `topK` and `safetySettings` genuinely have no + counterpart in the chat completions schema and are still dropped — the + dashboard names them rather than describing the loss in general terms, and a + test asserts they are not invented. +- **Non-2xx upstreams on `/v1/messages` reached the client as an empty `200` + stream.** `messages_direct` only intercepted `400`; a `401`, `403`, `429` or + `5xx` fell through and was wrapped in a `200 text/event-stream` whose body was + a JSON error object, so the client waited on a stream that never produced an + event and reported a stall instead of the auth or rate-limit failure that + actually happened. This was the one path Claude Code takes. All five streaming + paths now gate on a single `is_streamable_status()` predicate +- **A client that disconnected mid-stream left no record.** axum drops the + response body on disconnect, which drops the generator, so the `store.add` + after the loop never ran. Recording now happens from `Drop` +- Pre-flight failures (token refresh, rate gate, connect errors) returned early + without recording; they are captured with a `failure_kind` +- **Keepalive probes could be silenced exactly during a stall.** The boundary + flag was set from the last chunk, so a TCP split mid-event left it stuck until + a new chunk arrived — and an upstream that went quiet right then produced no + probes at all. Partial events are now held back so the downstream always sits + on an event boundary, and the Anthropic path sends `event: ping` rather than + an SSE comment, since comments are discarded by the parser and never reset a + client's idle watchdog +- **Token counts were read from one bucket.** Anthropic's `input_tokens`, + `cache_read_input_tokens` and `cache_creation_input_tokens` are disjoint, so a + fully-cached Claude Code turn reported single digits for a 348,483-token + prompt. The three protocols slice the total differently and now have separate + extractors; cost reprices the cached buckets instead of charging every input + token at the full rate +- **Reassembling one large SSE event was quadratic.** The line buffer rescanned + the whole retained buffer on every chunk, so a single event that arrived in + many pieces was re-read from the start each time. A 4 MB event delivered in + 4 KB chunks took **7.9 seconds** of pure CPU; the search now resumes where the + previous one stopped, bringing it to milliseconds. Correctness was never + affected — only the cost of getting there. +- The line buffer could grow without bound if an upstream never emitted a + newline. A single line is now capped at 64 MB. +- Stopped scraping the Arch User Repository for the latest VS Code version. The + AUR maintainers asked projects to stop; `dynamic_vscode_version` now uses + Microsoft's own `update.code.visualstudio.com` release API and ignores + non-`major.minor.patch` builds. +- Removed `scripts/__pycache__` from version control and added the matching + `.gitignore` entries. + ## [1.3.0] - 2026-07-27 ### Added diff --git a/Cargo.lock b/Cargo.lock index 5b59bdb..9775012 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -104,6 +104,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "31b698c5f9a010f6573133b09e0de5408834d0c82f8d7475a89fc1867a71cd90" dependencies = [ "axum-core", + "base64", "bytes", "form_urlencoded", "futures-util", @@ -122,8 +123,10 @@ dependencies = [ "serde_json", "serde_path_to_error", "serde_urlencoded", + "sha1", "sync_wrapper", "tokio", + "tokio-tungstenite 0.29.0", "tower", "tower-layer", "tower-service", @@ -394,6 +397,12 @@ dependencies = [ "syn 2.0.118", ] +[[package]] +name = "data-encoding" +version = "2.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a4ae5f15dda3c708c0ade84bfee31ccab44a3da4f88015ed22f63732abe300c8" + [[package]] name = "der" version = "0.7.10" @@ -759,6 +768,7 @@ dependencies = [ "serde_norway", "tiktoken-rs", "tokio", + "tokio-tungstenite 0.28.0", "toml", "tower", "tower-http 0.7.0", @@ -1905,6 +1915,17 @@ dependencies = [ "serde", ] +[[package]] +name = "sha1" +version = "0.10.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a978451301f4db1d02937a4ab3ccce137717b81826e79b7d49ffe3244a13c3b8" +dependencies = [ + "cfg-if", + "cpufeatures", + "digest", +] + [[package]] name = "sha2" version = "0.10.9" @@ -2237,6 +2258,34 @@ dependencies = [ "tokio", ] +[[package]] +name = "tokio-tungstenite" +version = "0.28.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d25a406cddcc431a75d3d9afc6a7c0f7428d4891dd973e4d54c56b46127bf857" +dependencies = [ + "futures-util", + "log", + "rustls", + "rustls-native-certs", + "rustls-pki-types", + "tokio", + "tokio-rustls", + "tungstenite 0.28.0", +] + +[[package]] +name = "tokio-tungstenite" +version = "0.29.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8f72a05e828585856dacd553fba484c242c46e391fb0e58917c942ee9202915c" +dependencies = [ + "futures-util", + "log", + "tokio", + "tungstenite 0.29.0", +] + [[package]] name = "tokio-util" version = "0.7.18" @@ -2428,6 +2477,41 @@ version = "0.2.5" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "e421abadd41a4225275504ea4d6566923418b7f05506fbc9c0fe86ba7396114b" +[[package]] +name = "tungstenite" +version = "0.28.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8628dcc84e5a09eb3d8423d6cb682965dea9133204e8fb3efee74c2a0c259442" +dependencies = [ + "bytes", + "data-encoding", + "http", + "httparse", + "log", + "rand", + "rustls", + "rustls-pki-types", + "sha1", + "thiserror", + "utf-8", +] + +[[package]] +name = "tungstenite" +version = "0.29.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6c01152af293afb9c7c2a57e4b559c5620b421f6d133261c60dd2d0cdb38e6b8" +dependencies = [ + "bytes", + "data-encoding", + "http", + "httparse", + "log", + "rand", + "sha1", + "thiserror", +] + [[package]] name = "typenum" version = "1.20.1" @@ -2522,6 +2606,12 @@ version = "2.1.3" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "daf8dba3b7eb870caf1ddeed7bc9d2a049f3cfdfae7cb521b087cc33ae4c49da" +[[package]] +name = "utf-8" +version = "0.7.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "09cc8ee72d2a9becf2f2febe0205bbed8fc6615b7cb429ad062dc7b7ddd036a9" + [[package]] name = "utf8-zero" version = "0.8.1" diff --git a/Cargo.toml b/Cargo.toml index a3ece90..530e6d6 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -16,8 +16,11 @@ name = "ghc-proxy" path = "src/main.rs" [dependencies] -axum = { version = "0.8", features = ["json"] } +axum = { version = "0.8", features = ["json", "ws"] } tokio = { version = "1", features = ["full"] } +# Upstream transport for the catalog's `ws:/responses` surface. rustls with the +# platform trust store, to match reqwest and keep enterprise CAs working. +tokio-tungstenite = { version = "0.28", default-features = false, features = ["connect", "rustls-tls-native-roots"] } tower = { version = "0.5", features = ["util"] } tower-http = { version = "0.7", features = ["fs"] } reqwest = { version = "0.13", default-features = false, features = ["json", "rustls", "stream", "form"] } @@ -38,3 +41,6 @@ self_update = { version = "0.44", default-features = false, features = ["reqwest toml = "1.1" [dev-dependencies] +# `test-util` (not part of tokio's `full`) enables the paused clock used to +# exercise the SSE keepalive timing deterministically. +tokio = { version = "1", features = ["full", "test-util"] } diff --git a/README.md b/README.md index 6ba6341..720fa97 100644 --- a/README.md +++ b/README.md @@ -42,7 +42,12 @@ file. upstream model supports it, otherwise translated through chat completions). - **Gemini-compatible** `/v1beta/models/{model}:generateContent`, `:streamGenerateContent`, and `:countTokens` endpoints (translated through chat - completions). + completions). `generationConfig` parameters with an OpenAI counterpart are + carried across — `temperature`, `topP`, `maxOutputTokens`, `stopSequences`, + `candidateCount`, `seed`, the penalties, and `responseMimeType`/`responseSchema` + as `response_format`. `topK` and `safetySettings` have no counterpart and are + dropped, which the dashboard states rather than leaving you to assume they + took effect. - **GitHub Models inference** — requests whose model id uses the `publisher/model` form (e.g. `openai/gpt-4o`) are transparently routed to the [GitHub Models](https://models.github.ai) API instead of Copilot, authenticated @@ -59,7 +64,23 @@ file. - **Copilot token management** with automatic refresh. - **Orphaned `tool_use_id` recovery** — retries with offending tool results stripped when the upstream returns the corresponding 400 error. -- **Request analytics dashboard** at `/` and a request browser at `/requests`. +- **The Responses API over WebSocket.** Ten models advertise a `ws:/responses` + transport in the Copilot catalog. `GET /v1/responses` with an `Upgrade: + websocket` header relays to it, recorded like any other request. A model that + does not advertise the transport is refused with an error frame naming the + endpoint that does work. +- **Request analytics dashboard** across three pages: an overview at `/` led by + what Copilot actually billed (in AI units, read from `copilot_usage`, not + estimated from a price list), a request browser at `/requests` where each + exchange renders as a conversation with the raw frames a tab away, and a + metrics UI at `/metrics/dashboard`. Prompt-cache statistics per model at + `GET /api/cache` show where input tokens came from and what the cache was + worth. Failed attempts are counted separately rather than diluting the + statistics. +- **Body capture toggled at runtime** from the dashboard or + `POST /api/config/debug`, so you can start recording request and response + bodies without a restart. Not persisted to `config.yaml`, since capture puts + prompts into memory and the log and should lapse on restart. - **Interactive setup wizard** (`--setup`, or first launch in a terminal): GitHub sign-in, live model catalog, and model-mapping configuration. - **1M-context support** — forwards the `anthropic-beta: context-1m-2025-08-07` @@ -91,6 +112,7 @@ ghc-proxy [options] --fetch-version Fetch the latest VS Code version at startup --no-fetch-version Disable dynamic VS Code version fetching --auto-upgrade Auto-upgrade app when a newer release is available + (default: on) --no-auto-upgrade Disable app auto-upgrade --update-config Persist non-schema config write-backs (schema upgrades apply automatically) -v, --version Show version @@ -157,22 +179,22 @@ Config file: `~/.ghc-tunnel/config.yaml` (`%APPDATA%/ghc-tunnel/config.yaml` on Windows). It is generated on first run or with `--config`. ```yaml -config_version: 2 +config_version: 4 address: 127.0.0.1 port: 8314 debug: false account_type: individual # individual | business | enterprise -vscode_version: "1.123.0" +vscode_version: "1.130.0" api_version: "2025-05-01" copilot_version: "0.48.1" -auto_upgrade: false +auto_upgrade: true # self-update on startup; false to disable model_mappings: exact: - opus: claude-opus-4.8 - sonnet: claude-opus-4.8 + opus: claude-opus-5 + sonnet: claude-opus-5 haiku: claude-haiku-4.5 prefix: - claude-sonnet-4-: claude-opus-4.8 + claude-sonnet-4-: claude-opus-5 github_models: enabled: true # route publisher/model ids to GitHub Models # org: my-org # attribute inference to an organization @@ -181,6 +203,7 @@ system_prompt_remove: [] system_prompt_add: [] tool_result_suffix_remove: [] max_connection_retries: 3 +upstream_read_timeout_seconds: 900 # max silence from upstream; 0 disables # Optional: require this key on all LLM endpoints (Bearer / x-api-key / # x-goog-api-key). Omit or leave empty to disable authentication. @@ -239,6 +262,7 @@ the dashboard and model listings. |----------|-------------| | `POST /v1/chat/completions` | OpenAI chat completions | | `POST /v1/responses` | OpenAI responses API (Codex) | +| `GET /v1/responses` *(with `Upgrade: websocket`)* | Responses API over WebSocket, for models advertising `ws:/responses` | | `GET /v1/models` | List available models | | `GET /v1/models/{model}` | Retrieve a single model (aliases resolved) | | `POST /v1/messages` | Anthropic messages API | @@ -248,13 +272,20 @@ the dashboard and model listings. | `POST /v1beta/models/{model}:countTokens` | Gemini token counting | | `GET /health` | Liveness/readiness probe (`?strict=true` for 503 when not ready) | | `GET /openapi.json` | OpenAPI v3 specification | -| `GET /` | Web dashboard | +| `GET /` | Web dashboard — overview | | `GET /metrics/dashboard` | Metrics dashboard UI | | `GET /metrics` | OpenMetrics endpoint | | `GET /requests` | Request browser | +| `GET /app.css` | Stylesheet shared by the three dashboard pages | | `POST /api/config/reload` | Reload config.yaml without restart | +| `POST /api/config/debug` | Turn body capture on or off (`{"debug": true}`) | +| `GET /api/stats` | Running totals, including what Copilot billed | +| `GET /api/cache` | Prompt-cache statistics, overall and per model | | `GET /api/models` | All supported models (used by the dashboard) | +The `/api/config/` routes are guarded by the API key when one is configured; +the read-only dashboard endpoints stay open. + ## Example Usage ### OpenAI SDK diff --git a/docs/api.md b/docs/api.md index b89baea..201fce1 100644 --- a/docs/api.md +++ b/docs/api.md @@ -32,16 +32,137 @@ require a key on the LLM endpoints; see [Authentication](#authentication) below. | `GET /v1/models/full/` | Raw upstream model catalog with capabilities | | `GET /usage` | Copilot plan and quota usage | | `GET /health` | Liveness/readiness probe | -| `GET /` | Web analytics dashboard | +| `GET /` | Web analytics dashboard — overview | | `GET /metrics/dashboard` | Metrics dashboard UI | | `GET /metrics` | OpenMetrics exposition endpoint | | `GET /requests` | Request browser | +| `GET /app.css` | Stylesheet shared by the three dashboard pages | | `GET /api/stats` | Dashboard statistics (JSON) | +| `GET /api/cache` | Prompt-cache statistics, overall and per model | | `GET /api/requests` | Recent requests (JSON) | | `GET /api/audit` | Filtered audit records | | `GET /api/audit/summary` | Aggregated audit summary | | `POST /api/config/reload` | Reload `config.yaml` without restart | +| `POST /api/config/debug` | Turn request/response body capture on or off | | `GET /openapi.json` | OpenAPI v3 specification of the LLM endpoints | +| `GET /v1/responses` (Upgrade) | Responses API over WebSocket | + +## Responses over WebSocket + +Several models advertise `ws:/responses` in `supported_endpoints` alongside +`/responses`. `GET /v1/responses` (or `/responses`) with a WebSocket upgrade +exposes that transport. + +The protocol is the streaming Responses API with a different carrier: the same +`response.*` event vocabulary, one event per text frame, so a client already +written against the SSE stream needs no new parsing. Send one frame to start a +turn: + +```json +{"type": "response.create", "model": "gpt-5.5", "input": "...", "stream": true} +``` + +The frame is flat — `model` sits at the top level beside the request fields, not +nested under a `response` object. Omitting `type` or nesting the body is +rejected. + +A model that does not advertise the transport is refused with an error frame +naming the alternative rather than left waiting on the socket: + +```json +{"type": "error", "error": {"code": "unsupported_api_for_model", + "message": "Model 'claude-opus-4.6' does not support ws:/responses. Use POST /v1/responses instead."}} +``` + +WebSocket turns are recorded like any other request, under the endpoint +`ws:/responses`. + +Read-only dashboard endpoints are reachable without an API key so local +monitoring keeps working. The `/api/config/` routes are not: they mutate the +running process, and one of them turns on body capture, which writes whatever +the client sent — credentials included — into the request log. + +## Statistics and failed attempts + +`GET /api/stats` counts requests that produced an answer. Attempts that did not +— a non-2xx status, or a `failure_kind` on an otherwise successful one — are +reported separately as `failed_requests` and excluded from the rest, so a burst +of rejected calls cannot dilute a rate computed over requests that consumed +nothing. Token and billing totals count either way: a stream cut off partway +consumed what it consumed. + +## Prompt cache statistics + +`GET /api/cache` reports where input tokens came from. The hit rate is the early +warning for a broken prompt prefix: on an agent workload it should sit high and +stable, and a sudden drop means the prompt stopped matching and every turn is +paying full input price again. + +```json +{ + "totals": { + "input_tokens": 9397, + "cache_read_tokens": 4682, + "cache_creation_tokens": 4682, + "fresh_tokens": 33, + "hit_rate": 0.498, + "request_count": 4 + }, + "dispositions": { "served_from_cache": 1, "wrote_to_cache": 1, "no_cache": 2 }, + "sampled_requests": 4, + "by_model": [ + { + "model": "gemini-3.5-flash", + "requests": 4, + "input_tokens": 9397, + "cache_read_tokens": 4682, + "cache_creation_tokens": 0, + "fresh_tokens": 33, + "hit_rate": 0.498, + "saved_nano_aiu": 547830000 + } + ] +} +``` + +`totals` are the all-time running counters. `by_model` is derived from the +retained ring buffer, so it describes the most recent `sampled_requests` calls +rather than every one ever served. + +### What a zero means + +Copilot states its own per-token rates on each response, in +`copilot_usage.token_details`. `saved_nano_aiu` is computed from those rates +rather than from a price list, so a model Copilot includes at no charge +contributes nothing instead of an imagined discount. Positive means the cache +paid for itself; negative is the normal shape of a turn that populated a cache +it has not read back yet. `null` means no response for this model reported the +rates to compute one from — distinct from `0`, which is a real figure: nothing +was cached, so nothing was saved. + +`cache_creation_tokens` is only ever non-zero on the Anthropic surface with an +explicit `cache_control` marker. Every other surface caches implicitly: the +first call reads nothing, the next reads the whole prefix back, and no write is +billed. Measured across twelve calls with a 27k-token prefix, exactly one +reported a write, on `/v1/messages`. The dashboard hides the column outright +when nothing wrote. + +A model with no cache activity at all is usually not a fault either: Copilot +needs a minimum cacheable prefix before any of the prompt is eligible. +`claude-haiku-4.5` was observed caching a 6902-token prefix but not a +4082-token one. + +## Body capture + +`POST /api/config/debug` with `{"debug": true}` or `{"debug": false}` turns +request/response body capture on or off for the running process. The flag is +read live on every request, so it applies from the next call — no restart, and +no need to have predicted in advance that you would want the bodies. + +It is deliberately not written back to `config.yaml`. Capture puts prompts, tool +output and any credentials they carry into memory and the log, so it lapses on +restart rather than staying on because someone forgot. `GET /health` reports the +current value as `debug`. Streaming (SSE) is supported on the chat, responses, and messages endpoints by setting `"stream": true` in the request body. The Gemini surface streams via the @@ -75,6 +196,12 @@ has loaded. A degraded proxy still answers `200` with `ready: false` so probes can distinguish "process alive" from "able to serve traffic". Add `?strict=true` to get `503 Service Unavailable` instead when the proxy is not ready. +The `quota` object holds the most recent per-SKU allowance reported by the +upstream. Copilot attaches it to every response, so it is current without any +extra API call — but it stays empty until the first request has been proxied. +The same figures are exported on `/metrics` as `ghc_proxy_quota_*` gauges +labelled by `sku`. + ## Retrieve a model `GET /v1/models/{model}` returns a single catalog entry in the OpenAI shape, @@ -84,7 +211,7 @@ the mapped Copilot model. Unknown ids return `404` with an OpenAI-style error body. ```bash -curl http://127.0.0.1:8314/v1/models/claude-opus-4.8 +curl http://127.0.0.1:8314/v1/models/claude-opus-5 ``` ## Token counting @@ -239,7 +366,14 @@ counts, estimated cost, and prompt-cache hit rate. - **SSE keepalive** — a `: keepalive` comment is emitted after 15 seconds of silence so extended thinking does not trip the ~60 second idle timeout enforced by the upstream load balancer. Comments are ignored by every - spec-compliant SSE client. + spec-compliant SSE client. The comment is only injected at an event boundary: + if the upstream goes quiet *part way through* writing an event, splicing one + in would corrupt it, so that case is left to the read timeout below. +- **Read timeout** — `upstream_read_timeout_seconds` (default 120) bounds how + long an upstream response may stay silent between reads. It does not cap the + total duration, so long answers stream normally, but a half-open connection + is turned into a reported stream error instead of hanging the request + forever. Set it to `0` to disable. - **`anthropic-beta` passthrough** — the client's beta flags are forwarded and merged with the ones the proxy derives (`context-1m-2025-08-07` for extended context models, `context-management-2025-06-27` when the request uses diff --git a/docs/claude-code.md b/docs/claude-code.md index 2a074a9..0bcb140 100644 --- a/docs/claude-code.md +++ b/docs/claude-code.md @@ -50,17 +50,23 @@ export ANTHROPIC_API_KEY="ghc-proxy" Claude Code sends specific model names (for example `claude-opus-4-7[1m]`). Use [model mappings](configuration.md#model-mappings) to route those to whichever -Copilot model you want. For example, to always use Claude Opus 4.8 with its +Copilot model you want. For example, to always use Claude Opus 5 with its native 1M context: ```yaml model_mappings: exact: - claude-opus-4-8: claude-opus-4.8 + opus: claude-opus-5 + "5[1m]": claude-opus-5 prefix: - claude-opus-4-7: claude-opus-4.8 + claude-opus-4-7: claude-opus-5 + claude-opus-4-8: claude-opus-5 ``` +The built-in defaults already do this for every Claude spelling. They apply to +a *new* config file only — an existing one keeps the targets it has, so if you +wrote yours before Opus 5 shipped, either edit it or re-run `--setup`. + Restart the proxy after editing `config.yaml` — mappings are read at startup. ## Codex CLI @@ -109,7 +115,18 @@ at `/v1beta/models/{model}:generateContent` (plus streaming and token counting). ## Tips - Use `GET /usage` (or `ghc-proxy check-usage`) to monitor your Copilot quota. -- The dashboard at `http://127.0.0.1:8314/` shows live request statistics and - the full list of supported models. +- The dashboard has three pages: the overview at `http://127.0.0.1:8314/` leads + with what Copilot billed for the session in AI units, alongside quota, traffic + and prompt-cache statistics; `/requests` browses each exchange as a + conversation; `/metrics/dashboard` shows the raw metric list. +- Prompt caching is where an agent session gets cheap or expensive. The cache + panel breaks the hit rate down per model, because a single global number + cannot tell you *which* conversation stopped matching its prefix. Copilot + needs a minimum cacheable prefix — around 4k tokens — before any of it is + eligible, so short prompts legitimately show nothing. +- To inspect what a tool is actually sending, turn on body capture from the + overview or with `curl -X POST http://127.0.0.1:8314/api/config/debug -d + '{"debug": true}'`. It takes effect on the next request and lapses on + restart. - If a tool reports a model is unavailable, check `GET /v1/models` for the exact model id and add a mapping. diff --git a/docs/configuration.md b/docs/configuration.md index b92fa38..f038072 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -21,7 +21,7 @@ Windows). Generated on first run, with `--config`, or through the setup wizard. ```yaml # Schema version for migration/write-back behavior -config_version: 2 +config_version: 4 # Server settings address: 127.0.0.1 @@ -32,21 +32,21 @@ debug: false account_type: individual # Header version strings (mimic the VS Code Copilot Chat client) -vscode_version: "1.123.0" +vscode_version: "1.130.0" api_version: "2025-05-01" copilot_version: "0.48.1" -# Self-update behavior -auto_upgrade: false +# Self-update behavior (enabled by default) +auto_upgrade: true # Model name mappings: exact (full match) and prefix (starts-with) model_mappings: exact: - opus: claude-opus-4.8 - sonnet: claude-sonnet-4.6 + opus: claude-opus-5 + sonnet: claude-sonnet-5 haiku: claude-haiku-4.5 prefix: - claude-sonnet-4-: claude-opus-4.8 + claude-sonnet-4-: claude-opus-5 # GitHub Models (https://models.github.ai) inference # Route publisher/model ids (e.g. openai/gpt-4o) to GitHub Models instead of @@ -64,6 +64,11 @@ tool_result_suffix_remove: [] # Retry: max retries for upstream connection errors (0 = none) max_connection_retries: 3 +# Max seconds of silence from an upstream response before it is treated as +# dead. Bounds silence, not total duration, so long streams are fine. +# 0 disables the timeout. +upstream_read_timeout_seconds: 900 + # Optional: require this key on all LLM endpoints (Bearer / x-api-key / # x-goog-api-key). Omit or leave empty to disable authentication. # api_key: my-secret-key @@ -80,6 +85,14 @@ Incoming model names are rewritten before the request is forwarded upstream: Exact matches take priority over prefix matches. Unmapped names pass through unchanged. Use the live catalog at `GET /v1/models` to discover valid targets. +The built-in mappings point every Claude spelling at the newest generally +available Opus — currently `claude-opus-5` — and every Haiku spelling at +`claude-haiku-4.5`. Anthropic writes the same version two ways (`4.8` and +`4-8`), so both forms are listed. + +These are defaults for a *new* config file. A file you already have is never +rewritten to follow them: see [Schema upgrades](#schema-upgrades). + ### Account type Controls the upstream base URL only: @@ -120,6 +133,21 @@ this number; on the next start the proxy fills the missing properties with their default values and rewrites `config.yaml` automatically, preserving every value you have set. No flag is needed for this. +**An upgrade only ever adds.** A model mapping already in the file keeps +pointing where you told it to, even when a newer release changes the built-in +default for that alias. There is no way to tell a mapping left at an old +default apart from one deliberately pinned to that version, and pinning is +common — so nothing existing is rewritten. To adopt the new defaults, run +`--setup` or start it from built-in defaults with `--default`. + +Schema versions so far: + +| Version | Introduced | +|---------|------------| +| 2 | Opus 4.8 aliases | +| 3 | `upstream_read_timeout_seconds`; `auto_upgrade` defaulting to true | +| 4 | Opus 5 and Sonnet 5 aliases | + `--update-config` remains for the other, non-schema write-backs (for example restoring the built-in `model_mappings` when the file has none). @@ -149,6 +177,7 @@ ghc-proxy [options] --fetch-version Fetch the latest VS Code version at startup --no-fetch-version Disable dynamic VS Code version fetching --auto-upgrade Auto-upgrade app when a newer release is available + (default: on) --no-auto-upgrade Disable app auto-upgrade --update-config Persist non-schema config write-backs (schema upgrades apply automatically) -v, --version Show version @@ -169,10 +198,11 @@ Every config field has a `GHC_PROXY_*` override: | `GHC_PROXY_API_VERSION` | `X-GitHub-Api-Version` string | | `GHC_PROXY_COPILOT_VERSION` | Copilot Chat plugin version string | | `GHC_PROXY_MAX_CONNECTION_RETRIES` | Max connection retries | +| `GHC_PROXY_UPSTREAM_READ_TIMEOUT` | Max seconds of upstream silence (`0` disables) | | `GHC_PROXY_REDIRECT_ANTHROPIC` | Always translate Anthropic via chat completions | | `GHC_PROXY_SHOW_TOKEN` | Log tokens on refresh (`true`/`1`) | | `GHC_PROXY_DYNAMIC_VSCODE_VERSION` | Fetch latest VS Code version (`true`/`1`) | -| `GHC_PROXY_AUTO_UPGRADE` | Auto-upgrade app on startup (`true`/`1`) | +| `GHC_PROXY_AUTO_UPGRADE` | Auto-upgrade app on startup (`true`/`1`); set `0` to disable | | `GHC_PROXY_RATE_LIMIT_SECONDS` | Minimum seconds between requests | | `GHC_PROXY_RATE_LIMIT_WAIT` | Wait instead of rejecting when limited (`true`/`1`) | | `GHC_PROXY_MANUAL_APPROVE` | Require manual approval per request (`true`/`1`) | @@ -215,3 +245,26 @@ stale clients: Enable `dynamic_vscode_version` (or `--fetch-version`) to refresh the VS Code version automatically at startup. + +## Self-update + +`auto_upgrade` is **enabled by default**, including for config files written +before the setting existed. On startup the proxy checks GitHub releases and, if +a newer version is published, downloads and replaces its own binary. The +replacement takes effect on the **next** start — the running process keeps +serving the old code until it restarts. + +Disable it with any of: + +```yaml +auto_upgrade: false +``` + +```bash +ghc-proxy --no-auto-upgrade +GHC_PROXY_AUTO_UPGRADE=0 ghc-proxy +``` + +Turn it off if the binary is managed by a package manager, or if it lives +somewhere the process should not rewrite — for example a build output directory +that `cargo build` or `cargo clean` also writes to. diff --git a/docs/getting-started.md b/docs/getting-started.md index 0ed26a8..8a40338 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -42,6 +42,7 @@ Once running, the proxy prints the endpoints it serves: ```text Starting GitHub Copilot API Proxy on 127.0.0.1:8314 Dashboard: http://127.0.0.1:8314/ +Health check: http://127.0.0.1:8314/health Metrics UI: http://127.0.0.1:8314/metrics/dashboard OpenMetrics: http://127.0.0.1:8314/metrics Reload config: POST http://127.0.0.1:8314/api/config/reload @@ -52,6 +53,10 @@ Gemini API: http://127.0.0.1:8314/v1beta/models/{model}:generateContent OpenAPI spec: http://127.0.0.1:8314/openapi.json ``` +The dashboard link is the overview; `/requests` browses individual exchanges. +See the [API reference](api.md) for the rest, including the WebSocket transport +and the statistics endpoints. + ## Authentication A GitHub token is resolved in this order: @@ -122,3 +127,5 @@ persisted `machine_id.txt` used for client disguise. - Tune behavior in the [Configuration](configuration.md) reference. - Explore the [API Reference](api.md) and client examples. - Wire up [Claude Code & Codex](claude-code.md). +- Watch what it costs: the overview at `/` leads with the AI units Copilot + billed, and `/api/cache` breaks prompt-cache effectiveness down per model. diff --git a/docs/index.md b/docs/index.md index 99e970c..a035526 100644 --- a/docs/index.md +++ b/docs/index.md @@ -33,6 +33,7 @@ so OpenAI- and Anthropic-native clients work unmodified. ## Highlights - **OpenAI-compatible** `/v1/chat/completions` and `/v1/responses` (Codex) endpoints. +- **The Responses API over WebSocket** — `GET /v1/responses` with `Upgrade: websocket`, for the models whose catalog advertises `ws:/responses`. - **Anthropic-compatible** `/v1/messages` endpoint with native passthrough or translation. - **Gemini-compatible** `/v1beta/models/{model}:generateContent` (+ streaming and token counting). - **Embeddings**, **model listing**, **token counting**, and a **usage** endpoint. @@ -43,7 +44,9 @@ so OpenAI- and Anthropic-native clients work unmodified. - **Optional API-key auth** on the LLM endpoints (Bearer / `x-api-key` / `x-goog-api-key`), off by default. - **One-click client setup** for Claude Code, Codex, and the Gemini CLI. - **OpenAPI spec** at `/openapi.json`. -- **Analytics dashboard** at `/` and a request browser at `/requests`. +- **Three-page dashboard** — an overview at `/` led by what Copilot actually billed, a request browser at `/requests` that renders each exchange as a conversation, and metrics at `/metrics/dashboard`. +- **Spend and cache analytics** — billed AI units read from `copilot_usage` rather than estimated, and per-model prompt-cache statistics at `/api/cache`. +- **Body capture toggled at runtime** via `POST /api/config/debug`, so you can start recording bodies without a restart. ## Quick start diff --git a/public/app.css b/public/app.css new file mode 100644 index 0000000..42f2ebe --- /dev/null +++ b/public/app.css @@ -0,0 +1,372 @@ +/* ghc-proxy dashboard — shared design system. + * + * Served from /app.css so the three pages share one visual language instead of + * carrying three drifting copies of the same rules. */ + +:root { + color-scheme: dark; + + --bg: #0d1117; + --surface: #161b22; + --surface-2: #21262d; + --border: #30363d; + --border-soft: #21262d; + --text: #e6edf3; + --muted: #8b949e; + --accent: #58a6ff; + --ok: #3fb950; + --warn: #d29922; + --danger: #f85149; + + --radius: 8px; + --radius-sm: 6px; + --mono: ui-monospace, SFMono-Regular, Menlo, Consolas, monospace; + + /* Page width is per-page, not global: the overview is a handful of large + * numbers and short labels, which just drift apart when stretched, while the + * request table and the metric list are data that a wide screen genuinely + * helps you read. Capped rather than fluid so an ultrawide does not turn a + * table row into a journey for the eye. */ + --page-max: 1280px; +} + +body[data-page="requests"] { --page-max: 2000px; } +body[data-page="metrics"] { --page-max: 1600px; } + +* { box-sizing: border-box; } + +body { + font-family: system-ui, -apple-system, "Segoe UI", Roboto, sans-serif; + margin: 0; + background: var(--bg); + color: var(--text); + font-size: 14px; + line-height: 1.5; +} + +a { color: var(--accent); text-decoration: none; } +a:hover { text-decoration: underline; } + +.wrap { + max-width: var(--page-max); + margin: 0 auto; + padding: 0 1.5rem 4rem; +} + +/* ---------------------------------------------------------------- nav ---- */ +/* Persistent across all three pages: the "trunk test" wants every page to + * answer what this is, where you are, and what the other sections are. */ + +.nav { + position: sticky; + top: 0; + z-index: 10; + background: rgba(13, 17, 23, 0.85); + backdrop-filter: blur(8px); + border-bottom: 1px solid var(--border); + margin-bottom: 1.75rem; +} + +.nav-inner { + max-width: var(--page-max); + margin: 0 auto; + padding: 0 1.5rem; + display: flex; + align-items: center; + gap: 1.5rem; + height: 52px; +} + +.brand { + font-weight: 600; + color: var(--text); + letter-spacing: -0.01em; + white-space: nowrap; +} +.brand:hover { text-decoration: none; } + +.nav-links { display: flex; gap: 0.25rem; flex: 1; } + +.nav-links a { + color: var(--muted); + padding: 0.3rem 0.7rem; + border-radius: var(--radius-sm); + font-size: 0.875rem; +} +.nav-links a:hover { color: var(--text); background: var(--surface-2); text-decoration: none; } + +/* Current section, set via */ +body[data-page="overview"] .nav-links a[href="/"], +body[data-page="requests"] .nav-links a[href="/requests"], +body[data-page="metrics"] .nav-links a[href="/metrics/dashboard"] { + color: var(--text); + background: var(--surface); + box-shadow: inset 0 0 0 1px var(--border); +} + +.nav-status { + display: flex; + align-items: center; + gap: 0.5rem; + font-size: 0.8rem; + color: var(--muted); + white-space: nowrap; +} + +.dot { width: 8px; height: 8px; border-radius: 50%; background: var(--muted); flex: none; } +.dot.ok { background: var(--ok); } +.dot.warn { background: var(--warn); } +.dot.err { background: var(--danger); } + +/* -------------------------------------------------------------- panels --- */ + +.panel { + background: var(--surface); + border: 1px solid var(--border); + border-radius: var(--radius); + margin-bottom: 1.25rem; + overflow: hidden; +} + +.panel-head { + display: flex; + align-items: baseline; + justify-content: space-between; + gap: 1rem; + padding: 0.7rem 1rem; + border-bottom: 1px solid var(--border); +} + +.panel-head h2 { + margin: 0; + font-size: 0.72rem; + font-weight: 600; + text-transform: uppercase; + letter-spacing: 0.08em; + color: var(--muted); +} + +.panel-head .aside { font-size: 0.8rem; color: var(--muted); } +.panel-body { padding: 1rem; } +.panel-body.flush { padding: 0; } + +/* -------------------------------------------------------------- metrics -- */ +/* Deliberately unequal: the hero row is what the page is *for*, so it gets + * size, and everything else is demoted rather than competing with it. */ + +.hero { + display: grid; + grid-template-columns: repeat(auto-fit, minmax(200px, 1fr)); + gap: 1px; + background: var(--border); +} + +.hero .metric { background: var(--surface); padding: 1.1rem 1.25rem; } + +.metric .label { + color: var(--muted); + font-size: 0.72rem; + text-transform: uppercase; + letter-spacing: 0.07em; +} + +.metric .value { + font-size: 2rem; + font-weight: 600; + line-height: 1.15; + margin-top: 0.35rem; + letter-spacing: -0.02em; + font-variant-numeric: tabular-nums; +} + +.metric .sub { color: var(--muted); font-size: 0.8rem; margin-top: 0.2rem; } +.metric .sub b { color: var(--text); font-weight: 600; } + +/* Secondary stats: same information architecture, a third of the weight. */ +.stat-grid { + display: grid; + grid-template-columns: repeat(auto-fit, minmax(150px, 1fr)); + gap: 0.9rem 1.5rem; +} + +.stat .label { + color: var(--muted); + font-size: 0.72rem; + text-transform: uppercase; + letter-spacing: 0.07em; +} +.stat .value { + font-size: 1.15rem; + font-weight: 600; + font-variant-numeric: tabular-nums; + margin-top: 0.15rem; +} +.stat .value.ok { color: var(--ok); } +.stat .value.warn { color: var(--warn); } +.stat .value.err { color: var(--danger); } + +.cols-2 { display: grid; grid-template-columns: repeat(auto-fit, minmax(320px, 1fr)); gap: 1.25rem; } +.cols-2 > .panel { margin-bottom: 0; } + +/* ---------------------------------------------------------------- quota -- */ + +.quota-row { display: grid; gap: 0.3rem; padding: 0.7rem 0; border-bottom: 1px solid var(--border-soft); } +.quota-row:last-child { border-bottom: none; padding-bottom: 0; } +.quota-row:first-child { padding-top: 0; } + +.quota-top { display: flex; justify-content: space-between; align-items: baseline; gap: 1rem; } +.quota-name { font-family: var(--mono); font-size: 0.82rem; } +.quota-val { font-size: 0.82rem; color: var(--muted); font-variant-numeric: tabular-nums; } +.quota-val b { color: var(--text); font-weight: 600; } + +.bar { height: 6px; border-radius: 3px; background: var(--surface-2); overflow: hidden; } +.bar > span { display: block; height: 100%; background: var(--ok); border-radius: 3px; } +.bar > span.warn { background: var(--warn); } +.bar > span.err { background: var(--danger); } + +/* --------------------------------------------------------------- tables -- */ + +table { width: 100%; border-collapse: collapse; font-size: 0.85rem; } + +th, td { + text-align: left; + padding: 0.5rem 0.75rem; + border-bottom: 1px solid var(--border); + white-space: nowrap; +} + +th { + color: var(--muted); + text-transform: uppercase; + font-size: 0.7rem; + letter-spacing: 0.05em; + font-weight: 600; +} + +tbody tr:last-child td { border-bottom: none; } +td.num, th.num { text-align: right; font-variant-numeric: tabular-nums; } +.mono { font-family: var(--mono); } +.muted { color: var(--muted); } +.status-ok { color: var(--ok); } +.status-err { color: var(--danger); } + +/* -------------------------------------------------------------- controls -- */ + +button, .btn { + background: var(--surface-2); + color: var(--text); + border: 1px solid var(--border); + border-radius: var(--radius-sm); + padding: 0.35rem 0.75rem; + cursor: pointer; + font: inherit; + font-size: 0.85rem; +} +button:hover:not(:disabled) { border-color: var(--muted); } +button:disabled { opacity: 0.45; cursor: default; } + +input[type="text"], input:not([type]) { + background: var(--bg); + color: var(--text); + border: 1px solid var(--border); + border-radius: var(--radius-sm); + padding: 0.45rem 0.7rem; + font: inherit; + font-size: 0.85rem; +} +input:focus { outline: none; border-color: var(--accent); } + +.toolbar { + display: flex; + gap: 0.75rem; + align-items: center; + flex-wrap: wrap; + font-size: 0.85rem; + color: var(--muted); +} +.toolbar label { display: flex; gap: 0.35rem; align-items: center; cursor: pointer; } + +.pager { display: flex; gap: 0.5rem; align-items: center; padding: 0.75rem 1rem; color: var(--muted); font-size: 0.85rem; } + +/* Collapsible reference material — present but never competing with the + * live numbers above it. */ +details.fold > summary { + cursor: pointer; + padding: 0.7rem 1rem; + color: var(--muted); + font-size: 0.72rem; + text-transform: uppercase; + letter-spacing: 0.08em; + font-weight: 600; + list-style: none; + user-select: none; +} +details.fold > summary::-webkit-details-marker { display: none; } +details.fold > summary::before { content: "▸ "; } +details.fold[open] > summary::before { content: "▾ "; } +details.fold > summary:hover { color: var(--text); } +details.fold[open] > summary { border-bottom: 1px solid var(--border); } + +/* Where the input tokens came from, at a glance. Three shares of one bar read + * faster than three numbers you have to divide in your head. */ +.split { display: flex; height: 8px; border-radius: 4px; overflow: hidden; background: var(--surface-2); } +.split > span { display: block; height: 100%; } +.split .s-read { background: var(--ok); } +.split .s-write { background: var(--warn); } +.split .s-fresh { background: #6e7681; } +/* Inline in a table row, where 8px would crowd the text beside it. */ +.split.mini { height: 6px; min-width: 120px; } + +.legend { display: flex; flex-wrap: wrap; gap: 0.9rem; margin-top: 0.55rem; font-size: 0.78rem; color: var(--muted); } +/* Standing on its own as a key rather than captioning a bar above it. */ +.legend:first-child { margin-top: 0; } +.legend span { display: flex; align-items: center; gap: 0.35rem; } +.legend i { width: 8px; height: 8px; border-radius: 2px; flex: none; } +.legend b { color: var(--text); font-weight: 600; font-variant-numeric: tabular-nums; } + +.dim-note { color: var(--muted); font-size: 0.85rem; } + +/* A setting you can see is a setting you should be able to change. */ +.toggle { + display: inline-flex; + align-items: center; + gap: 0.45rem; + margin-top: 0.15rem; + padding: 0.2rem 0.6rem 0.2rem 0.3rem; + border-radius: 999px; + font-size: 0.9rem; + font-weight: 600; + color: var(--muted); +} +.toggle .knob { + width: 26px; + height: 15px; + border-radius: 999px; + background: var(--border); + position: relative; + transition: background 0.12s ease; + flex: none; +} +.toggle .knob::after { + content: ""; + position: absolute; + top: 2px; + left: 2px; + width: 11px; + height: 11px; + border-radius: 50%; + background: var(--muted); + transition: transform 0.12s ease, background 0.12s ease; +} +.toggle.on { color: var(--warn); border-color: var(--warn); } +.toggle.on .knob { background: rgba(210, 153, 34, 0.35); } +.toggle.on .knob::after { transform: translateX(11px); background: var(--warn); } +.toggle:disabled { opacity: 0.6; cursor: progress; } + +.empty { color: var(--muted); padding: 1.25rem 1rem; text-align: center; font-size: 0.875rem; } + +.scroll-y { max-height: 420px; overflow-y: auto; } + +/* The request table carries more columns than fit on a laptop. Scroll it + * rather than let the panel clip columns off the right edge. */ +.scroll-x { overflow-x: auto; } diff --git a/public/dashboard.html b/public/dashboard.html index 90c033d..4d05ab9 100644 --- a/public/dashboard.html +++ b/public/dashboard.html @@ -3,83 +3,436 @@ - ghc-proxy · Dashboard - + ghc-proxy · Overview + - -

GitHub Copilot API Proxy

-
-

Supported models

- - - -
IDDisplay nameOwned by
-