Skip to content

fix: restore model selection header + DSH/agent SSE stream termination & true tool streaming - #95

Open
kirigayakazima wants to merge 2 commits into
Sophomoresty:mainfrom
kirigayakazima:feat/model-selection-header
Open

fix: restore model selection header + DSH/agent SSE stream termination & true tool streaming#95
kirigayakazima wants to merge 2 commits into
Sophomoresty:mainfrom
kirigayakazima:feat/model-selection-header

Conversation

@kirigayakazima

@kirigayakazima kirigayakazima commented Sep 5, 2026

Copy link
Copy Markdown

Closes #82 — model selection is broken upstream: payload slot79 no longer routes models, every request is served by the account default.

Problem 1: Model routing is ignored

Google moved model routing from the payload slot79 field to the x-goog-ext-525001261-jspb request header (internal model ID selector). Without that header, slot79 is ignored and every model name in MODELS comes back as the same account-default model. Issue #82 documents this in detail.

Problem 2: Streams never terminate for strict OpenAI clients (DSH / Codex)

The SSE responses were sent over HTTP/1.1 without Content-Length and without Transfer-Encoding: chunked. Python's BaseHTTPRequestHandler does not emit chunked encoding automatically, so strict clients (OpenAI SDK, used by coding agents like DeepSeek Harness) wait forever for a body terminator that never arrives — the connection stays open (keep-alive), the stream never ends, and the client shows an endless thinking state until its idle timeout (300s) fires.

Fix: protocol_version = HTTP/1.0 → the connection closes after the SSE body (EOF), which is the canonical SSE termination signal.

Problem 3: Tool requests were not truly streaming

When tools were present, the handler fell back to a blocking full-generate-then-single-chunk path: it waited for Google to finish the entire response, parsed tool_calls, then returned one chunk. For agent clients sending 40+ tools, this produced multi-minute latencies and idle timeouts.

Fix: true streaming with tools — stream text chunks as they arrive (fast time-to-first-token), then emit the parsed tool_calls delta + finish_reason: tool_calls at the end.

Problem 4: Verbose tool descriptions blow up the prompt

Coding agents send 43 tools with long descriptions (~32KB JSON). Google Web takes minutes to process such a large prompt, again hitting client idle timeouts.

Fix: compact tool descriptions (trim each to ~150 chars, keep full parameter schemas) — ~32KB → ~12KB, responses in seconds while tools stay usable.

Changes

  • MODEL_IDS map + build_model_header() emitting x-goog-ext-525001261-jspb (threaded through both stream and non-stream paths)
  • fetch_xsrf_token() auto-discovery (FdrFJe, successor of SNlM0e)
  • load_cookie() accepts gemini-auth.json and injects xsrf/gemini_bl/auth_user
  • Add gemini-3.8-flash model entry
  • HTTP/1.0 SSE termination (fixes endless thinking / idle timeout)
  • True streaming with tools (fast TTFT + proper tool_calls delta)
  • Tool description compaction (full parameters, trimmed descriptions)
  • Real-time server.log output (stderr + file)

Verification

Tested with a paid AI Pro account + DeepSeek Harness (DSH) as the OpenAI-compatible client:

  • Model routing confirmed via authoritative slot39 (56fdd199312815e2 served, not the cf41b0e0dd7d53e5 reject marker)
  • Streaming terminates correctly: OpenAI SDK receives content chunks, then finish_reason, then [DONE] — no more endless thinking
  • 43-tool agent requests complete in ~9s instead of timing out at 300s

Related

#88 (xsrf auto-refresh) and #91 (3.8-flash) are open PRs covering similar territory; happy to drop overlapping parts if those merge first.

Google moved model routing from payload slot79 to the
x-goog-ext-525001261-jspb request header (Issue Sophomoresty#82). Without it, every
request is served by the account default model and model selection silently
no-ops.

Changes:
- Add MODEL_IDS mapping (verified internal IDs from browser captures)
- Add build_model_header() and send the header on both stream and
  non-stream paths with model_name threaded through call sites
- Add fetch_xsrf_token() auto-discovery (FdrFJe, successor of SNlM0e)
- load_cookie() now also accepts gemini-auth.json (exported by the
  bundled extension) and injects xsrf/gemini_bl/auth_user into CONFIG
- Add gemini-3.8-flash model entry
… with tools, lean tool prompt

Resolves turn never completing in strict OpenAI clients (DSH, Codex):
- protocol_version=HTTP/1.0: SSE streams now end via connection close (EOF).
  HTTP/1.1 without chunked transfer encoding leaves strict clients waiting
  forever for a body terminator that BaseHTTPRequestHandler never sends,
  causing idle timeouts and endless 'thinking' states.
- True streaming with tools: stream text chunks as they arrive (fast TTFT),
  emit parsed tool_calls + finish_reason=tool_calls at the end. Previously
  tool requests fell back to a blocking full-generate-then-single-chunk path.
- Compact tool descriptions (trim to 150 chars, keep full parameters): DSH
  sends 43 tools (~32KB) which took Google Web minutes to process and hit
  clients' stream idle timeout (300s). Now ~12KB, responses in seconds.
- Log to server.log in real time (stderr + file, unbuffered).
@kirigayakazima kirigayakazima changed the title fix: restore model selection via x-goog-ext-525001261-jspb header fix: restore model selection header + DSH/agent SSE stream termination & true tool streaming Sep 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

slot79 (MODE_CATEGORY) is ignored upstream: gemini-3.1-pro returns "3.7 Flash" with no thinking

1 participant