feat(auth): configurable runtime-token header (server + SDK) to avoid gateway Authorization collision [HYBIM-741]

[josjeon]

Draft yet

Problem

When Agent Control runs behind the O11y gateway, the gateway overwrites Authorization with its own downstream identity JWT — clobbering AC's runtime-eval token on the hot path (HYBIM-741). The runtime token and the gateway's identity JWT both want the Authorization header.

Fix

Make the runtime token ride a configurable header on both sides, selected by AGENT_CONTROL_RUNTIME_TOKEN_HEADER (default Authorization). Behind the gateway, point both sides at a dedicated header (e.g. X-Agent-Control-Runtime-Token); the runtime token rides that header while the gateway keeps Authorization for its identity JWT — no collision.

• Authorization: keeps the mandatory Bearer scheme (existing contract).
• Dedicated header: carries the raw token (no Bearer prefix).
• Default unchanged: with the env unset, behavior is byte-identical to today.

Server (verify side)

• auth_framework/providers/local_jwt.py: LocalJwtVerifyProvider takes a header_name (default Authorization) and reads the token from it. Bearer required only on Authorization; raw token accepted on a dedicated header.
• auth_framework/config.py: _resolve_runtime_token_header() reads AGENT_CONTROL_RUNTIME_TOKEN_HEADER (blank → default), passed into the provider when runtime mode is jwt.

SDK (send side)

• sdks/python/.../client.py: AgentControlClient gains a runtime_token_header param (+ same env var). Sends the runtime token on the configured header — raw on a dedicated header, Bearer on Authorization (single _format_runtime_token helper is the sole authority for that rule). When the runtime token rides its own header, the API-key fallback is suppressed so a runtime request carries a single credential; when no runtime token is minted (e.g. auto mode + exchange unavailable), the API key still authenticates the request.

Configuration (behind the gateway)

# server
AGENT_CONTROL_RUNTIME_AUTH_MODE=jwt
AGENT_CONTROL_RUNTIME_TOKEN_SECRET=<secret>
AGENT_CONTROL_RUNTIME_TOKEN_HEADER=X-Agent-Control-Runtime-Token

# SDK (must match the server header)
AgentControlClient(..., runtime_token_header="X-Agent-Control-Runtime-Token")
# or AGENT_CONTROL_RUNTIME_TOKEN_HEADER=X-Agent-Control-Runtime-Token

Tests

• Server (test_auth_framework.py): default Bearer path; default rejects raw on Authorization; dedicated header reads raw token and coexists with a gateway Authorization JWT; Bearer also accepted on dedicated header; missing-header error names the configured header; blank header_name rejected; env resolver (unset → default, set → trimmed, whitespace → default).
• SDK (test_client.py): header resolution (param/env/default, blank rejected, whitespace-env fallback); raw token on dedicated header with Authorization free and no API-key ridealong; default sends Bearer on Authorization; auto-mode fallback keeps the API key when the exchange is unavailable.

Security notes

• Header isolation: with a dedicated header configured, the verifier reads only that header — a token presented on Authorization is ignored, so it can't be smuggled past the gateway boundary.
• No token leakage: auth error messages reference the header name only, never the token value.
• Signature / scope / target-binding checks (verify_runtime_token) are unchanged.

Notes / scope

• Backwards compatible: unset env → identical behavior. No change to the API-key or none runtime modes.
• Design follows the O11y api-service precedent: support both auth methods, gateway stays neutral, opt-in, default preserved.
• Server-side unit tests require the repo's Postgres test fixture (run in CI); the SDK suite runs standalone (40 passing).
• Gateway forwarding of the custom header is confirmed with the gateway owners; end-to-end lab0 validation is pending.