diff --git a/docs-site/src/content/docs/reference/configuration/providers.md b/docs-site/src/content/docs/reference/configuration/providers.md index 29fa6ef833..2c1d2e3eef 100644 --- a/docs-site/src/content/docs/reference/configuration/providers.md +++ b/docs-site/src/content/docs/reference/configuration/providers.md @@ -69,9 +69,9 @@ differing backup and rewrites known legacy namespaced selected ids to bare ids. | `requestPacing?` | `{ enabled, requestsPerMinute?, minIntervalMs?, models? }` | Optional client-side outbound request-start pacing, separate from upstream usage, billing, and rate-limit indicators. RPM is converted to an even interval; `minIntervalMs` may impose a longer interval. Provider limits apply across all models, while `models` entries use exact upstream model IDs (for example `nvidia/llama-3.1-nemotron-ultra-253b-v1`) and can only add delay. Queue waits do not consume the upstream response-header timeout. HTTP, Responses WebSocket, and explicit adapter `fetchResponse`/`runTurn` dispatches are covered. | | `upstreamHttpVersion?` | `"auto" \| "http1.1" \| "h1" \| "http2" \| "h2"` | Pin the HTTP version used for upstream requests to this provider. Defaults to `auto`, which lets Bun negotiate. Set `http1.1` when a provider's HTTP/2 SSE stream stalls instead of delivering events — the symptom is a long-running streaming request that produces nothing and eventually times out. Management `POST`/`PATCH` accept `null` to clear it back to `auto`. | | `responsesPath?` | `string` | Relative resource path for key-auth `openai-responses` requests. It must start with `/` and contain no scheme, query, or fragment. | -| `supportsServiceTier?` | `boolean` | Tri-state `service_tier` capability fallback. `true`: fast mode may inject and caller values are preserved. `false`: the field is stripped and never injected, and exact model declarations cannot reopen it. Absent: the provider is unclassified — caller-supplied values are preserved untouched and fast mode never injects unless an exact model is enabled. The registry classifies canonical OpenAI (`true`), DeepSeek, and Volcengine Ark (`false`); set it explicitly only for custom gateways that genuinely support tiers. Chat routes additionally need provider-wide or exact-model Chat authorization. | -| `modelSupportsServiceTier?` | `Record` | Exact upstream model capability overrides. Exact `true` authorizes that Chat model even without `chatServiceTier`; exact `false` narrows provider defaults and Chat authorization. An explicit provider-level `supportsServiceTier: false` remains fail-closed and cannot be reopened. Undeclared models fall back to provider-wide behavior. Management `PATCH /api/providers` merges entries and accepts `null` to clear one. | -| `chatServiceTier?` | `boolean` | Provider-wide wire opt-in for serializing `service_tier` on `/chat/completions`. Exact models may instead opt in through `modelSupportsServiceTier`; undeclared models remain blocked when this flag is absent or false. | +| `supportsServiceTier?` | `boolean` | Tri-state canonical Fast capability fallback. `true` publishes Fast in the catalog, satisfies service-tier routing requirements, contributes a supported fingerprint, and lets fast mode inject the provider's canonical wire value on a compatible final adapter. `false` strips the field and never injects, and exact model declarations cannot reopen it. Absent leaves the provider unclassified: fast mode does not inject or normalize a canonical caller value, and caller values obey the final wire's forwarding permission (`chatServiceTier` on Chat; passthrough on Responses). The registry classifies canonical OpenAI (`true`), DeepSeek, and Volcengine Ark (`false`); set it explicitly only for custom gateways that genuinely support tiers. | +| `modelSupportsServiceTier?` | `Record` | Exact upstream model capability overrides. Exact `true` enables canonical Fast for that model; exact `false` narrows provider defaults. An explicit provider-level `supportsServiceTier: false` remains fail-closed and cannot be reopened. Exact `true` does not authorize foreign caller-tier forwarding on Chat. Undeclared models fall back to provider-wide behavior. Management `PATCH /api/providers` merges entries and accepts `null` to clear one. | +| `chatServiceTier?` | `boolean` | Provider-wide Chat-wire opt-in for forwarding caller `service_tier` values. On a classified route it governs foreign values such as `flex`, not proxy-owned canonical Fast after capability validation; on an unclassified route it governs every caller value because no Fast capability has been validated. Exact model capability does not authorize foreign forwarding. Responses routes retain their capability-based caller forwarding behavior. | | `preserveResponsesReasoningContent?` | `boolean` | Keep plaintext reasoning content on replayed Responses reasoning items instead of blanking it (blanking is the ChatGPT backend's rule). Enable for upstreams whose contract accepts reasoning replay, such as DeepSeek. Proxy-minted `ocxr1` envelopes are always stripped. | | `disabled?` | `boolean` | Keep the provider on disk but exclude it from routing and model/catalog listings. | | `apiKey?` | `string` | API key, or an `${ENV_VAR}` / `$ENV_VAR` reference resolved at request time. | @@ -130,6 +130,27 @@ differing backup and rewrites known legacy namespaced selected ids to bare ids. | `unsafeAllowNativeLocalExec?` | `boolean` | Cursor legacy boolean, equivalent to `nativeLocalExec: "on"` only when the newer field is unset. | | `nativeLocalExec?` | `"off" \| "codex-sandbox" \| "on"` | Cursor local-exec policy. `off` is default; `codex-sandbox` currently fails closed like `off`. | +### FastWire B1 capability migration + +Fast capability and arbitrary Chat caller-tier forwarding are independent after FastWire B1. The +[provider-field definitions](#provider-entries-ocxproviderconfig) above remain the authoritative +contract; existing configurations see these migration deltas: + +1. A Chat provider/model declared Fast-capable no longer needs `chatServiceTier: true` for canonical + Fast. Publication, routing eligibility, and injection still require an eligible policy and a + compatible FastWire mapping on the final adapter. On classified routes, `fastMode: false` still + removes canonical Fast. Set `supportsServiceTier: false` or an exact-model `false` when the route + is not Fast-capable. +2. On an eligible classified route, caller spellings `fast` and `FAST` normalize through + `fastWire.canonicalToWire.priority`; caller `priority` remains canonical. Configure a verified + mapping to `fast` only when that is the upstream's canonical value. Unclassified routes retain + their existing forwarding behavior. +3. Exact-model `true` no longer authorizes foreign Chat tiers such as `flex` or vendor-specific + values. Those still require `chatServiceTier: true`; otherwise they are removed and recorded as + dropped caller tiers. + +Explicit capability `false` and Responses caller-tier forwarding retain their existing contracts. + API-key providers may hold a literal key or an environment reference. OAuth providers use the credential store populated by `ocx login`; subscription-backed Claude Code launch behavior is configured under [`claudeCode.authMode`](/reference/configuration/server/#claude-code). diff --git a/src/adapters/base.ts b/src/adapters/base.ts index 8789a03463..f5ca7a1c7f 100644 --- a/src/adapters/base.ts +++ b/src/adapters/base.ts @@ -1,5 +1,6 @@ import type { AdapterEvent, OcxParsedRequest } from "../types"; import type { TranslatorBudget } from "../lib/translator-budget"; +import type { AdapterTierMetadata } from "../providers/fastwire"; /** Metadata about the caller's incoming request, for auth-forwarding adapters. */ export interface IncomingMeta { @@ -39,6 +40,9 @@ export interface ProviderAdapter { incoming: IncomingMeta, emit: (event: AdapterEvent) => void, ): Promise; + + /** Exact no-field observation for runTurn adapters, which expose no AdapterRequest object. */ + tierLogForRunTurn?(parsed: OcxParsedRequest): AdapterTierMetadata | undefined; } export interface AdapterRequest { @@ -67,6 +71,12 @@ export interface AdapterRequest { wireField: "reasoning_effort" | "reasoning.effort" | "thinking.type"; wireValue: string; }; + /** + * Exact tier outcome seeded after this adapter serialized the outbound request. + * This is a live shared observer: response-phase methods mutate `outcome`, so retain + * the reference rather than cloning or snapshotting it. + */ + tierLog?: AdapterTierMetadata; usageLog?: { inputTokens?: number; estimated?: boolean; diff --git a/src/adapters/openai-chat.ts b/src/adapters/openai-chat.ts index 8275a5f3de..a06ec51f34 100644 --- a/src/adapters/openai-chat.ts +++ b/src/adapters/openai-chat.ts @@ -12,7 +12,14 @@ import { identifyRoutedModel } from "./identity"; import { peekReasoningForCall } from "../responses/reasoning-replay-cache"; import { buildNonOpenAIToolCatalogNudgeForTools, shouldInjectNonOpenAIToolCatalogNudge } from "./tool-catalog-nudge"; import { openRouterProviderPayload, resolveOpenRouterRouting } from "../providers/openrouter-routing"; -import { canSerializeServiceTierForChatModel } from "../providers/service-tier"; +import { + canForwardForeignServiceTierForChatModel, + supportsServiceTierForModel, +} from "../providers/service-tier"; +import { + canonicalFastTierMarker, + createAdapterTierMetadata, +} from "../providers/fastwire"; import { openaiChatCompletionsUrl } from "./openai-chat-url"; import { stripResponsesOnlyEncryptedMarker } from "./responses-tool-schema"; import { @@ -1287,17 +1294,20 @@ export function createOpenAIChatAdapter(provider: OcxProviderConfig): ProviderAd messages, stream: parsed.stream, }; - // Preserve a caller-selected service tier for OpenAI-compatible chat gateways. The - // request pipeline deliberately does not inject fast mode for this adapter, but dropping - // an explicit value here makes the Responses parser's serviceTier projection ineffective. - // - // Opt-in, like `prompt_cache_key` directly below: `service_tier` is an OpenAI-specific - // extension and 66 registry providers share this adapter. A provider-wide Chat opt-in - // authorizes undeclared models; an exact model declaration can authorize or deny one - // model. Provider-level false remains fail-closed. - if (canSerializeServiceTierForChatModel(provider, parsed.modelId) - && parsed.options.serviceTier !== undefined) { - body.service_tier = parsed.options.serviceTier; + // A policy-produced canonical decision has already passed capability validation. Without + // that decision, a canonical caller value still requires an explicit true capability; + // unclassified Chat routes remain behind the caller-forwarding opt-in. + const serviceTier = parsed.options.serviceTier; + const tierDecision = parsed.options.tierDecision; + const callerCanonicalFast = canonicalFastTierMarker(serviceTier) !== undefined; + const callerTierForwardAllowed = canForwardForeignServiceTierForChatModel(provider, parsed.modelId); + const canonicalFastCapability = callerCanonicalFast + && supportsServiceTierForModel(provider, parsed.modelId) === true; + const canSerializeServiceTier = tierDecision?.kind === "set" + || tierDecision?.kind === "forward-caller" + || (tierDecision === undefined && (callerTierForwardAllowed || canonicalFastCapability)); + if (canSerializeServiceTier && serviceTier !== undefined) { + body.service_tier = serviceTier; } if (modelInList(provider.reasoningSplitModels, parsed.modelId)) body.reasoning_split = true; const maxTokens = resolveMaxTokens(provider, parsed); @@ -1430,6 +1440,13 @@ export function createOpenAIChatAdapter(provider: OcxProviderConfig): ProviderAd if (parsed.stream) body.stream_options = { include_usage: true }; const bodyJson = JSON.stringify(body); + const actualServiceTier = typeof body.service_tier === "string" ? body.service_tier : null; + const tierLog = createAdapterTierMetadata( + parsed.options.tierObservation, + parsed.options.tierDecision, + actualServiceTier === null ? null : "service-tier", + actualServiceTier, + ); if (isDebugEnabled()) { let host = "upstream"; try { host = new URL(url).host; } catch { /* keep fallback */ } @@ -1450,6 +1467,7 @@ export function createOpenAIChatAdapter(provider: OcxProviderConfig): ProviderAd headers, body: bodyJson, ...(reasoningLog ? { reasoningLog } : {}), + ...(tierLog ? { tierLog } : {}), }; }, diff --git a/src/adapters/openai-responses.ts b/src/adapters/openai-responses.ts index 177ba0e1f6..ac103b72f4 100644 --- a/src/adapters/openai-responses.ts +++ b/src/adapters/openai-responses.ts @@ -12,6 +12,9 @@ import { modelRecordValue } from "../reasoning-effort"; import type { TranslatorBudget } from "../lib/translator-budget"; import { rewriteRoutedCustomToolsForUpstream } from "../responses/custom-tool-compat"; import { openaiResponsesUrl } from "./openai-responses-url"; +import { + createAdapterTierMetadata, +} from "../providers/fastwire"; // Headers relayed verbatim from the caller in OAuth-passthrough ("forward") mode. // Exported so the web-search sidecar reuses the exact same forwarded-auth set for its ChatGPT call. @@ -1426,11 +1429,21 @@ export function createResponsesPassthroughAdapter(provider: OcxProviderConfig): convertedRoutedCustomToolNames = rewritten.names; } const sanitizedBody = normalizeToolSchemas(stripSparkCompatibility(stripUnsupportedReasoningParams(stripItemIdsWhenUnstored(stripInvalidItemIds(stripUnsupportedHostedTools(sanitizeReasoningInputContent(scrubOcxCompactionItems(outBody), { preserveRawReasoningContent: provider.preserveResponsesReasoningContent === true }))))))); - const body = JSON.stringify(stripDisabledReasoningSummaries( + const finalBody = stripDisabledReasoningSummaries( normalizeConfiguredReasoningSummaryDelivery(sanitizedBody, provider, parsed.modelId), provider, parsed.modelId, - )); + ); + const actualServiceTier = isPlainObject(finalBody) && typeof finalBody.service_tier === "string" + ? finalBody.service_tier + : null; + const tierLog = createAdapterTierMetadata( + parsed.options?.tierObservation, + parsed.options?.tierDecision, + actualServiceTier === null ? null : "service-tier", + actualServiceTier, + ); + const body = JSON.stringify(finalBody); const releaseBodyObservation = translatorBudget.observeExternallyCapped( "passthrough_serialization", new TextEncoder().encode(body).byteLength, @@ -1442,6 +1455,7 @@ export function createResponsesPassthroughAdapter(provider: OcxProviderConfig): body, releaseBodyObservation, ...(convertedRoutedCustomToolNames ? { convertedRoutedCustomToolNames } : {}), + ...(tierLog ? { tierLog } : {}), }; }, diff --git a/src/adapters/registry.ts b/src/adapters/registry.ts index 896dd381bc..e9f54bcb8a 100644 --- a/src/adapters/registry.ts +++ b/src/adapters/registry.ts @@ -10,6 +10,7 @@ import { createMimoFreeAdapter } from "./mimo-free"; import { createOpenAIChatAdapter } from "./openai-chat"; import { createResponsesPassthroughAdapter } from "./openai-responses"; import type { OcxProviderConfig } from "../types"; +import { createAdapterTierMetadata } from "../providers/fastwire"; export type AdapterCacheRetention = "none" | "short" | "long"; @@ -142,5 +143,33 @@ export function createRegisteredAdapter( ): ProviderAdapter { const definition = getAdapterDefinition(provider.adapter); if (!definition) throw new Error(`Unknown adapter: ${provider.adapter}`); - return definition.create(provider, context); + const adapter = definition.create(provider, context); + const buildRequest = adapter.buildRequest.bind(adapter); + adapter.buildRequest = (parsed, incoming) => { + const attachTierMetadata = (request: Awaited>) => { + // OpenAI-family adapters report the exact emitted field themselves. Other adapters + // still report an exact absence at this serialization boundary, which makes a routed + // Fast downgrade observable without asking core to infer an outbound body shape. + request.tierLog ??= createAdapterTierMetadata( + parsed.options.tierObservation, + parsed.options.tierDecision, + null, + null, + ); + return request; + }; + const request = buildRequest(parsed, incoming); + return request instanceof Promise + ? request.then(attachTierMetadata) + : attachTierMetadata(request); + }; + if (adapter.runTurn && !adapter.tierLogForRunTurn) { + adapter.tierLogForRunTurn = parsed => createAdapterTierMetadata( + parsed.options.tierObservation, + parsed.options.tierDecision, + null, + null, + ); + } + return adapter; } diff --git a/src/codex/catalog/provider-fetch.ts b/src/codex/catalog/provider-fetch.ts index 7597a455e6..82da5c7ffa 100644 --- a/src/codex/catalog/provider-fetch.ts +++ b/src/codex/catalog/provider-fetch.ts @@ -408,12 +408,12 @@ function captureProviderGather( const enriched = detachedClone(withCanonicalOpenAiForwardAuthDefault(name, configured)); enrichProviderFromRegistry(name, enriched); const registryTransportMatch = providerMatchesRegistryTransport(name, enriched); + const provider = recursivelyFreeze(enriched); const fastPolicyAuthority = captureFastPolicyAuthority( name, - enriched, + provider, registryTransportMatch, ); - const provider = recursivelyFreeze(enriched); const observedAuth = authResolver.kind === "observed" && provider.authMode !== "forward" && provider.liveModels !== false diff --git a/src/config.ts b/src/config.ts index 87d009e555..4d5547f5fd 100644 --- a/src/config.ts +++ b/src/config.ts @@ -733,15 +733,7 @@ const providerConfigSchema = z.object({ repairInvalidIds: z.boolean().optional(), }).strict().optional(), responsesSnapshotRepair: z.boolean().optional(), -}).passthrough().superRefine((provider, ctx) => { - if (hasFastWireCapabilityConflict(provider)) { - ctx.addIssue({ - code: "custom", - path: ["fastWire"], - message: "fastWire=null conflicts with supportsServiceTier=true", - }); - } -}); +}).passthrough(); const RESERVED_PROVIDER_NAMES = new Set([ // JavaScript prototype-pollution guards. @@ -1403,6 +1395,13 @@ const configSchema = z.object({ }); } const provider = config.providers[name]; + if (hasFastWireCapabilityConflict(provider)) { + ctx.addIssue({ + code: "custom", + path: ["providers", redactSecretString(name), "fastWire"], + message: "fastWire=null conflicts with supportsServiceTier=true", + }); + } const openRouterRoutingError = openRouterRoutingConfigError(provider); if (openRouterRoutingError) { ctx.addIssue({ @@ -2143,8 +2142,10 @@ function warnDegradedNativeSubagentConfig(rawParsed: unknown, config: OcxConfig) /** * Registry metadata can gain service-tier capability after a config was written. An explicit - * `fastWire: null` remains authoritative on load; rejecting the file would discard unrelated - * providers and API keys. Live writes remain strict through validateConfigCandidate(). + * `fastWire: null` remains authoritative on load and on whole-document writes; rejecting either + * would discard or lock access to unrelated providers and API keys. Direct contradictions within + * one provider row remain schema errors through the outer config refinement, where the dynamic + * provider name can be redacted before it reaches diagnostics. */ function inheritedFastWireConflictProviderNames( config: Pick, @@ -2530,13 +2531,6 @@ export function validateConfigCandidate(value: unknown): { ok: true; config: Ocx const result = configSchema.safeParse(value); if (result.success) { const config = normalizeApiKeyIds(result.data as OcxConfig); - const inheritedConflicts = inheritedFastWireConflictProviderNames(config); - if (inheritedConflicts.length > 0) { - return { - ok: false, - error: `schema_invalid: ${inheritedFastWireConflictWarning(inheritedConflicts[0]!)}`, - }; - } return { ok: true, config }; } return { ok: false, error: schemaDiagnosticsError(result.error) }; diff --git a/src/lab/subject/behavior-fingerprint.ts b/src/lab/subject/behavior-fingerprint.ts index 4cdc571aad..8d878e9df4 100644 --- a/src/lab/subject/behavior-fingerprint.ts +++ b/src/lab/subject/behavior-fingerprint.ts @@ -5,7 +5,7 @@ import type { LabBehaviorSource, LabBehaviorValues } from "../live/types"; const CLOSED_KEYS = new Set([ "wire.adapter", "wire.upstreamProtocol", "wire.responsesPath", "wire.commandCodeVersion", "wire.modelSuffixMode", "auth.mode", "auth.transport", - "responses.stateful", "responses.upstreamStreaming", "responses.serviceTier", "responses.snapshotRepair", "responses.itemIdRepair", + "responses.stateful", "responses.upstreamStreaming", "responses.serviceTier", "responses.fastWireKind", "responses.fastWireValue", "responses.snapshotRepair", "responses.itemIdRepair", "limits.contextWindow", "limits.maxInputTokens", "limits.maxOutputTokens", "modalities.input", "sampling.omitTemperature", "sampling.omitTopP", "sampling.omitPenalties", @@ -72,6 +72,6 @@ export function normalizeBehaviorValues(values: LabBehaviorValues): LabBehaviorV /** Hash the authoritative production resolver output; Lab performs validation/canonicalization only. */ export function buildBehaviorFingerprintV1(values: LabBehaviorValues): string { - const payload = { schemaVersion: 1, resolverVersion: 1, values: normalizeBehaviorValues(values) }; + const payload = { schemaVersion: 1, resolverVersion: 2, values: normalizeBehaviorValues(values) }; return createHash("sha256").update(jcsStringify(payload)).digest("hex"); } diff --git a/src/lib/redact.ts b/src/lib/redact.ts index 45a468b41d..5561d13c0c 100644 --- a/src/lib/redact.ts +++ b/src/lib/redact.ts @@ -443,6 +443,17 @@ export function redactSecretString(value: string): string { return redacted; } +/** Shared bounded representation for caller-controlled scalar metadata stored in logs. */ +export function sanitizeLogMetadataString(value: unknown, maxLength = 64): string | undefined { + if (typeof value !== "string" || !Number.isInteger(maxLength) || maxLength < 1) return undefined; + // Remove every control/line-separator code point that common terminals and log viewers + // can render as a record boundary before the value reaches a single-line log field. + const filtered = value.trim().replace(/[\u0000-\u001f\u007f-\u009f\u2028\u2029]/g, ""); + if (!filtered) return undefined; + const redacted = redactSecretString(filtered).trim(); + return redacted ? redacted.slice(0, maxLength) : undefined; +} + export function redactSecrets(value: unknown): unknown { if (typeof value === "string") return redactSecretString(value); if (Array.isArray(value)) return value.map(item => redactSecrets(item)); diff --git a/src/providers/derive.ts b/src/providers/derive.ts index 1b3a5d842f..c5e08065b3 100644 --- a/src/providers/derive.ts +++ b/src/providers/derive.ts @@ -1,4 +1,5 @@ import type { CodexAccountMode, OcxProviderConfig } from "../types"; +import { cloneFastWire } from "./fastwire"; import { PROVIDER_REGISTRY, registryEntryForProviderDestination, @@ -460,11 +461,7 @@ export function enrichProviderFromRegistry(name: string, prov: OcxProviderConfig // Registry-only metadata (never seeded into saved config): backfill straight from // the entry so an explicit user value stays distinguishable from the default. if (prov.fastWire === undefined && entry.fastWire !== undefined) { - prov.fastWire = entry.fastWire === null ? null : { - ...entry.fastWire, - canonicalToWire: { ...entry.fastWire.canonicalToWire }, - ...(entry.fastWire.betas ? { betas: [...entry.fastWire.betas] } : {}), - }; + prov.fastWire = cloneFastWire(entry.fastWire); } if (prov.supportsServiceTier === undefined && entry.supportsServiceTier !== undefined) prov.supportsServiceTier = entry.supportsServiceTier; if (prov.preserveResponsesReasoningContent === undefined && entry.preserveResponsesReasoningContent !== undefined) prov.preserveResponsesReasoningContent = entry.preserveResponsesReasoningContent; diff --git a/src/providers/fastwire.ts b/src/providers/fastwire.ts index d7aec93a4a..d36a9cf19f 100644 --- a/src/providers/fastwire.ts +++ b/src/providers/fastwire.ts @@ -1,5 +1,12 @@ -import type { FastWire, OcxProviderConfig, TierDecision } from "../types"; +import type { + AttemptTierOutcome, + FastWire, + OcxProviderConfig, + TierDecision, + TierObservationContext, +} from "../types"; import { MODEL_ADAPTER_OVERRIDE_ALLOWED } from "../types"; +import { sanitizeLogMetadataString } from "../lib/redact"; import type { InboundWire, ModelWireDefault } from "./registry"; const SERVICE_TIER_ADAPTERS = new Set(["openai-chat", "openai-responses"]); @@ -50,6 +57,33 @@ export interface ResolvedFastPolicy { readonly forwardCallerTier: boolean; } +/** Adapter-owned response observer paired with the exact body that adapter serialized. */ +export interface AdapterTierMetadata { + readonly outcome: AttemptTierOutcome; + observeResponseServiceTier(value: unknown): void; + markResponseUnparseable(): void; +} + +/** Detach a FastWire declaration from config or registry ownership. */ +export function cloneFastWire( + value: FastWire | null | undefined, + options: { freeze?: boolean } = {}, +): FastWire | null | undefined { + if (value === null || value === undefined) return value; + const canonicalToWire = { ...value.canonicalToWire }; + const betas = value.betas ? [...value.betas] : undefined; + if (options.freeze) { + Object.freeze(canonicalToWire); + if (betas) Object.freeze(betas); + } + const clone: FastWire = { + ...value, + canonicalToWire, + ...(betas ? { betas } : {}), + }; + return options.freeze ? Object.freeze(clone) : clone; +} + function exactModelValue(record: Readonly>, modelId: string): T | undefined { if (Object.prototype.hasOwnProperty.call(record, modelId)) return record[modelId]; const folded = modelId.toLowerCase(); @@ -81,7 +115,9 @@ function registryDefaultForModel( modelId: string, inbound: InboundWire, ): string | undefined { - const declared = defaults[modelId.trim().toLowerCase()]; + const normalizedModelId = modelId.trim().toLowerCase(); + if (!Object.hasOwn(defaults, normalizedModelId)) return undefined; + const declared = defaults[normalizedModelId]; if (declared === undefined) return undefined; if (typeof declared !== "string" && !declared.inbound.includes(inbound)) return undefined; const wire = typeof declared === "string" ? declared : declared.wire; @@ -95,11 +131,15 @@ function resolvePolicyAdapter( ): { adapter: string; hardPinned: boolean } { // Hard pins and configured overrides deliberately use the same exact-key semantics as // resolveWireProtocolOverride(). Registry defaults alone normalize ids at their boundary. - const hardPin = authority.hardPins[modelId]; - if (hardPin !== undefined) return { adapter: hardPin, hardPinned: true }; + const hardPin = Object.hasOwn(authority.hardPins, modelId) + ? authority.hardPins[modelId] + : undefined; + if (typeof hardPin === "string") return { adapter: hardPin, hardPinned: true }; if (authority.modelWireOverrideAllowed) { - const configured = authority.modelAdapters[modelId]; - if (configured !== undefined && MODEL_ADAPTER_OVERRIDE_ALLOWED.has(configured)) { + const configured = Object.hasOwn(authority.modelAdapters, modelId) + ? authority.modelAdapters[modelId] + : undefined; + if (typeof configured === "string" && MODEL_ADAPTER_OVERRIDE_ALLOWED.has(configured)) { return { adapter: configured, hardPinned: false }; } if (MODEL_ADAPTER_OVERRIDE_ALLOWED.has(authority.providerAdapter)) { @@ -110,13 +150,6 @@ function resolvePolicyAdapter( return { adapter: authority.providerAdapter, hardPinned: false }; } -/** A1's retained Chat serializer gate (`chatServiceTier || exact model true`). */ -export function legacyChatEligibility(authority: FastPolicyAuthority, modelId: string): boolean { - const exact = exactModelValue(authority.capability.models, modelId); - if (authority.capability.provider === false || exact === false) return false; - return authority.capability.chatServiceTier === true || exact === true; -} - export function resolveFastPolicy( authority: FastPolicyAuthority, modelId: string, @@ -131,12 +164,16 @@ export function resolveFastPolicy( ? defaultFastWireForAdapter(adapter) : authority.fastWireDeclaration; const wireAvailable = fastWire !== null && FAST_WIRE_ADAPTERS[fastWire.kind].has(adapter); - const chatEligible = adapter !== "openai-chat" || legacyChatEligibility(authority, modelId); // Explicit null disables Fast injection, but the defensive true+null branch still preserves // a caller tier on an existing OpenAI service-tier wire. const callerWireAvailable = wireAvailable || (fastWire === null && SERVICE_TIER_ADAPTERS.has(adapter)); - const forwardCallerTier = capability !== false && callerWireAvailable && chatEligible; + // On classified routes this permission applies only to a caller's foreign tier: proxy-owned + // canonical Fast has already passed capability validation. On unclassified routes every caller + // tier still needs the final wire's forwarding permission. + const forwardCallerTier = capability !== false + && callerWireAvailable + && (adapter !== "openai-chat" || authority.capability.chatServiceTier === true); let eligibility: ResolvedFastPolicy["eligibility"]; if (capability === false) eligibility = "capability-unsupported"; @@ -145,7 +182,6 @@ export function resolveFastPolicy( ? "pin-unavailable" : "wire-unavailable"; } - else if (!chatEligible) eligibility = "capability-unsupported"; else if (capability === undefined) eligibility = "unclassified"; else eligibility = "eligible"; @@ -157,7 +193,149 @@ export function canonicalFastTierMarker(callerTier: string | undefined): "priori return folded === "priority" || folded === "fast" ? "priority" : undefined; } -/** Pure A1 tier state machine. It never changes a caller spelling on inherit. */ +/** Capture Fast demand before the final A1 serialization action rewrites the parsed tier view. */ +export function tierObservationContext( + policy: ResolvedFastPolicy, + fastMode: boolean | undefined, + callerTier: string | undefined, +): TierObservationContext { + return { + capability: policy.capability, + eligibility: policy.eligibility, + fastWire: policy.fastWire, + demandDecision: fastMode === true ? "force-fast" : fastMode === false ? "force-default" : "inherit", + ...(callerTier !== undefined ? { callerTier } : {}), + }; +} + +function canonicalFromWire( + fastWire: FastWire | null, + wireValue: string, +): string | undefined { + if (!fastWire) return undefined; + for (const [canonical, mapped] of Object.entries(fastWire.canonicalToWire)) { + if (mapped === wireValue) return canonical; + } + return undefined; +} + +function downgradeReasonForUnavailable( + context: TierObservationContext, +): AttemptTierOutcome["fastDowngradeReason"] { + if (context.capability === false || context.eligibility === "capability-unsupported") { + return "route-unsupported"; + } + return "wire-unavailable"; +} + +/** + * Build the mutable observation record only after an adapter has completed serialization. + * `wireKind`/`wireValue` describe the field the adapter actually emitted, never a route guess. + */ +export function createAdapterTierMetadata( + context: TierObservationContext | undefined, + decision: TierDecision | undefined, + wireKind: FastWire["kind"] | null, + wireValue: string | null, +): AdapterTierMetadata | undefined { + if (!context || !decision) return undefined; + + const callerCanonicalFast = canonicalFastTierMarker(context.callerTier) === "priority"; + const callerTierDropped = context.callerTier !== undefined + && !callerCanonicalFast + && wireValue === null; + const callerFastSuppressedByConfig = context.capability !== undefined + && context.demandDecision === "force-default" + && callerCanonicalFast; + const loggedWireValue = wireValue === null ? null : sanitizeLogMetadataString(wireValue); + const outcome: AttemptTierOutcome = { + wireKind, + ...(wireValue === null + ? { wireValue: null } + : loggedWireValue ? { wireValue: loggedWireValue } : {}), + fastOutcome: "unknown", + confirmation: "unknown", + ...(callerTierDropped ? { callerTierDropped: true } : {}), + ...(callerFastSuppressedByConfig ? { callerFastSuppressedByConfig: true } : {}), + }; + + // A0/A1 deliberately make fastMode inert for unclassified routes. Preserve that uncertainty: + // do not infer demand, suppression, or a canonical tier from a verbatim caller passthrough. + if (context.capability === undefined || context.eligibility === "unclassified") { + delete outcome.callerFastSuppressedByConfig; + return { + outcome, + observeResponseServiceTier(value: unknown) { + const sanitized = sanitizeLogMetadataString(value); + if (sanitized) outcome.responseServiceTier = sanitized; + }, + markResponseUnparseable() {}, + }; + } + + const effectiveFastRequested = context.capability === true + && context.fastWire !== null + && (context.demandDecision === "force-fast" + || (context.demandDecision === "inherit" && callerCanonicalFast)); + // Known-unsupported routes still need a downgrade when the caller/config expressed Fast intent, + // but they are deliberately outside the effective-demand calculation above. + const fastIntent = context.demandDecision === "force-fast" + || (context.demandDecision === "inherit" && callerCanonicalFast); + + if (!fastIntent) { + outcome.fastOutcome = "not-requested"; + } else if (!effectiveFastRequested || context.eligibility !== "eligible" || wireValue === null) { + outcome.fastOutcome = "downgraded"; + outcome.fastDowngradeReason = downgradeReasonForUnavailable(context); + outcome.confirmation = "downgraded"; + } else if (canonicalFromWire(context.fastWire, wireValue) === "priority") { + outcome.canonical = "priority"; + outcome.fastOutcome = "applied"; + outcome.confirmation = "assumed"; + } + + const responseCanConfirmFast = effectiveFastRequested + && context.eligibility === "eligible" + && wireValue !== null; + return { + outcome, + observeResponseServiceTier(value: unknown) { + if (typeof value !== "string" || !value.trim()) { + if (value !== undefined && responseCanConfirmFast) { + delete outcome.canonical; + delete outcome.fastDowngradeReason; + outcome.fastOutcome = "unknown"; + outcome.confirmation = "unknown"; + } + return; + } + const sanitized = sanitizeLogMetadataString(value); + if (sanitized) outcome.responseServiceTier = sanitized; + if (!responseCanConfirmFast) return; + if (canonicalFromWire(context.fastWire, value) === "priority") { + outcome.canonical = "priority"; + delete outcome.fastDowngradeReason; + outcome.fastOutcome = "applied"; + outcome.confirmation = "confirmed"; + } else { + delete outcome.canonical; + outcome.fastOutcome = "downgraded"; + outcome.fastDowngradeReason = "response-declined"; + outcome.confirmation = "downgraded"; + } + }, + markResponseUnparseable() { + if (!responseCanConfirmFast) return; + delete outcome.canonical; + delete outcome.fastDowngradeReason; + delete outcome.responseServiceTier; + outcome.fastOutcome = "unknown"; + outcome.confirmation = "unknown"; + }, + }; +} + +/** Pure tier state machine. B1 normalizes canonical Fast on classified inherit routes. */ export function decideTier( policy: ResolvedFastPolicy, fastMode: boolean | undefined, @@ -178,9 +356,16 @@ export function decideTier( : { kind: "drop" }; } if (fastMode === false) return { kind: "drop" }; + const callerCanonicalFast = canonicalFastTierMarker(callerTier); + if (callerCanonicalFast !== undefined) { + const value = policy.fastWire.canonicalToWire[callerCanonicalFast]; + return typeof value === "string" && value.length > 0 + ? { kind: "set", value } + : { kind: "drop" }; + } + if (callerTier !== undefined && !policy.forwardCallerTier) return { kind: "drop" }; if ( callerTier !== undefined - && canonicalFastTierMarker(callerTier) === undefined && policy.fastWire.foreignCallerTiers === "drop" ) { return { kind: "drop" }; diff --git a/src/providers/service-tier.ts b/src/providers/service-tier.ts index b2f81cbb9a..4eebbcbe05 100644 --- a/src/providers/service-tier.ts +++ b/src/providers/service-tier.ts @@ -1,4 +1,4 @@ -import type { FastWire, OcxProviderConfig } from "../types"; +import type { OcxProviderConfig } from "../types"; import { captureWireAdapterHardPins } from "../types"; import { isCanonicalOpenAiForwardProvider } from "./openai-tiers"; import { @@ -8,6 +8,7 @@ import { type ModelWireDefault, } from "./registry"; import { + cloneFastWire, resolveFastPolicy, resolveProviderAuthTransport, type FastPolicyAuthority, @@ -48,16 +49,6 @@ function cloneRegistryWireDefaults( return Object.freeze(clone); } -function cloneFastWire(value: FastWire | null | undefined): FastWire | null | undefined { - if (value === null || value === undefined) return value; - return Object.freeze({ - kind: value.kind, - canonicalToWire: Object.freeze({ ...value.canonicalToWire }), - foreignCallerTiers: value.foreignCallerTiers, - ...(value.betas ? { betas: Object.freeze([...value.betas]) } : {}), - }); -} - /** * Capture every registry-owned input before an asynchronous catalog flight begins. * The resolver itself is pure and never reads the live provider registry. @@ -72,6 +63,7 @@ function buildFastPolicyAuthority( providerAdapter: provider.adapter, fastWireDeclaration: cloneFastWire( provider.fastWire !== undefined ? provider.fastWire : registry?.fastWire, + { freeze: true }, ), modelWireOverrideAllowed: !isCanonicalOpenAiForwardProvider(provider as OcxProviderConfig), authTransport: resolveProviderAuthTransport( @@ -97,7 +89,7 @@ export function captureFastPolicyAuthority( registryTransportMatch: boolean, ): FastPolicyAuthority { const authority = buildFastPolicyAuthority(providerName, provider, registryTransportMatch); - capturedFastPolicyAuthorities.set(provider, authority); + if (Object.isFrozen(provider)) capturedFastPolicyAuthorities.set(provider, authority); return authority; } @@ -127,7 +119,9 @@ function authorityForProvider( registryWireDefaults: Object.freeze({}), }); } - const captured = capturedFastPolicyAuthorities.get(provider); + const captured = Object.isFrozen(provider) + ? capturedFastPolicyAuthorities.get(provider) + : undefined; if (captured) return captured; const registryTransportMatch = providerMatchesRegistryTransport(providerName, provider); const authority = buildFastPolicyAuthority(providerName, provider, registryTransportMatch); @@ -171,16 +165,13 @@ export function supportsServiceTierForModel( return resolveFastPolicy(authority, modelId).capability; } -/** A1 name retained for the legacy Chat serializer gate. */ -export function canSerializeServiceTierForChatModel( +/** Whether a Chat route may forward an arbitrary caller tier rather than canonical Fast. */ +export function canForwardForeignServiceTierForChatModel( provider: Pick, modelId: string, ): boolean { - const exact = supportsServiceTierForModel({ - modelSupportsServiceTier: provider.modelSupportsServiceTier, - }, modelId); - if (provider.supportsServiceTier === false || exact === false) return false; - return provider.chatServiceTier === true || exact === true; + const capability = supportsServiceTierForModel(provider, modelId); + return capability !== false && provider.chatServiceTier === true; } /** Final adapter selected by the Fast policy's four-level wire resolver. */ diff --git a/src/router.ts b/src/router.ts index a25b14add6..723ff8ca84 100644 --- a/src/router.ts +++ b/src/router.ts @@ -13,6 +13,7 @@ import { assertProviderDestinationAllowed } from "./lib/destination-policy"; import { redactSecretString, redactUrlForLog } from "./lib/redact"; import { PROVIDER_REGISTRY, providerCodexAccountMode } from "./providers/registry"; import { applyDirectReasoningEffortContracts, hasLegacyClinePassReasoningEfforts } from "./providers/derive"; +import { cloneFastWire } from "./providers/fastwire"; import { providerMatchesRegistryTransportWithStaticGuards, providerSupportsLiveModelDiscovery, @@ -335,11 +336,7 @@ export function routedProviderConfig(providerName: string, provider: OcxProvider : {}), ...(provider.fastWire === undefined && registryEntry.fastWire !== undefined ? { - fastWire: registryEntry.fastWire === null ? null : { - ...registryEntry.fastWire, - canonicalToWire: { ...registryEntry.fastWire.canonicalToWire }, - ...(registryEntry.fastWire.betas ? { betas: [...registryEntry.fastWire.betas] } : {}), - }, + fastWire: cloneFastWire(registryEntry.fastWire), } : {}), ...(provider.supportsServiceTier === undefined && registryEntry.supportsServiceTier !== undefined diff --git a/src/routing/compatibility/behavior.ts b/src/routing/compatibility/behavior.ts index 87abca2282..553e531bb1 100644 --- a/src/routing/compatibility/behavior.ts +++ b/src/routing/compatibility/behavior.ts @@ -1,6 +1,6 @@ import type { OcxConfig, OcxProviderConfig } from "../../types"; import { PROVIDER_REGISTRY } from "../../providers/registry"; -import { serviceTierSupportForModel } from "../../providers/service-tier"; +import { fastPolicyForModel, serviceTierSupportForModel } from "../../providers/service-tier"; import { resolveProviderAuthTransport } from "../../providers/fastwire"; import { localFingerprint } from "../../lab/digest"; import type { LabBehaviorSource, LabBehaviorValues } from "../../lab/live/types"; @@ -93,6 +93,7 @@ export function resolveProductionBehaviorValues( const project = typeof effective.project === "string" && effective.project ? effective.project : null; const location = typeof effective.location === "string" && effective.location ? effective.location : null; const nativeLocalExec = effective.nativeLocalExec === "on" || effective.unsafeAllowNativeLocalExec === true; + const fastPolicy = fastPolicyForModel(effective, modelId, providerName); const values: LabBehaviorValues = { "wire.adapter": behaviorRow("provider_config", adapter), @@ -113,6 +114,14 @@ export function resolveProductionBehaviorValues( "provider_config", serviceTierSupportForModel(effective, modelId, providerName) ?? null, ), + "responses.fastWireKind": behaviorRow( + "provider_config", + fastPolicy.fastWire?.kind ?? null, + ), + "responses.fastWireValue": behaviorRow( + "provider_config", + fastPolicy.fastWire?.canonicalToWire.priority ?? null, + ), "responses.snapshotRepair": behaviorRow("provider_config", effective.responsesSnapshotRepair === true), "responses.itemIdRepair": behaviorRow("provider_config", effective.responsesItemIdRepair ?? null), "limits.contextWindow": behaviorRow( diff --git a/src/server/management/shared.ts b/src/server/management/shared.ts index 428f7df0eb..77d1f74264 100644 --- a/src/server/management/shared.ts +++ b/src/server/management/shared.ts @@ -92,7 +92,7 @@ export type CostResult = | { kind: "value"; estimate: NonNullable>; estimateReasons: CostEstimateReason[] } | { kind: "unavailable"; reason: MetricUnavailableReason }; -export type MetricSource = Pick & { +export type MetricSource = Pick & { attempts?: readonly PersistedUsageAttempt[]; }; diff --git a/src/server/request-log.ts b/src/server/request-log.ts index c69ea2a110..9658cbd2fa 100644 --- a/src/server/request-log.ts +++ b/src/server/request-log.ts @@ -9,10 +9,11 @@ import { } from "../lib/errors"; import { CODEX_CONFIG_PATH, readRootTomlString } from "../codex/paths"; import { readCodexCatalogPath } from "../codex/catalog"; -import type { OcxUsage } from "../types"; +import type { AttemptTierOutcome, OcxUsage } from "../types"; import { normalizeRouteDecisionTrace, type RouteDecisionTraceV1 } from "../routing/trace"; import type { AdapterRequest } from "../adapters/base"; -import { redactSecretString } from "../lib/redact"; +import type { AdapterTierMetadata } from "../providers/fastwire"; +import { redactSecretString, sanitizeLogMetadataString } from "../lib/redact"; import { appendUsageEntry, isKnownAdmissionKind, @@ -70,12 +71,15 @@ export interface RequestLogContext { effectiveEffort?: string; reasoningWireField?: string; reasoningWireValue?: string | number | boolean; + callerServiceTier?: string; requestedServiceTier?: string; requestedSpeedLabel?: string; configuredServiceTier?: string; configuredSpeedLabel?: string; modelSupportsServiceTier?: boolean; responseServiceTier?: string; + /** Final-attempt tier summary; attempt rows remain the accounting source of truth. */ + tierOutcome?: AttemptTierOutcome; resolvedModel?: string; /** Internal: client-facing response metadata must not replace the physical routed model. */ preserveResolvedModelFromRoute?: boolean; @@ -86,6 +90,8 @@ export interface RequestLogContext { activeAttempt?: PersistedUsageAttempt; /** Internal wall-clock origin for the committed final attempt; never persisted. */ activeAttemptStartedAt?: number; + /** Internal adapter response observer paired with activeAttempt.tierOutcome. */ + activeTierMetadata?: AdapterTierMetadata; usageDebugBodyKind?: UsageDebugBodyKind; usageDebugBodySample?: string; usageDebugContentType?: string; @@ -140,12 +146,14 @@ export interface RequestLogEntry { effectiveEffort?: string; reasoningWireField?: string; reasoningWireValue?: string | number | boolean; + callerServiceTier?: string; requestedServiceTier?: string; requestedSpeedLabel?: string; configuredServiceTier?: string; configuredSpeedLabel?: string; modelSupportsServiceTier?: boolean; responseServiceTier?: string; + tierOutcome?: AttemptTierOutcome; resolvedModel?: string; status: number; durationMs: number; @@ -251,6 +259,7 @@ export function requestLogEntryFromPersistedUsage(entry: PersistedUsageEntry): R ...(entry.effectiveEffort ? { effectiveEffort: entry.effectiveEffort } : {}), ...(entry.reasoningWireField ? { reasoningWireField: entry.reasoningWireField } : {}), ...(entry.reasoningWireValue !== undefined ? { reasoningWireValue: entry.reasoningWireValue } : {}), + ...(entry.callerServiceTier ? { callerServiceTier: entry.callerServiceTier } : {}), ...(entry.requestedServiceTier ? { requestedServiceTier: entry.requestedServiceTier } : {}), ...(entry.requestedSpeedLabel ? { requestedSpeedLabel: entry.requestedSpeedLabel } : {}), ...(entry.configuredServiceTier ? { configuredServiceTier: entry.configuredServiceTier } : {}), @@ -259,6 +268,7 @@ export function requestLogEntryFromPersistedUsage(entry: PersistedUsageEntry): R ? { modelSupportsServiceTier: entry.modelSupportsServiceTier } : {}), ...(entry.responseServiceTier ? { responseServiceTier: entry.responseServiceTier } : {}), + ...(entry.tierOutcome ? { tierOutcome: entry.tierOutcome } : {}), ...(entry.resolvedModel ? { resolvedModel: entry.resolvedModel } : {}), status: entry.status, durationMs: entry.durationMs, @@ -352,6 +362,7 @@ export function addRequestLog(entry: RequestLogEntry) { ...(entry.effectiveEffort ? { effectiveEffort: entry.effectiveEffort } : {}), ...(entry.reasoningWireField ? { reasoningWireField: entry.reasoningWireField } : {}), ...(entry.reasoningWireValue !== undefined ? { reasoningWireValue: entry.reasoningWireValue } : {}), + ...(entry.callerServiceTier ? { callerServiceTier: entry.callerServiceTier } : {}), ...(entry.requestedServiceTier ? { requestedServiceTier: entry.requestedServiceTier } : {}), ...(entry.requestedSpeedLabel ? { requestedSpeedLabel: entry.requestedSpeedLabel } : {}), ...(entry.configuredServiceTier ? { configuredServiceTier: entry.configuredServiceTier } : {}), @@ -360,6 +371,7 @@ export function addRequestLog(entry: RequestLogEntry) { ? { modelSupportsServiceTier: entry.modelSupportsServiceTier } : {}), ...(entry.responseServiceTier ? { responseServiceTier: entry.responseServiceTier } : {}), + ...(entry.tierOutcome ? { tierOutcome: entry.tierOutcome } : {}), status: entry.status, durationMs: entry.durationMs, ...(entry.firstOutputMs !== undefined ? { firstOutputMs: entry.firstOutputMs } : {}), @@ -464,6 +476,35 @@ export function recordAdapterReasoning( } } +/** Attach the serializing adapter's tier observation to the active durable attempt. */ +export function recordAdapterTier( + logCtx: RequestLogContext, + request: AdapterRequest, +): void { + recordAdapterTierMetadata(logCtx, request.tierLog); +} + +/** Attach adapter-owned metadata for transports that expose no AdapterRequest (runTurn). */ +export function recordAdapterTierMetadata( + logCtx: RequestLogContext, + metadata: AdapterTierMetadata | undefined, +): void { + delete logCtx.tierOutcome; + delete logCtx.activeTierMetadata; + const attempt = logCtx.activeAttempt; + if (attempt) delete attempt.tierOutcome; + + try { + const outcome = metadata?.outcome; + if (!metadata || !outcome) return; + logCtx.tierOutcome = outcome; + logCtx.activeTierMetadata = metadata; + if (attempt) attempt.tierOutcome = outcome; + } catch { + // Request logging is best-effort and must not affect request delivery. + } +} + export function requestLogErrorCode( status: number, upstreamError?: string, @@ -552,7 +593,13 @@ export function applyResponseLogMetadata(logCtx: RequestLogContext, payload: unk && model.trim() ) logCtx.resolvedModel = model; const serviceTier = (source as { service_tier?: unknown }).service_tier; - if (typeof serviceTier === "string" && serviceTier.trim()) logCtx.responseServiceTier = serviceTier; + if (typeof serviceTier === "string" && serviceTier.trim()) { + const sanitized = sanitizeLogMetadataString(serviceTier); + if (sanitized) logCtx.responseServiceTier = sanitized; + logCtx.activeTierMetadata?.observeResponseServiceTier(serviceTier); + } else if (Object.prototype.hasOwnProperty.call(source, "service_tier")) { + logCtx.activeTierMetadata?.observeResponseServiceTier(serviceTier); + } const usage = usageFromResponsesPayload((source as { usage?: unknown }).usage); if (usage && !logCtx.usageFromBridge) { logCtx.usage = usage; @@ -618,6 +665,7 @@ export function inspectResponseLogJson(logCtx: RequestLogContext, text: string): try { applyResponseLogMetadata(logCtx, JSON.parse(text)); } catch { + logCtx.activeTierMetadata?.markResponseUnparseable(); /* body may not be JSON; request log metadata is best-effort only */ } captureUpstreamError(logCtx, text); @@ -648,6 +696,7 @@ export function inspectResponseLogSsePayloadParsed( const debugEnabled = isUsageDebugEnabled(); const sseAlreadyMarked = logCtx.usageDebugBodyKind === "sse"; if (parsed !== undefined) applyResponseLogMetadata(logCtx, parsed); + else logCtx.activeTierMetadata?.markResponseUnparseable(); captureUpstreamErrorParsed(logCtx, payload, parsed); if (debugEnabled) { if (!sseAlreadyMarked) { @@ -849,6 +898,7 @@ export function addFinalRequestLog( ...attempt, recoveryKinds: [...attempt.recoveryKinds], ...(attempt.usage ? { usage: { ...attempt.usage } } : {}), + ...(attempt.tierOutcome ? { tierOutcome: { ...attempt.tierOutcome } } : {}), })); const isCombo = logCtx.comboId !== undefined && (attempts?.length ?? 0) > 0; const aggregate = isCombo ? aggregateAttemptUsage(attempts ?? []) : null; @@ -873,12 +923,16 @@ export function addFinalRequestLog( ...(logCtx.effectiveEffort ? { effectiveEffort: logCtx.effectiveEffort } : {}), ...(logCtx.reasoningWireField ? { reasoningWireField: logCtx.reasoningWireField } : {}), ...(logCtx.reasoningWireValue !== undefined ? { reasoningWireValue: logCtx.reasoningWireValue } : {}), + ...(logCtx.callerServiceTier ? { callerServiceTier: logCtx.callerServiceTier } : {}), ...(logCtx.requestedServiceTier ? { requestedServiceTier: logCtx.requestedServiceTier } : {}), ...(logCtx.requestedSpeedLabel ? { requestedSpeedLabel: logCtx.requestedSpeedLabel } : {}), ...(logCtx.configuredServiceTier ? { configuredServiceTier: logCtx.configuredServiceTier } : {}), ...(logCtx.configuredSpeedLabel ? { configuredSpeedLabel: logCtx.configuredSpeedLabel } : {}), ...(logCtx.modelSupportsServiceTier !== undefined ? { modelSupportsServiceTier: logCtx.modelSupportsServiceTier } : {}), ...(logCtx.responseServiceTier ? { responseServiceTier: logCtx.responseServiceTier } : {}), + ...((attempts?.at(-1)?.tierOutcome ?? logCtx.tierOutcome) + ? { tierOutcome: attempts?.at(-1)?.tierOutcome ?? { ...logCtx.tierOutcome! } } + : {}), ...(logCtx.resolvedModel ? { resolvedModel: logCtx.resolvedModel } : {}), status: effectiveStatus, durationMs: Date.now() - start, diff --git a/src/server/responses/core.ts b/src/server/responses/core.ts index d12b54b0d6..978916063e 100644 --- a/src/server/responses/core.ts +++ b/src/server/responses/core.ts @@ -57,7 +57,7 @@ import { injectionDebugLog } from "../../lib/injection-debug-log"; import { resolveClientRetryAfter } from "../../lib/retry-after"; import { enrichOpenCodeZenRateLimitMessage } from "../../providers/opencode-zen-rate-limit"; import { modelInList, namespacedToolName } from "../../types"; -import type { AdapterEvent, OcxConfig, OcxParsedRequest, OcxProviderConfig, OcxProviderContinuationState, OcxUsage } from "../../types"; +import type { AdapterEvent, OcxConfig, OcxParsedRequest, OcxProviderConfig, OcxProviderContinuationState, OcxUsage, TierDecision } from "../../types"; import { forceRefreshOAuthAccessSnapshot, getOAuthCredentialApiBaseUrl, @@ -128,7 +128,13 @@ import { serviceTierSupportFromPolicy, SERVICE_TIER_ADAPTERS, } from "../../providers/service-tier"; -import { decideTier, tierValueAfterDecision, type ResolvedFastPolicy } from "../../providers/fastwire"; +import { + canonicalFastTierMarker, + decideTier, + tierObservationContext, + tierValueAfterDecision, + type ResolvedFastPolicy, +} from "../../providers/fastwire"; import { RequestPacingQueueOverloadError, waitForProviderRequestSlot, @@ -154,7 +160,7 @@ import { shouldAttemptImageTierRetry } from "../image-retry"; import { resolveProviderTransport } from "../../providers/xai-transport"; import type { WsData } from "../ws-bridge"; import { codexAccountSelectionForTurn, registerTurn, trackStreamLifetime, unregisterTurn } from "../lifecycle"; -import { redactSecretString } from "../../lib/redact"; +import { redactSecretString, sanitizeLogMetadataString } from "../../lib/redact"; import { readBoundedResponseBody } from "../../lib/bounded-body"; import type { AdmissionLease } from "../../lib/admission"; import { supportedLadderFor } from "../effort-policy"; @@ -172,6 +178,8 @@ import { noteAttemptSend, readConfiguredCodexServiceTier, recordAdapterReasoning, + recordAdapterTier, + recordAdapterTierMetadata, recordAttemptRequestedEffort, requestLogSpeedLabel, sealRequestAttemptIdentity, @@ -605,6 +613,7 @@ async function retryCodexPoolOnAlternateAccount( translatorBudget: options.translatorBudget, }); recordAdapterReasoning(logCtx, request); + recordAdapterTier(logCtx, request); await firstResponse.body?.cancel().catch(() => undefined); options.onCodexAuthContextResolved?.(retryAuthCtx); @@ -978,8 +987,8 @@ const MAX_FAST_WIRE_CAPABILITY_WARNINGS = 256; const warnedFastWireCapabilityGaps = new Set(); function warnFastWireCapabilityGap(providerName: string, modelId: string): void { - const safeProvider = redactSecretString(providerName); - const safeModel = redactSecretString(modelId); + const safeProvider = sanitizeLogMetadataString(providerName) ?? "unknown"; + const safeModel = sanitizeLogMetadataString(modelId) ?? "unknown"; const key = `${safeProvider}\0${safeModel}`; if (warnedFastWireCapabilityGaps.has(key)) return; if (warnedFastWireCapabilityGaps.size >= MAX_FAST_WIRE_CAPABILITY_WARNINGS) { @@ -1186,6 +1195,7 @@ async function applyFinalRouteRequestNormalization(args: { ); const modelServiceTierSupport = serviceTierSupportFromPolicy(fastPolicy); const callerTier = parsed.options.serviceTier; + parsed.options.tierObservation = tierObservationContext(fastPolicy, config.fastMode, callerTier); parsed.options.tierDecision = decideTier(fastPolicy, config.fastMode, callerTier); parsed.options.serviceTier = tierValueAfterDecision(parsed.options.tierDecision, callerTier); if (fastPolicy.capability === true && fastPolicy.fastWire === null) { @@ -1586,16 +1596,15 @@ function finalizeOwnedTranslatorBudget(response: Response, budget: TranslatorBud * Service-tier capability gate, applied after the final route/wire is settled. A * provider explicitly documented as NOT supporting `service_tier` must never * receive it: strip the field and clear the logging value even when the caller - * supplied one (fail closed). Tri-state contract: `true` supports (injection - * allowed, caller values preserved), `false` strips, and an UNCLASSIFIED custom - * provider (`undefined`) preserves caller-supplied values but never gets an - * injection — deleting the caller's field there would silently change their - * request against a gateway we know nothing about. + * supplied one (fail closed). A policy-produced canonical Fast decision has + * already passed capability validation and cannot be vetoed by Chat's caller + * forwarding permission. On unclassified routes every caller tier remains subject + * to `forwardCallerTier`. */ export function applyServiceTierGate( provider: OcxProviderConfig, rawBody: unknown, - options: { serviceTier?: string }, + options: { serviceTier?: string; tierDecision?: TierDecision }, modelId?: string, providerName?: string, inbound: InboundWire = "responses", @@ -1606,10 +1615,24 @@ export function applyServiceTierGate( // model adapter as well: an explicit override to Anthropic (or another non-OpenAI wire) must // not carry a caller-supplied `service_tier` through a route that cannot forward it. if (modelId === undefined && !SERVICE_TIER_ADAPTERS.has(provider.adapter)) return; + const policy = modelId === undefined + ? undefined + : resolvedPolicy ?? fastPolicyForModel(provider, modelId, providerName, inbound); const forwardCallerTier = modelId === undefined ? provider.supportsServiceTier !== false - : (resolvedPolicy ?? fastPolicyForModel(provider, modelId, providerName, inbound)).forwardCallerTier; - if (forwardCallerTier) return; + : policy!.forwardCallerTier; + const rawTier = rawBody && typeof rawBody === "object" + ? (rawBody as Record).service_tier + : undefined; + const canonicalDecision = options.tierDecision?.kind === "set"; + const callerTierIsForeign = rawTier !== undefined + && (typeof rawTier !== "string" || canonicalFastTierMarker(rawTier) === undefined); + const dropForeignCallerTier = policy?.capability === true + && policy.fastWire?.kind === "service-tier" + && policy.fastWire?.foreignCallerTiers === "drop" + && callerTierIsForeign; + if (policy && policy.capability !== false && canonicalDecision) return; + if (forwardCallerTier && !dropForeignCallerTier) return; if (rawBody && typeof rawBody === "object") { delete (rawBody as Record).service_tier; } @@ -1745,6 +1768,7 @@ async function handleResponsesInner( } logCtx.requestedModel = parsed.modelId; logCtx.requestedEffort = parsed.options.reasoning; + logCtx.callerServiceTier = sanitizeLogMetadataString(parsed.options.serviceTier); logCtx.requestedServiceTier = parsed.options.serviceTier; logCtx.requestedSpeedLabel = requestLogSpeedLabel(parsed.options.serviceTier); logCtx.configuredServiceTier = readConfiguredCodexServiceTier(); @@ -2187,6 +2211,9 @@ async function handleResponsesInner( (logCtx.attempts ??= []).push(attempt); } sealRequestAttemptIdentity(logCtx.activeAttempt, logCtx.provider, adapter.name, logCtx.accountLogLabel); + if (adapter.runTurn) { + recordAdapterTierMetadata(logCtx, adapter.tierLogForRunTurn?.(parsed)); + } // Optional route-identity linkage for attempt correlation (CL-09 consumes it). The slot // resolves to null unless an opt-in subsystem registered a linker, so an install without // routing profiles does no work here and loads no additional module. The non-throwing @@ -2421,6 +2448,7 @@ async function handleResponsesInner( } : undefined; recordAdapterReasoning(logCtx, request); + recordAdapterTier(logCtx, request); const actualHostKey = upstreamHostHealthKey( route.providerName, safeOriginLabel(request.url), @@ -3217,7 +3245,10 @@ async function handleResponsesInner( stallTimeoutSec: config.stallTimeoutSec, waitForRequestSlot: imageProviderFetch.waitForPacing, fetchImpl: imageProviderFetch.unpacedFetch ?? imageProviderFetch, - onRequestBuilt: request => recordAdapterReasoning(logCtx, request), + onRequestBuilt: request => { + recordAdapterReasoning(logCtx, request); + recordAdapterTier(logCtx, request); + }, ...(vidPlan?.timeoutMs ? { videoTimeoutMs: vidPlan.timeoutMs } : {}), onUsage: usage => { // Cursor may assign _cursorConversationId inside the image loop's first runTurn; @@ -3295,7 +3326,10 @@ async function handleResponsesInner( forceEmptyResponseId: true, abortSignal: options.abortSignal, ...(options.onFirstOutput ? { onFirstOutput: options.onFirstOutput } : {}), - onRequestBuilt: request => recordAdapterReasoning(logCtx, request), + onRequestBuilt: request => { + recordAdapterReasoning(logCtx, request); + recordAdapterTier(logCtx, request); + }, onAttemptSend: (recovery?: AttemptRecoveryKind) => noteAttemptSend(logCtx.activeAttempt, logCtx.usageLogInputTokens, recovery), onUsage: usage => { @@ -3565,6 +3599,7 @@ async function handleResponsesInner( try { initialRequest = await activeAdapter.buildRequest(parsed, { headers: selectedForwardHeaders, translatorBudget }); recordAdapterReasoning(logCtx, initialRequest); + recordAdapterTier(logCtx, initialRequest); inputTokenEstimate = typeof initialRequest.usageLog?.inputTokens === "number" ? initialRequest.usageLog.inputTokens : undefined; @@ -3669,6 +3704,7 @@ async function handleResponsesInner( ...(imageTierBias > 0 ? { imageTierBias } : {}), }); recordAdapterReasoning(logCtx, retryRequest); + recordAdapterTier(logCtx, retryRequest); } catch (err) { // A rotated/rebuilt adapter build failure is a request-shaping error, not an // upstream connect failure: tear the abort link down and map it as 400 (no 413 @@ -3991,6 +4027,7 @@ async function handleResponsesInner( ...(imageTierBias > 0 ? { imageTierBias } : {}), }); recordAdapterReasoning(logCtx, continuationRequest); + recordAdapterTier(logCtx, continuationRequest); } catch (err) { // The main body is already streaming, so there is no HTTP error surface: release // any partial body observation and surface the failure as an in-stream error via diff --git a/src/types.ts b/src/types.ts index ada56d8110..d54860a8ba 100644 --- a/src/types.ts +++ b/src/types.ts @@ -303,6 +303,8 @@ export interface OcxRequestOptions { serviceTier?: string; /** Final outbound tier action, resolved after the provider/model wire is settled. */ tierDecision?: TierDecision; + /** Internal B0 observation inputs; adapters combine these with the wire they actually serialize. */ + tierObservation?: TierObservationContext; presencePenalty?: number; frequencyPenalty?: number; /** Responses prompt-cache affinity key. Passthrough preserves it via _rawBody; routed adapters do not consume it unless their upstream wire supports it. */ @@ -1327,6 +1329,36 @@ export interface FastWire { betas?: readonly string[]; } +/** Durable per-attempt service-tier fact produced at the adapter serialization boundary. */ +export interface AttemptTierOutcome { + canonical?: "priority"; + wireKind?: FastWire["kind"] | null; + wireValue?: string | null; + fastOutcome: "not-requested" | "applied" | "downgraded" | "unknown"; + fastDowngradeReason?: "route-unsupported" | "wire-unavailable" | "response-declined"; + callerTierDropped?: boolean; + callerFastSuppressedByConfig?: boolean; + confirmation: "confirmed" | "assumed" | "downgraded" | "unknown"; + responseServiceTier?: string; +} + +/** + * Request-local observation inputs captured before the final tier action mutates the parsed view. + * This is not persisted; the final adapter turns it into AttemptTierOutcome after serialization. + */ +export interface TierObservationContext { + capability: boolean | undefined; + eligibility: + | "eligible" + | "capability-unsupported" + | "unclassified" + | "wire-unavailable" + | "pin-unavailable"; + fastWire: FastWire | null; + demandDecision: "force-fast" | "force-default" | "inherit"; + callerTier?: string; +} + export type TierDecision = | { readonly kind: "forward-caller" } | { readonly kind: "drop" } @@ -1387,14 +1419,14 @@ export interface OcxProviderConfig { */ requiresAdjacentResponsesToolResults?: boolean; /** - * Provider fallback for the OpenAI `service_tier` parameter. On Responses routes this - * is the complete wire opt-in; Chat routes additionally require `chatServiceTier` or an - * exact-model true declaration. - * Tri-state: `true` lets fast mode inject/remove the field (an unset - * fast mode preserves a caller-supplied value); `false` strips the field and + * Provider fallback for canonical Fast capability over an OpenAI `service_tier` wire. + * This pure tri-state feeds catalog publication, routing eligibility, compatibility + * fingerprints, and proxy-owned canonical Fast injection on both Responses and Chat routes. + * Tri-state: `true` lets fast mode inject/remove the canonical field; `false` strips it and * never injects, because an upstream documented as not supporting the parameter - * must not receive it; absent (`undefined`) leaves the provider unclassified — - * caller-supplied values are preserved untouched, and fast mode never injects. + * must not receive it; absent (`undefined`) leaves the provider unclassified — fast mode never + * injects or translates, and caller values pass only under the final wire's forwarding permission. + * On Chat, that CallerTierForward permission is `chatServiceTier`; Responses retains passthrough. * An explicit config value always wins over the registry default. */ supportsServiceTier?: boolean; @@ -1611,13 +1643,16 @@ export interface OcxProviderConfig { */ promptCacheKey?: boolean; /** - * Opt-in: forward `service_tier` to the upstream `/chat/completions` body. + * Opt-in: forward caller `service_tier` values to the upstream `/chat/completions` body. + * On a classified route it governs foreign values (for example `flex`), not proxy-owned + * canonical Fast after capability validation. On an unclassified route it governs every caller + * value, including canonical spellings, because no Fast capability has been validated. * OpenAI-specific extension with the same hazard as `promptCacheKey` — strict backends * reject unknown fields, and 66 registry providers share the `openai-chat` adapter, so a * caller-supplied `service_tier` would otherwise turn working requests into upstream 400s. - * Exact models may opt in through `modelSupportsServiceTier` instead; provider-level - * `supportsServiceTier: false` remains a global denial. Default off; only enable for - * providers that document this parameter on the chat wire. + * Exact-model `true` enables canonical Fast capability but does not grant foreign-tier + * forwarding; provider-level `supportsServiceTier: false` remains a global denial. Default off; + * only enable for providers that document this parameter on the chat wire. */ chatServiceTier?: boolean; /** @@ -1772,9 +1807,15 @@ const ANTHROPIC_WIRE_MODELS: Record> = { "opencode-go": new Set(["minimax-m2.5", "minimax-m2.7", "minimax-m3"]), }; +function anthropicWireModelsForProvider(providerName: string): ReadonlySet | undefined { + return Object.hasOwn(ANTHROPIC_WIRE_MODELS, providerName) + ? ANTHROPIC_WIRE_MODELS[providerName] + : undefined; +} + /** Detached provider-local hard-pin table for pure wire-policy resolution. */ export function captureWireAdapterHardPins(providerName: string): Readonly> { - const models = ANTHROPIC_WIRE_MODELS[providerName]; + const models = anthropicWireModelsForProvider(providerName); if (!models) return Object.freeze({}); return Object.freeze(Object.fromEntries([...models].map(modelId => [modelId, "anthropic"]))); } @@ -1788,7 +1829,7 @@ export function captureWireAdapterHardPins(providerName: string): Readonly, + attempt: Pick, overlays: readonly ExpectedPriceOverlay[] = EXPECTED_PRICE_OVERLAYS, serviceTier?: ServiceTierInput, userOverlays: readonly ExpectedPriceOverlay[] = activeUserCostOverlays(), @@ -438,15 +464,18 @@ export function estimateAttemptCost( if (!tokens) return null; const price = resolveMatchedPrice(attempt.provider, attempt.model, overlays, userOverlays); if (!price) return null; + const attemptServiceTier = attempt.tierOutcome + ? serviceTierContextFromOutcome(attempt.tierOutcome) + : serviceTier; const [tieredCost4, contextTier] = applyContextTier( - price.cost4, attempt.provider, attempt.model, attempt.usage.inputTokens, serviceTier, + price.cost4, attempt.provider, attempt.model, attempt.usage.inputTokens, attemptServiceTier, ); // Exclusive both ways: if the long rate applied, the request was NOT served as // Fast (Fast does not support long context), so the Fast multiplier must not // also apply — otherwise a downgraded request bills at both rates. const [effectiveCost4, multiplier] = contextTier ? [tieredCost4, 1] as const - : applyPriorityMultiplier(tieredCost4, attempt.provider, attempt.model, serviceTier); + : applyPriorityMultiplier(tieredCost4, attempt.provider, attempt.model, attemptServiceTier); return { ordinal: attempt.ordinal, provider: attempt.provider, @@ -465,7 +494,7 @@ export function estimateAttemptCost( * attempt is unpriced or unnormalizable, return null rather than a partial sum. */ export function estimateComboCost( - attempts: readonly Pick[], + attempts: readonly Pick[], overlays: readonly ExpectedPriceOverlay[] = EXPECTED_PRICE_OVERLAYS, serviceTier?: ServiceTierInput, userOverlays: readonly ExpectedPriceOverlay[] = activeUserCostOverlays(), diff --git a/src/usage/log.ts b/src/usage/log.ts index 08cac2d408..a526d8ddf5 100644 --- a/src/usage/log.ts +++ b/src/usage/log.ts @@ -4,8 +4,9 @@ import { join } from "node:path"; import { getConfigDir } from "../config"; import { enforceAppOwnedMemoryBudget } from "../lib/app-owned-memory"; import { recordOwnedConfigPath } from "../lib/config-ownership"; +import { sanitizeLogMetadataString } from "../lib/redact"; import { usageDisplayTotalTokens } from "./totals"; -import type { OcxUsage } from "../types"; +import type { AttemptTierOutcome, OcxUsage } from "../types"; import { normalizeRouteDecisionTrace, type RouteDecisionTraceV1 } from "../routing/trace"; import { CODEX_ACCOUNT_LOG_LABEL_RE } from "../codex/account-label"; @@ -61,6 +62,8 @@ export interface PersistedUsageAttempt { effectiveEffort?: string; reasoningWireField?: string; reasoningWireValue?: string | number | boolean; + /** Adapter-produced tier fact for this physical attempt; absent on pre-B0 rows. */ + tierOutcome?: AttemptTierOutcome; } export interface PersistedUsageEntry { @@ -87,12 +90,16 @@ export interface PersistedUsageEntry { effectiveEffort?: string; reasoningWireField?: string; reasoningWireValue?: string | number | boolean; + /** Raw caller tier captured before routing, sanitized and bounded for durable logs. */ + callerServiceTier?: string; requestedServiceTier?: string; requestedSpeedLabel?: string; configuredServiceTier?: string; configuredSpeedLabel?: string; modelSupportsServiceTier?: boolean; responseServiceTier?: string; + /** Summary of the final physical attempt for dashboard consumers. */ + tierOutcome?: AttemptTierOutcome; status: number; durationMs: number; /** TTFT relative to the request start (WP4); unset for non-streaming/tool-only. */ @@ -218,6 +225,15 @@ const USAGE_STATUSES = new Set([ "estimated", ]); const LAB_ROUTE_SUBJECT_ID_RE = /^[0-9a-f]{64}$/; +const FAST_OUTCOMES = new Set([ + "not-requested", "applied", "downgraded", "unknown", +]); +const TIER_CONFIRMATIONS = new Set([ + "confirmed", "assumed", "downgraded", "unknown", +]); +const FAST_DOWNGRADE_REASONS = new Set>([ + "route-unsupported", "wire-unavailable", "response-declined", +]); export function isLabRouteSubjectId(value: unknown): value is string { return typeof value === "string" && LAB_ROUTE_SUBJECT_ID_RE.test(value); @@ -246,6 +262,53 @@ function normalizeAttemptUsage(raw: unknown): OcxUsage | null { return normalizeUsageValue(usage as unknown as OcxUsage) ?? null; } +function normalizeAttemptTierOutcome(raw: unknown): AttemptTierOutcome | null { + if (!raw || typeof raw !== "object" || Array.isArray(raw)) return null; + const outcome = raw as Record; + if (typeof outcome.fastOutcome !== "string" + || !FAST_OUTCOMES.has(outcome.fastOutcome as AttemptTierOutcome["fastOutcome"]) + || typeof outcome.confirmation !== "string" + || !TIER_CONFIRMATIONS.has(outcome.confirmation as AttemptTierOutcome["confirmation"])) { + return null; + } + if ("canonical" in outcome && outcome.canonical !== "priority") return null; + if ("wireKind" in outcome + && outcome.wireKind !== null + && outcome.wireKind !== "service-tier" + && outcome.wireKind !== "anthropic-speed") return null; + if ("wireValue" in outcome && outcome.wireValue !== null && typeof outcome.wireValue !== "string") return null; + if ("fastDowngradeReason" in outcome + && (typeof outcome.fastDowngradeReason !== "string" + || !FAST_DOWNGRADE_REASONS.has(outcome.fastDowngradeReason as NonNullable))) { + return null; + } + if ("callerTierDropped" in outcome && typeof outcome.callerTierDropped !== "boolean") return null; + if ("callerFastSuppressedByConfig" in outcome + && typeof outcome.callerFastSuppressedByConfig !== "boolean") return null; + if ("responseServiceTier" in outcome && typeof outcome.responseServiceTier !== "string") return null; + const wireValue = sanitizeLogMetadataString(outcome.wireValue); + const responseServiceTier = sanitizeLogMetadataString(outcome.responseServiceTier); + return { + ...(outcome.canonical === "priority" ? { canonical: "priority" as const } : {}), + ...(outcome.wireKind === null || outcome.wireKind === "service-tier" || outcome.wireKind === "anthropic-speed" + ? { wireKind: outcome.wireKind } + : {}), + ...(outcome.wireValue === null + ? { wireValue: null } + : wireValue ? { wireValue } : {}), + fastOutcome: outcome.fastOutcome as AttemptTierOutcome["fastOutcome"], + ...(typeof outcome.fastDowngradeReason === "string" + ? { fastDowngradeReason: outcome.fastDowngradeReason as NonNullable } + : {}), + ...(typeof outcome.callerTierDropped === "boolean" ? { callerTierDropped: outcome.callerTierDropped } : {}), + ...(typeof outcome.callerFastSuppressedByConfig === "boolean" + ? { callerFastSuppressedByConfig: outcome.callerFastSuppressedByConfig } + : {}), + confirmation: outcome.confirmation as AttemptTierOutcome["confirmation"], + ...(responseServiceTier ? { responseServiceTier } : {}), + }; +} + function normalizeUsageAttempt(raw: unknown): PersistedUsageAttempt | null { if (!raw || typeof raw !== "object" || Array.isArray(raw)) return null; const attempt = raw as Record; @@ -272,6 +335,9 @@ function normalizeUsageAttempt(raw: unknown): PersistedUsageAttempt | null { && !isNonNegativeFiniteNumber(attempt.totalTokens)) return null; const usage = "usage" in attempt ? normalizeAttemptUsage(attempt.usage) : undefined; if ("usage" in attempt && usage === null) return null; + const tierOutcome = "tierOutcome" in attempt + ? normalizeAttemptTierOutcome(attempt.tierOutcome) + : undefined; const recoveryKinds = Array.isArray(attempt.recoveryKinds) ? [...new Set(attempt.recoveryKinds.filter( (value): value is AttemptRecoveryKind => typeof value === "string" @@ -321,6 +387,7 @@ function normalizeUsageAttempt(raw: unknown): PersistedUsageAttempt | null { ? { reasoningWireValue: capMetadataString(attempt.reasoningWireValue) } : { reasoningWireValue: attempt.reasoningWireValue } : {}), + ...(tierOutcome ? { tierOutcome } : {}), }; } @@ -357,6 +424,9 @@ export function normalizeUsageEntryForTest(entry: PersistedUsageEntry): Persiste function normalizeUsageEntry(entry: PersistedUsageEntry): PersistedUsageEntry { const attempts = normalizedAttempts(entry.attempts); + const tierOutcome = entry.tierOutcome ? normalizeAttemptTierOutcome(entry.tierOutcome) : undefined; + const callerServiceTier = sanitizeLogMetadataString(entry.callerServiceTier); + const responseServiceTier = sanitizeLogMetadataString(entry.responseServiceTier); const routeDecision = entry.routeDecision ? normalizeRouteDecisionTrace(entry.routeDecision) : undefined; @@ -397,6 +467,7 @@ function normalizeUsageEntry(entry: PersistedUsageEntry): PersistedUsageEntry { ? { reasoningWireValue: capMetadataString(entry.reasoningWireValue) } : { reasoningWireValue: entry.reasoningWireValue } : {}), + ...(callerServiceTier ? { callerServiceTier } : {}), ...(typeof entry.requestedServiceTier === "string" && entry.requestedServiceTier ? { requestedServiceTier: capMetadataString(entry.requestedServiceTier) } : {}), @@ -412,9 +483,8 @@ function normalizeUsageEntry(entry: PersistedUsageEntry): PersistedUsageEntry { ...(typeof entry.modelSupportsServiceTier === "boolean" ? { modelSupportsServiceTier: entry.modelSupportsServiceTier } : {}), - ...(typeof entry.responseServiceTier === "string" && entry.responseServiceTier - ? { responseServiceTier: capMetadataString(entry.responseServiceTier) } - : {}), + ...(responseServiceTier ? { responseServiceTier } : {}), + ...(tierOutcome ? { tierOutcome } : {}), status: entry.status, durationMs: entry.durationMs, ...(isNonNegativeFiniteNumber(entry.firstOutputMs) diff --git a/structure/04_transports-and-sidecars.md b/structure/04_transports-and-sidecars.md index f3287a65f7..28f45ea39d 100644 --- a/structure/04_transports-and-sidecars.md +++ b/structure/04_transports-and-sidecars.md @@ -44,12 +44,17 @@ OpenAI-compatible service-tier support is resolved only after the final provider known. `supportsServiceTier` remains the provider fallback, while the exact `modelSupportsServiceTier` map can override it per upstream model, including an explicit `false`. The catalog and request path share this decision: a routed row publishes `service_tiers` only when -the resolved adapter is capable, and the final-route normalizer applies the same gate to -`service_tier`. `openai-responses` uses the resolved provider/model declaration directly; -`openai-chat` accepts either its provider-wide `chatServiceTier` serializer opt-in or an exact-model -`true` declaration. Exact `false` narrows provider defaults, and provider-level -`supportsServiceTier: false` cannot be reopened. Capability is namespaced by the selected provider -and model; model-name similarity and adapter type alone never opt a gateway in. +the resolved policy is eligible, and the final-route normalizer applies the same gate to +`service_tier`. Both `openai-responses` and `openai-chat` use the resolved provider/model capability +for catalog publication, routing evidence, and fingerprints. Canonical Fast injection additionally +requires a compatible FastWire mapping on the final adapter and an eligible policy. Setting +`fastMode: false` drops it. On classified Chat routes, `chatServiceTier` separately authorizes +foreign caller values; an exact-model `true` does not grant that forwarding permission. On +unclassified Chat routes it gates every caller tier because no canonical Fast capability has been +validated. Exact `false` +narrows provider defaults, and provider-level `supportsServiceTier: false` cannot be reopened. +Capability is namespaced by the selected provider and model; model-name similarity and adapter type +alone never opt a gateway in. `POST /v1/responses/compact` handles remote compaction v1 before the generic `/v1/responses` branch and before the `/v1/*` guard. Unknown `/v1/*` paths return JSON 404 errors instead of falling through @@ -653,9 +658,11 @@ normalization, credential and provider headers, capability-specific fields, and `openaiChatCompletionsUrl()` path. The passthrough builder uses an explicit Chat-field whitelist so messages (including `name` and separate `system`/`developer` entries), Chat token controls, sampling/logprob fields, caller identity/metadata, and caller stream options retain their wire -shape. For streams, caller `stream_options` are merged with mandatory `include_usage: true`. -`service_tier` remains gated by `chatServiceTier: true`; `parallel_tool_calls` is emitted only for -providers opted into parallel tools (or pinned false by the existing provider opt-out contract). +shape. For streams, caller `stream_options` are merged with mandatory `include_usage: true`. On +classified Fast-capable routes, canonical Fast follows the resolved Fast policy and does not require +`chatServiceTier`; foreign caller tiers still require `chatServiceTier: true`, as does every caller +tier on an unclassified Chat route. `parallel_tool_calls` is emitted only for providers opted into +parallel tools (or pinned false by the existing provider opt-out contract). Combo/policy routes and requests that need Responses-only hosted tools, continuation, background, or storage semantics retain the existing Chat -> Responses -> Chat bridge. diff --git a/tests/fastwire-characterization-routing.test.ts b/tests/fastwire-characterization-routing.test.ts index c1f4322296..3286a09c36 100644 --- a/tests/fastwire-characterization-routing.test.ts +++ b/tests/fastwire-characterization-routing.test.ts @@ -1,4 +1,5 @@ import { describe, expect, test } from "bun:test"; +import { applyProviderConfigHints } from "../src/codex/catalog"; import { applyCatalogModelMetadata } from "../src/codex/catalog/effort"; import type { CatalogModel, RawEntry } from "../src/codex/catalog/parsing"; import { candidateCapabilityEvidence } from "../src/routing/capability"; @@ -7,7 +8,9 @@ import { evaluatePolicyProfile } from "../src/routing/evaluator"; import type { OcxConfig, OcxProviderConfig } from "../src/types"; describe("FastWire characterization: routing profile service-tier evidence", () => { - test("require.serviceTier sees supportsServiceTier=true plus chatServiceTier=false as unsupported", () => { + // FastWire #1886 B1 capability semantic migration: Chat caller-forward permission no longer + // downgrades the provider/model capability seen by routing. + test("require.serviceTier accepts supportsServiceTier=true without chatServiceTier", () => { const provider: OcxProviderConfig = { adapter: "openai-chat", baseUrl: "https://chat-no-tier.example.test/v1", @@ -29,22 +32,48 @@ describe("FastWire characterization: routing profile service-tier evidence", () } as OcxConfig; const capability = candidateCapabilityEvidence(config, "chat-no-tier", "model"); - expect(capability.serviceTier).toBe("unsupported"); + expect(capability.serviceTier).toBe("supported"); const result = evaluatePolicyProfile(config, "fast", {}, [{ provider: "chat-no-tier", model: "model", capability, }]); expect(result.candidates[0]).toMatchObject({ - eligible: false, + eligible: true, requirements: [{ id: "service-tier", expected: "supported", - actual: "unsupported", - outcome: "unsatisfied", + actual: "supported", + outcome: "satisfied", }], }); - expect(result.selectedIndex).toBeNull(); + expect(result.selectedIndex).toBe(0); + }); + + test("supportsServiceTier=false remains ineligible without chatServiceTier", () => { + const provider: OcxProviderConfig = { + adapter: "openai-chat", + baseUrl: "https://chat-no-tier.example.test/v1", + supportsServiceTier: false, + }; + const config = { + port: 0, + defaultProvider: "chat-no-tier", + providers: { "chat-no-tier": provider }, + routingProfiles: { + fast: { + candidates: [{ provider: "chat-no-tier", model: "model" }], + require: { serviceTier: "supported" }, + }, + }, + } as OcxConfig; + const capability = candidateCapabilityEvidence(config, "chat-no-tier", "model"); + expect(capability.serviceTier).toBe("unsupported"); + expect(evaluatePolicyProfile(config, "fast", {}, [{ + provider: "chat-no-tier", + model: "model", + capability, + }]).selectedIndex).toBeNull(); }); }); @@ -68,6 +97,24 @@ describe("FastWire characterization: compatibility fingerprint projection", () = supportsServiceTier: false, }, }, + { + label: "chat-supported-without-caller-forward", + expected: true, + provider: { + adapter: "openai-chat", + baseUrl: "https://chat-supported.example.test/v1", + supportsServiceTier: true, + }, + }, + { + label: "chat-explicitly-unsupported", + expected: false, + provider: { + adapter: "openai-chat", + baseUrl: "https://chat-unsupported.example.test/v1", + supportsServiceTier: false, + }, + }, ]; test.each(cases)("projects $label service-tier behavior", ({ label, expected, provider }) => { @@ -129,4 +176,32 @@ describe("FastWire characterization: catalog service-tier bytes", () => { expect(entry).not.toHaveProperty("service_tiers"); expect(entry).not.toHaveProperty("additional_speed_tiers"); }); + + test.each([ + { supportsServiceTier: true, publishesFast: true }, + { supportsServiceTier: false, publishesFast: false }, + ])( + "Chat provider capability=$supportsServiceTier publishes Fast=$publishesFast without chatServiceTier", + ({ supportsServiceTier, publishesFast }) => { + const provider: OcxProviderConfig = { + adapter: "openai-chat", + baseUrl: "https://chat-catalog.example.test/v1", + supportsServiceTier, + }; + const model = applyProviderConfigHints("chat-catalog", provider, { + id: "model", + provider: "chat-catalog", + }); + const entry: RawEntry = {}; + applyCatalogModelMetadata(entry, model); + expect(model.supportsServiceTier).toBe(supportsServiceTier); + if (publishesFast) { + expect(entry.service_tiers).toEqual([expect.objectContaining({ id: "priority", name: "Fast" })]); + expect(entry.additional_speed_tiers).toEqual(["fast"]); + } else { + expect(entry).not.toHaveProperty("service_tiers"); + expect(entry).not.toHaveProperty("additional_speed_tiers"); + } + }, + ); }); diff --git a/tests/fastwire-characterization-wire.test.ts b/tests/fastwire-characterization-wire.test.ts index 5f1efc39bc..378f3d41c2 100644 --- a/tests/fastwire-characterization-wire.test.ts +++ b/tests/fastwire-characterization-wire.test.ts @@ -103,6 +103,7 @@ describe("FastWire characterization: supported-route fastMode tri-state", () => test("a capability-without-wire warning is redacted and throttled per provider/model", async () => { const providerName = `sk-ant-api03-${"A".repeat(40)}`; + const model = `model\n${"x".repeat(100)}`; const warnSpy = spyOn(console, "warn").mockImplementation(() => {}); const provider: OcxProviderConfig = { ...supportedResponsesProvider(), @@ -110,13 +111,15 @@ describe("FastWire characterization: supported-route fastMode tri-state", () => }; try { - await driveResponses({ provider, providerName, callerTier: "flex" }); - await driveResponses({ provider, providerName, callerTier: "flex" }); + await driveResponses({ provider, providerName, model, callerTier: "flex" }); + await driveResponses({ provider, providerName, model, callerTier: "flex" }); const fastWireWarnings = warnSpy.mock.calls .map(call => String(call[0])) .filter(message => message.includes("Fast policy")); expect(fastWireWarnings).toHaveLength(1); expect(fastWireWarnings[0]).not.toContain(providerName); + expect(fastWireWarnings[0]).not.toContain("\n"); + expect(fastWireWarnings[0]).not.toContain("x".repeat(65)); } finally { warnSpy.mockRestore(); } @@ -168,10 +171,12 @@ describe("FastWire characterization: unclassified support matrix", () => { }); describe("FastWire characterization: exact-model Chat tier forwarding", () => { + // FastWire #1886 B1 capability semantic migration: exact capability no longer grants + // permission to forward a caller's foreign Chat tier. test.each(["flex", "turbo-x"])( - "exact model true forwards foreign caller tier %s without chatServiceTier", + "exact model true drops foreign caller tier %s without chatServiceTier", async callerTier => { - const { outboundBody } = await driveResponses({ + const { outboundBody, logCtx } = await driveResponses({ provider: { adapter: "openai-chat", baseUrl: "https://chat.example.test/v1", @@ -181,9 +186,107 @@ describe("FastWire characterization: exact-model Chat tier forwarding", () => { }, callerTier, }); - expect(outboundBody.service_tier).toBe(callerTier); + expect(outboundBody).not.toHaveProperty("service_tier"); + expect(logCtx.tierOutcome).toMatchObject({ + callerTierDropped: true, + fastOutcome: "not-requested", + }); + expect(logCtx.tierOutcome?.canonical).toBeUndefined(); }, ); + + test("chatServiceTier still forwards an exact model's foreign caller tier", async () => { + const { outboundBody } = await driveResponses({ + provider: { + adapter: "openai-chat", + baseUrl: "https://chat.example.test/v1", + authMode: "key", + apiKey: "sk-test", + modelSupportsServiceTier: { model: true }, + chatServiceTier: true, + }, + callerTier: "flex", + }); + expect(outboundBody.service_tier).toBe("flex"); + }); + + test("exact model capability translates canonical caller Fast without foreign-tier permission", async () => { + const { outboundBody, logCtx } = await driveResponses({ + provider: { + adapter: "openai-chat", + baseUrl: "https://chat.example.test/v1", + authMode: "key", + apiKey: "sk-test", + modelSupportsServiceTier: { model: true }, + }, + callerTier: "fast", + }); + expect(outboundBody.service_tier).toBe("priority"); + expect(logCtx.tierOutcome).toMatchObject({ + canonical: "priority", + fastOutcome: "applied", + }); + }); +}); + +describe("FastWire B1: Chat capability and canonical inherit", () => { + test.each([ + { supportsServiceTier: true, expectedTier: "priority", expectedOutcome: "applied" }, + { supportsServiceTier: false, expectedTier: undefined, expectedOutcome: "downgraded" }, + ] as const)( + "provider capability=$supportsServiceTier controls canonical Fast injection without chatServiceTier", + async ({ supportsServiceTier, expectedTier, expectedOutcome }) => { + const { outboundBody, logCtx } = await driveResponses({ + provider: { + adapter: "openai-chat", + baseUrl: "https://chat-capability.example.test/v1", + authMode: "key", + apiKey: "sk-test", + supportsServiceTier, + }, + fastMode: true, + }); + if (expectedTier === undefined) expect(outboundBody).not.toHaveProperty("service_tier"); + else expect(outboundBody.service_tier).toBe(expectedTier); + expect(logCtx.tierOutcome?.fastOutcome).toBe(expectedOutcome); + }, + ); + + test.each(["fast", "FAST", "priority"])( + "supported Chat route normalizes inherited caller %s to priority", + async callerTier => { + const { outboundBody, logCtx } = await driveResponses({ + provider: { + adapter: "openai-chat", + baseUrl: "https://chat-capability.example.test/v1", + authMode: "key", + apiKey: "sk-test", + supportsServiceTier: true, + }, + callerTier, + }); + expect(outboundBody.service_tier).toBe("priority"); + expect(logCtx.tierOutcome).toMatchObject({ + canonical: "priority", + fastOutcome: "applied", + }); + }, + ); + + test("unclassified Chat route drops caller fast without CallerTierForward", async () => { + const { outboundBody, logCtx } = await driveResponses({ + provider: { + adapter: "openai-chat", + baseUrl: "https://chat-unclassified.example.test/v1", + authMode: "key", + apiKey: "sk-test", + }, + callerTier: "fast", + }); + expect(outboundBody).not.toHaveProperty("service_tier"); + expect(logCtx.tierOutcome).toMatchObject({ fastOutcome: "unknown" }); + expect(logCtx.tierOutcome?.canonical).toBeUndefined(); + }); }); describe("FastWire characterization: requestedServiceTier timing", () => { diff --git a/tests/fastwire-observability.test.ts b/tests/fastwire-observability.test.ts new file mode 100644 index 0000000000..058ca6c961 --- /dev/null +++ b/tests/fastwire-observability.test.ts @@ -0,0 +1,579 @@ +import { describe, expect, test } from "bun:test"; +import type { AdapterRequest } from "../src/adapters/base"; +import { createResponsesPassthroughAdapter } from "../src/adapters/openai-responses"; +import { buildBehaviorFingerprintV1 } from "../src/lab/subject/behavior-fingerprint"; +import { sanitizeLogMetadataString } from "../src/lib/redact"; +import { + createAdapterTierMetadata, + type ResolvedFastPolicy, +} from "../src/providers/fastwire"; +import { resolveProductionBehaviorValues } from "../src/routing/compatibility/behavior"; +import { + addFinalRequestLog, + applyResponseLogMetadata, + beginRequestAttempt, + inspectResponseLogJson, + inspectResponseLogSsePayloadParsed, + recordAdapterTier, + type RequestLogContext, + type RequestLogEntry, +} from "../src/server/request-log"; +import { applyServiceTierGate } from "../src/server/responses/core"; +import type { OcxConfig, OcxParsedRequest, TierObservationContext } from "../src/types"; +import { estimateComboCost, serviceTierContextFromOutcome } from "../src/usage/cost"; +import type { ExpectedPriceOverlay } from "../src/usage/expected-prices"; +import { normalizeUsageEntryForTest } from "../src/usage/log"; +import { withTestTranslatorBudget } from "./helpers/translator-budget"; + +const SERVICE_WIRE = { + kind: "service-tier" as const, + canonicalToWire: { priority: "priority" }, + foreignCallerTiers: "verbatim" as const, +}; + +function observation( + overrides: Partial = {}, +): TierObservationContext { + return { + capability: true, + eligibility: "eligible", + fastWire: SERVICE_WIRE, + demandDecision: "force-fast", + ...overrides, + }; +} + +describe("FastWire attempt outcomes", () => { + test("force-default suppresses caller Fast without classifying a downgrade", () => { + const tracker = createAdapterTierMetadata( + observation({ demandDecision: "force-default", callerTier: "priority" }), + { kind: "drop" }, + null, + null, + ); + expect(tracker?.outcome).toEqual({ + wireKind: null, + wireValue: null, + fastOutcome: "not-requested", + callerFastSuppressedByConfig: true, + confirmation: "unknown", + }); + }); + + test("unclassified passthrough stays unknown and ignores Fast config", () => { + const tracker = createAdapterTierMetadata( + observation({ + capability: undefined, + eligibility: "unclassified", + demandDecision: "force-default", + callerTier: "priority", + }), + { kind: "forward-caller" }, + "service-tier", + "priority", + ); + expect(tracker?.outcome).toEqual({ + wireKind: "service-tier", + wireValue: "priority", + fastOutcome: "unknown", + confirmation: "unknown", + }); + }); + + test.each([ + { + label: "route unsupported", + context: observation({ capability: false, eligibility: "capability-unsupported" }), + reason: "route-unsupported", + }, + { + label: "wire unavailable", + context: observation({ fastWire: null, eligibility: "wire-unavailable" }), + reason: "wire-unavailable", + }, + ] as const)("Fast demand records $label downgrade", ({ context, reason }) => { + const tracker = createAdapterTierMetadata(context, { kind: "drop" }, null, null); + expect(tracker?.outcome).toEqual({ + wireKind: null, + wireValue: null, + fastOutcome: "downgraded", + fastDowngradeReason: reason, + confirmation: "downgraded", + }); + }); + + test("foreign-tier drop records only callerTierDropped", () => { + const tracker = createAdapterTierMetadata( + observation({ demandDecision: "inherit", callerTier: "flex" }), + { kind: "drop" }, + null, + null, + ); + expect(tracker?.outcome).toEqual({ + wireKind: null, + wireValue: null, + fastOutcome: "not-requested", + callerTierDropped: true, + confirmation: "unknown", + }); + }); + + test("confirmation covers assumed, confirmed, downgraded, and unknown", () => { + const assumed = createAdapterTierMetadata( + observation(), + { kind: "set", value: "priority" }, + "service-tier", + "priority", + )!; + expect(assumed.outcome).toMatchObject({ + canonical: "priority", + fastOutcome: "applied", + confirmation: "assumed", + }); + + const confirmed = createAdapterTierMetadata( + observation({ + fastWire: { ...SERVICE_WIRE, canonicalToWire: { priority: "performance" } }, + }), + { kind: "set", value: "performance" }, + "service-tier", + "performance", + )!; + confirmed.observeResponseServiceTier("performance"); + expect(confirmed.outcome).toMatchObject({ + canonical: "priority", + fastOutcome: "applied", + confirmation: "confirmed", + responseServiceTier: "performance", + }); + expect(serviceTierContextFromOutcome(confirmed.outcome)).toEqual({ + responseServiceTier: "priority", + }); + + const declined = createAdapterTierMetadata( + observation(), + { kind: "set", value: "priority" }, + "service-tier", + "priority", + )!; + declined.observeResponseServiceTier("default"); + expect(declined.outcome).toMatchObject({ + fastOutcome: "downgraded", + fastDowngradeReason: "response-declined", + confirmation: "downgraded", + responseServiceTier: "default", + }); + expect(declined.outcome.canonical).toBeUndefined(); + + const unknown = createAdapterTierMetadata( + observation(), + { kind: "set", value: "priority" }, + "service-tier", + "priority", + )!; + unknown.markResponseUnparseable(); + expect(unknown.outcome).toMatchObject({ fastOutcome: "unknown", confirmation: "unknown" }); + expect(unknown.outcome.canonical).toBeUndefined(); + }); +}); + +describe("FastWire logging and persistence", () => { + test("the serializing adapter returns metadata for the exact emitted tier", () => { + const rawBody = { model: "gpt-5.6-sol", input: "ping", service_tier: "flex" }; + const parsed: OcxParsedRequest = { + modelId: "gpt-5.6-sol", + context: { messages: [] }, + stream: true, + options: { + serviceTier: "priority", + tierDecision: { kind: "set", value: "priority" }, + tierObservation: observation({ callerTier: "flex" }), + }, + _rawBody: rawBody, + }; + const adapter = withTestTranslatorBudget(createResponsesPassthroughAdapter({ + adapter: "openai-responses", + baseUrl: "https://example.test/v1", + authMode: "key", + apiKey: "sk-test", + })); + const request = adapter.buildRequest(parsed) as AdapterRequest; + + expect(JSON.parse(request.body).service_tier).toBe("priority"); + expect(request.tierLog?.outcome).toMatchObject({ + canonical: "priority", + wireKind: "service-tier", + wireValue: "priority", + fastOutcome: "applied", + confirmation: "assumed", + }); + expect(parsed._rawBody).toBe(rawBody); + expect(rawBody.service_tier).toBe("flex"); + }); + + test("adapter metadata lands on its attempt and final-attempt summary", () => { + const tracker = createAdapterTierMetadata( + observation(), + { kind: "set", value: "priority" }, + "service-tier", + "priority", + )!; + const attempt = beginRequestAttempt(1, "openai", "gpt-5.6-sol", "openai-responses"); + const logCtx: RequestLogContext = { + model: "gpt-5.6-sol", + provider: "openai", + activeAttempt: attempt, + activeAttemptStartedAt: Date.now(), + attempts: [attempt], + }; + recordAdapterTier(logCtx, { + url: "https://example.test/v1/responses", + method: "POST", + headers: {}, + body: "{}", + tierLog: tracker, + } satisfies AdapterRequest); + applyResponseLogMetadata(logCtx, { response: { service_tier: "priority" } }); + + let logged: RequestLogEntry | undefined; + addFinalRequestLog("ocx-tier", Date.now(), logCtx, 200, undefined, entry => { + logged = entry; + }); + expect(logged?.attempts?.[0]?.tierOutcome).toMatchObject({ + fastOutcome: "applied", + confirmation: "confirmed", + responseServiceTier: "priority", + }); + expect(logged?.tierOutcome).toEqual(logged?.attempts?.[0]?.tierOutcome); + }); + + test.each([ + { + label: "JSON", + inspect: (logCtx: RequestLogContext) => inspectResponseLogJson(logCtx, "not-json"), + }, + { + label: "SSE", + inspect: (logCtx: RequestLogContext) => { + inspectResponseLogSsePayloadParsed(logCtx, "not-json", undefined); + }, + }, + ])("$label inspection marks an unparseable response outcome unknown", ({ inspect }) => { + const tracker = createAdapterTierMetadata( + observation(), + { kind: "set", value: "priority" }, + "service-tier", + "priority", + )!; + const attempt = beginRequestAttempt(1, "openai", "gpt-5.6-sol", "openai-responses"); + const logCtx: RequestLogContext = { + model: "gpt-5.6-sol", + provider: "openai", + activeAttempt: attempt, + activeAttemptStartedAt: Date.now(), + attempts: [attempt], + }; + recordAdapterTier(logCtx, { + url: "https://example.test/v1/responses", + method: "POST", + headers: {}, + body: "{}", + tierLog: tracker, + } satisfies AdapterRequest); + expect(attempt.tierOutcome).toMatchObject({ + canonical: "priority", + fastOutcome: "applied", + confirmation: "assumed", + }); + + inspect(logCtx); + + expect(attempt.tierOutcome).toMatchObject({ + fastOutcome: "unknown", + confirmation: "unknown", + }); + expect(attempt.tierOutcome).not.toHaveProperty("canonical"); + }); + + test("old attempts remain valid and new outcomes survive normalization", () => { + const oldAttempt = { + ordinal: 1, + provider: "openai", + model: "gpt-5.6-sol", + adapter: "openai-responses", + status: 200, + durationMs: 1, + sendCount: 1, + recoveryKinds: [], + usageStatus: "reported" as const, + usage: { inputTokens: 10, outputTokens: 1 }, + }; + const normalized = normalizeUsageEntryForTest({ + requestId: "ocx-old", + timestamp: 1, + provider: "openai", + model: "gpt-5.6-sol", + status: 200, + durationMs: 1, + usageStatus: "reported", + attempts: [oldAttempt, { + ...oldAttempt, + ordinal: 2, + tierOutcome: { + canonical: "priority", + wireKind: "service-tier", + wireValue: "priority", + fastOutcome: "applied", + confirmation: "assumed", + }, + }], + }); + expect(normalized.attempts?.[0]).not.toHaveProperty("tierOutcome"); + expect(normalized.attempts?.[1]?.tierOutcome).toMatchObject({ + canonical: "priority", + fastOutcome: "applied", + confirmation: "assumed", + }); + }); + + test("callerServiceTier is trimmed, control-filtered, redacted, and capped", () => { + const secret = ["sk", "proj", "abcdefghijklmnopqrstuvwxyz0123456789"].join("-"); + const sanitized = sanitizeLogMetadataString( + ` \u0000authorization: Bearer ${secret}\n\u0085\u2028\u2029${"x".repeat(80)} `, + ); + expect(sanitized).not.toContain(secret); + expect(sanitized).not.toMatch(/[\u0000-\u001f\u007f-\u009f\u2028\u2029]/); + expect(sanitized?.length).toBeLessThanOrEqual(64); + + const normalized = normalizeUsageEntryForTest({ + requestId: "ocx-caller-tier", + timestamp: 1, + provider: "openai", + model: "gpt-5.6-sol", + callerServiceTier: ` priority\n${"y".repeat(100)} `, + status: 200, + durationMs: 1, + usageStatus: "unreported", + }); + expect(normalized.callerServiceTier).toBe(`priority${"y".repeat(56)}`); + }); + + test("upstream service tiers are sanitized before live and durable logging", () => { + const secret = ["sk", "proj", "upstream", "A".repeat(40)].join("-"); + const rawTier = ` authorization: Bearer ${secret}\n\u0085\u2028\u2029${"x".repeat(80)} `; + const expected = sanitizeLogMetadataString(rawTier)!; + const tracker = createAdapterTierMetadata( + observation({ capability: undefined, eligibility: "unclassified" }), + { kind: "forward-caller" }, + "service-tier", + "priority", + )!; + const attempt = beginRequestAttempt(1, "openai", "gpt-5.6-sol", "openai-responses"); + const logCtx: RequestLogContext = { + model: "gpt-5.6-sol", + provider: "openai", + activeAttempt: attempt, + activeAttemptStartedAt: Date.now(), + attempts: [attempt], + }; + recordAdapterTier(logCtx, { + url: "https://example.test/v1/responses", + method: "POST", + headers: {}, + body: "{}", + tierLog: tracker, + } satisfies AdapterRequest); + + applyResponseLogMetadata(logCtx, { response: { service_tier: rawTier } }); + expect(logCtx.responseServiceTier).toBe(expected); + expect(attempt.tierOutcome?.responseServiceTier).toBe(expected); + + const normalized = normalizeUsageEntryForTest({ + requestId: "ocx-upstream-tier", + timestamp: 1, + provider: "openai", + model: "gpt-5.6-sol", + responseServiceTier: rawTier, + tierOutcome: { + wireKind: "service-tier", + wireValue: "priority", + fastOutcome: "unknown", + confirmation: "unknown", + responseServiceTier: rawTier, + }, + status: 200, + durationMs: 1, + usageStatus: "unreported", + }); + expect(normalized.responseServiceTier).toBe(expected); + expect(normalized.tierOutcome?.responseServiceTier).toBe(expected); + }); +}); + +describe("FastWire per-attempt cost", () => { + const overlays: ExpectedPriceOverlay[] = [{ + provider: "openai", + modelId: "gpt-5.6-sol", + cost4: { input: 5, output: 30, cacheRead: 0.5, cacheWrite: 6.25 }, + source: "test", + verifiedAt: "2026-08-17", + status: "verified", + }]; + const usage = { inputTokens: 200_000, outputTokens: 20_000 }; + + test("an unknown unclassified outcome prices from the serialized caller tier", () => { + const outcome = { + wireKind: "service-tier" as const, + wireValue: "priority", + fastOutcome: "unknown" as const, + confirmation: "unknown" as const, + }; + expect(serviceTierContextFromOutcome(outcome)).toEqual({ + requestedServiceTier: "priority", + }); + + const estimate = estimateComboCost([ + { + ordinal: 1, + provider: "openai", + model: "gpt-5.6-sol", + usageStatus: "reported", + usage, + tierOutcome: outcome, + }, + ], overlays, { requestedServiceTier: "priority" })!; + expect(estimate.priorityMultiplier).toBe(2); + expect(estimate.cost.total).toBeCloseTo(3.2, 9); + }); + + test("combo prices each attempt from its own outcome before the top-level tier", () => { + const attempts = [ + { + ordinal: 1, + provider: "openai", + model: "gpt-5.6-sol", + usageStatus: "reported" as const, + usage, + tierOutcome: { + canonical: "priority" as const, + wireKind: "service-tier" as const, + wireValue: "priority", + fastOutcome: "applied" as const, + confirmation: "confirmed" as const, + responseServiceTier: "priority", + }, + }, + { + ordinal: 2, + provider: "openai", + model: "gpt-5.6-sol", + usageStatus: "reported" as const, + usage, + tierOutcome: { + wireKind: "service-tier" as const, + wireValue: "priority", + fastOutcome: "downgraded" as const, + fastDowngradeReason: "response-declined" as const, + confirmation: "downgraded" as const, + responseServiceTier: "default", + }, + }, + ]; + const estimate = estimateComboCost( + attempts, + overlays, + { requestedServiceTier: "priority" }, + )!; + expect(estimate.attempts?.[0]?.cost.total).toBeCloseTo(3.2, 9); + expect(estimate.attempts?.[1]?.cost.total).toBeCloseTo(1.6, 9); + expect(estimate.cost.total).toBeCloseTo(4.8, 9); + expect(estimate.cost.total).not.toBeCloseTo(6.4, 9); + }); + + test("old attempts without outcomes retain the top-level fallback", () => { + const estimate = estimateComboCost([ + { ordinal: 1, provider: "openai", model: "gpt-5.6-sol", usageStatus: "reported", usage }, + { ordinal: 2, provider: "openai", model: "gpt-5.6-sol", usageStatus: "reported", usage }, + ], overlays, { requestedServiceTier: "priority" })!; + expect(estimate.cost.total).toBeCloseTo(6.4, 9); + expect(estimate.attempts?.every(attempt => attempt.priorityMultiplier === 2)).toBe(true); + }); +}); + +describe("FastWire gate and compatibility fingerprint", () => { + test("foreignCallerTiers is value-aware while unclassified passthrough stays unchanged", () => { + const dropPolicy: ResolvedFastPolicy = { + capability: true, + eligibility: "eligible", + adapter: "openai-responses", + fastWire: { ...SERVICE_WIRE, foreignCallerTiers: "drop" }, + forwardCallerTier: true, + }; + const droppedBody: Record = { service_tier: "flex" }; + const droppedOptions = { serviceTier: "flex" }; + applyServiceTierGate( + { adapter: "openai-responses", baseUrl: "https://example.test", supportsServiceTier: true }, + droppedBody, + droppedOptions, + "model", + "fixture", + "responses", + dropPolicy, + ); + expect(droppedBody).not.toHaveProperty("service_tier"); + expect(droppedOptions.serviceTier).toBeUndefined(); + + const unknownPolicy: ResolvedFastPolicy = { ...dropPolicy, capability: undefined, eligibility: "unclassified" }; + const passthroughBody = { service_tier: "flex" }; + const passthroughOptions = { serviceTier: "flex" }; + applyServiceTierGate( + { adapter: "openai-responses", baseUrl: "https://example.test" }, + passthroughBody, + passthroughOptions, + "model", + "fixture", + "responses", + unknownPolicy, + ); + expect(passthroughBody.service_tier).toBe("flex"); + expect(passthroughOptions.serviceTier).toBe("flex"); + }); + + test("Fast wire projections change the digest without changing serviceTier projection", () => { + const config = { + port: 10100, + defaultProvider: "fixture", + providers: { + fixture: { + adapter: "openai-responses", + baseUrl: "https://example.test/v1", + supportsServiceTier: true, + }, + }, + } as OcxConfig; + const base = resolveProductionBehaviorValues( + config, + "fixture", + "model", + config.providers.fixture!, + "salt", + )!; + const performanceProvider = { + ...config.providers.fixture!, + fastWire: { ...SERVICE_WIRE, canonicalToWire: { priority: "performance" } }, + }; + const performance = resolveProductionBehaviorValues( + { ...config, providers: { fixture: performanceProvider } }, + "fixture", + "model", + performanceProvider, + "salt", + )!; + + expect(base["responses.serviceTier"]).toEqual(performance["responses.serviceTier"]); + expect(base["responses.fastWireKind"]?.value).toBe("service-tier"); + expect(base["responses.fastWireValue"]?.value).toBe("priority"); + expect(performance["responses.fastWireValue"]?.value).toBe("performance"); + expect(buildBehaviorFingerprintV1(base)).not.toBe(buildBehaviorFingerprintV1(performance)); + }); +}); diff --git a/tests/fastwire-policy.test.ts b/tests/fastwire-policy.test.ts index bcd94c8653..7aed643290 100644 --- a/tests/fastwire-policy.test.ts +++ b/tests/fastwire-policy.test.ts @@ -4,16 +4,23 @@ import { createResponsesPassthroughAdapter } from "../src/adapters/openai-respon import { validateConfigCandidate } from "../src/config"; import { canonicalFastTierMarker, + cloneFastWire, decideTier, - legacyChatEligibility, resolveFastPolicy, tierValueAfterDecision, type FastPolicyAuthority, type ResolvedFastPolicy, } from "../src/providers/fastwire"; -import { fastPolicyForModel } from "../src/providers/service-tier"; +import { captureFastPolicyAuthority, fastPolicyForModel } from "../src/providers/service-tier"; import { PROVIDER_REGISTRY, providerRegistryFastWireError } from "../src/providers/registry"; -import type { FastWire, OcxConfig, OcxParsedRequest, TierDecision } from "../src/types"; +import { + captureWireAdapterHardPins, + isWirePinnedModel, + type FastWire, + type OcxConfig, + type OcxParsedRequest, + type TierDecision, +} from "../src/types"; import { withTestTranslatorBudget } from "./helpers/translator-budget"; const MODEL = "model"; @@ -32,7 +39,7 @@ function authorityForMatrix(args: { declaration: DeclarationState; overrideAllowed: boolean; capability: CapabilityState; - legacyChatEligible: boolean; + chatForeignTierForward: boolean; }): FastPolicyAuthority { const providerAdapter = args.source === "provider-adapter" ? "openai-chat" : "openai-responses"; return { @@ -45,7 +52,7 @@ function authorityForMatrix(args: { capability: { ...(args.capability === "undefined" ? {} : { provider: args.capability === "true" }), models: {}, - chatServiceTier: args.legacyChatEligible, + chatServiceTier: args.chatForeignTierForward, }, modelAdapters: args.source === "hard-pin" || args.source === "override" ? { [MODEL]: args.source === "override" ? "openai-chat" : "openai-responses" } @@ -61,12 +68,12 @@ const policyMatrix = (["undefined", "null", "explicit"] as const).flatMap(declar ([false, true] as const).flatMap(overrideAllowed => (["hard-pin", "override", "registry-default", "provider-adapter"] as const).flatMap(source => (["false", "undefined", "true"] as const).flatMap(capability => - ([false, true] as const).map(legacyChatEligible => ({ + ([false, true] as const).map(chatForeignTierForward => ({ declaration, overrideAllowed, source, capability, - legacyChatEligible, + chatForeignTierForward, })), ), ), @@ -75,7 +82,7 @@ const policyMatrix = (["undefined", "null", "explicit"] as const).flatMap(declar describe("resolveFastPolicy matrix", () => { test.each(policyMatrix)( - "$declaration declaration, overrideAllowed=$overrideAllowed, $source, capability=$capability, legacy=$legacyChatEligible", + "$declaration declaration, overrideAllowed=$overrideAllowed, $source, capability=$capability, chatForeign=$chatForeignTierForward", row => { const authority = authorityForMatrix(row); const policy = resolveFastPolicy(authority, MODEL); @@ -85,15 +92,12 @@ describe("resolveFastPolicy matrix", () => { : overrideCanWin ? "openai-chat" : row.source === "provider-adapter" ? "openai-chat" : "openai-responses"; const capability = row.capability === "undefined" ? undefined : row.capability === "true"; - const chatEligible = expectedAdapter !== "openai-chat" || row.legacyChatEligible; const wireAvailable = row.declaration !== "null"; const expectedEligibility: ResolvedFastPolicy["eligibility"] = capability === false ? "capability-unsupported" : !wireAvailable ? "wire-unavailable" - : !chatEligible - ? "capability-unsupported" - : capability === undefined ? "unclassified" : "eligible"; + : capability === undefined ? "unclassified" : "eligible"; expect(policy.adapter).toBe(expectedAdapter); expect(policy.capability).toBe(capability); @@ -101,7 +105,10 @@ describe("resolveFastPolicy matrix", () => { expect(policy.fastWire === null ? null : policy.fastWire?.kind).toBe( row.declaration === "null" ? null : "service-tier", ); - expect(policy.forwardCallerTier).toBe(capability !== false && chatEligible); + expect(policy.forwardCallerTier).toBe( + capability !== false + && (expectedAdapter !== "openai-chat" || row.chatForeignTierForward), + ); }, ); @@ -112,7 +119,7 @@ describe("resolveFastPolicy matrix", () => { declaration: "undefined", overrideAllowed: true, capability: "true", - legacyChatEligible: true, + chatForeignTierForward: true, }), providerAdapter: "openai-responses", registryWireDefaults: { [MODEL]: { wire: "openai-chat", inbound: ["chat"] } }, @@ -128,7 +135,7 @@ describe("resolveFastPolicy matrix", () => { declaration: "undefined", overrideAllowed: true, capability: "true", - legacyChatEligible: true, + chatForeignTierForward: true, }), providerAdapter: "openai-responses", modelAdapters: { Model: "openai-chat" }, @@ -147,7 +154,7 @@ describe("resolveFastPolicy matrix", () => { declaration: "undefined", overrideAllowed: true, capability: "true", - legacyChatEligible: true, + chatForeignTierForward: true, }), modelAdapters: { [MODEL]: "anthropic" }, registryWireDefaults: { [MODEL]: "openai-chat" }, @@ -162,7 +169,7 @@ describe("resolveFastPolicy matrix", () => { declaration: "undefined", overrideAllowed: true, capability: "true", - legacyChatEligible: true, + chatForeignTierForward: true, }), providerAdapter: "anthropic", registryWireDefaults: { [MODEL]: "openai-chat" }, @@ -177,7 +184,7 @@ describe("resolveFastPolicy matrix", () => { declaration: "explicit", overrideAllowed: true, capability: "true", - legacyChatEligible: true, + chatForeignTierForward: true, }), fastWireDeclaration: { kind: "anthropic-speed", @@ -196,13 +203,62 @@ describe("resolveFastPolicy matrix", () => { declaration: "explicit", overrideAllowed: true, capability: "true", - legacyChatEligible: true, + chatForeignTierForward: true, }), hardPins: { [MODEL]: "anthropic" }, }, MODEL); expect(policy).toMatchObject({ adapter: "anthropic", eligibility: "pin-unavailable" }); }); + test("an explicitly disabled wire reports wire-unavailable even when hard pinned", () => { + const policy = resolveFastPolicy({ + ...authorityForMatrix({ + source: "provider-adapter", + declaration: "null", + overrideAllowed: true, + capability: "true", + chatForeignTierForward: true, + }), + hardPins: { [MODEL]: "anthropic" }, + }, MODEL); + expect(policy).toMatchObject({ adapter: "anthropic", eligibility: "wire-unavailable" }); + }); + + test("mutable providers rebuild authority after a capture", () => { + const provider = { + adapter: "openai-responses", + baseUrl: "https://fixture.example/v1", + supportsServiceTier: true, + }; + expect(captureFastPolicyAuthority("fixture", provider, false).capability.provider).toBe(true); + provider.supportsServiceTier = false; + expect(fastPolicyForModel(provider, MODEL, "fixture").capability).toBe(false); + }); + + test("prototype-named providers and models use only own wire-policy rows", () => { + expect(captureWireAdapterHardPins("toString")).toEqual({}); + expect(isWirePinnedModel("toString", MODEL)).toBe(false); + const authority = authorityForMatrix({ + source: "provider-adapter", + declaration: "undefined", + overrideAllowed: true, + capability: "true", + chatForeignTierForward: true, + }); + const responsesAuthority = { ...authority, providerAdapter: "openai-responses" }; + expect(resolveFastPolicy(responsesAuthority, "constructor")).toMatchObject({ + adapter: "openai-responses", + eligibility: "eligible", + }); + expect(resolveFastPolicy({ + ...responsesAuthority, + hardPins: Object.fromEntries([["constructor", "anthropic"]]), + }, "constructor")).toMatchObject({ + adapter: "anthropic", + eligibility: "pin-unavailable", + }); + }); + test("a missing provider name preserves the legacy provider-adapter short circuit", () => { const provider = { adapter: "anthropic", @@ -223,51 +279,6 @@ describe("resolveFastPolicy matrix", () => { }); }); -describe("legacyChatEligibility", () => { - test.each([ - { - label: "chatServiceTier opt-in", - provider: undefined, - models: {}, - chatServiceTier: true, - expected: true, - }, - { - label: "case-insensitive exact-model opt-in", - provider: undefined, - models: { MODEL: true }, - chatServiceTier: false, - expected: true, - }, - { - label: "provider false closes an exact-model opt-in", - provider: false, - models: { model: true }, - chatServiceTier: true, - expected: false, - }, - { - label: "exact false closes a provider Chat opt-in", - provider: true, - models: { model: false }, - chatServiceTier: true, - expected: false, - }, - ])("$label", ({ provider, models, chatServiceTier, expected }) => { - const authority = authorityForMatrix({ - source: "provider-adapter", - declaration: "undefined", - overrideAllowed: true, - capability: "undefined", - legacyChatEligible: false, - }); - expect(legacyChatEligibility({ - ...authority, - capability: { ...(provider === undefined ? {} : { provider }), models, chatServiceTier }, - }, MODEL)).toBe(expected); - }); -}); - const tierGrid = ([false, undefined, true] as const).flatMap(support => ([false, undefined, true] as const).flatMap(fastMode => (["priority", "fast", "flex", undefined] as const).map(callerTier => ({ @@ -296,10 +307,14 @@ describe("TierDecision state machine", () => { const expectedValue = support === false ? undefined : support === undefined ? callerTier - : fastMode === true ? "priority" : fastMode === false ? undefined : callerTier; + : fastMode === true ? "priority" : fastMode === false ? undefined + : callerTier === "fast" ? "priority" : callerTier; + const inheritedCanonicalFast = support === true + && fastMode === undefined + && (callerTier === "priority" || callerTier === "fast"); const expectedKind: TierDecision["kind"] = support === false || (support === true && fastMode === false) ? "drop" - : support === true && fastMode === true ? "set" : "forward-caller"; + : support === true && (fastMode === true || inheritedCanonicalFast) ? "set" : "forward-caller"; expect(decision.kind).toBe(expectedKind); expect(tierValueAfterDecision(decision, callerTier)).toBe(expectedValue); expect(canonicalFastTierMarker(callerTier)).toBe( @@ -322,14 +337,16 @@ describe("TierDecision state machine", () => { expect(tierValueAfterDecision(decision, callerTier)).toBe(callerTier); }); - test.each(["Priority", "FAST", " fast "])("normalizes %s only into an internal marker", callerTier => { + test.each(["Priority", "FAST", " fast "])("normalizes inherited canonical spelling %s to the wire value", callerTier => { expect(canonicalFastTierMarker(callerTier)).toBe("priority"); - expect(tierValueAfterDecision({ kind: "forward-caller" }, callerTier)).toBe(callerTier); + const decision = decideTier(tierPolicy(true), undefined, callerTier); + expect(decision).toEqual({ kind: "set", value: "priority" }); + expect(tierValueAfterDecision(decision, callerTier)).toBe("priority"); }); test.each([ - { callerTier: "priority", expected: { kind: "forward-caller" } }, - { callerTier: "fast", expected: { kind: "forward-caller" } }, + { callerTier: "priority", expected: { kind: "set", value: "priority" } }, + { callerTier: "fast", expected: { kind: "set", value: "priority" } }, { callerTier: "flex", expected: { kind: "drop" } }, { callerTier: undefined, expected: { kind: "forward-caller" } }, ])("foreign-tier drop policy resolves caller=$callerTier to $expected.kind", ({ callerTier, expected }) => { @@ -339,12 +356,31 @@ describe("TierDecision state machine", () => { }, undefined, callerTier)).toEqual(expected); }); - test("unclassified capability keeps the full caller passthrough contract", () => { + test("Chat foreign-tier permission is independent from canonical Fast", () => { + const policy = { ...tierPolicy(true), adapter: "openai-chat", forwardCallerTier: false }; + expect(decideTier(policy, undefined, "flex")).toEqual({ kind: "drop" }); + expect(decideTier(policy, undefined, "fast")).toEqual({ kind: "set", value: "priority" }); + }); + + test("unclassified Responses capability keeps the full caller passthrough contract", () => { expect(decideTier({ ...tierPolicy(undefined), fastWire: { ...SERVICE_WIRE, foreignCallerTiers: "drop" }, }, true, "flex")).toEqual({ kind: "forward-caller" }); }); + + test("unclassified caller tiers honor the final adapter forwarding permission", () => { + expect(decideTier({ + ...tierPolicy(undefined), + adapter: "openai-chat", + forwardCallerTier: false, + }, undefined, "fast")).toEqual({ kind: "drop" }); + expect(decideTier({ + ...tierPolicy(undefined), + adapter: "openai-responses", + forwardCallerTier: true, + }, undefined, "fast")).toEqual({ kind: "forward-caller" }); + }); }); function configWithFastWire(fastWire: unknown, capability?: { provider?: boolean; exact?: boolean }): unknown { @@ -364,6 +400,26 @@ function configWithFastWire(fastWire: unknown, capability?: { provider?: boolean } describe("FastWire config and registry validation", () => { + test("the shared clone detaches nested FastWire records and arrays", () => { + const canonicalToWire = { priority: "priority" }; + const betas = ["beta-one"]; + const original: FastWire = { + kind: "service-tier", + canonicalToWire, + foreignCallerTiers: "verbatim", + betas, + }; + const cloned = cloneFastWire(original)!; + canonicalToWire.priority = "performance"; + betas[0] = "changed"; + expect(cloned).toEqual({ + kind: "service-tier", + canonicalToWire: { priority: "priority" }, + foreignCallerTiers: "verbatim", + betas: ["beta-one"], + }); + }); + test("accepts a complete declaration and trims its wire values", () => { const result = validateConfigCandidate(configWithFastWire({ kind: "service-tier", @@ -403,12 +459,33 @@ describe("FastWire config and registry validation", () => { expect(validateConfigCandidate(configWithFastWire(null, capability)).ok).toBe(false); }); + test("redacts a token-shaped provider name in a FastWire conflict path", () => { + const providerName = ["sk", "proj", "fastwire", "A".repeat(40)].join("-"); + const result = validateConfigCandidate({ + port: 10100, + defaultProvider: providerName, + providers: { + [providerName]: { + adapter: "openai-responses", + baseUrl: "https://fixture.example/v1", + supportsServiceTier: true, + fastWire: null, + }, + }, + }); + expect(result.ok).toBe(false); + if (!result.ok) { + expect(result.error).not.toContain(providerName); + expect(result.error).toContain("providers.[REDACTED].fastWire"); + } + }); + test("provider-level false keeps null valid even with an exact-model true", () => { expect(validateConfigCandidate(configWithFastWire(null, { provider: false, exact: true })).ok) .toBe(true); }); - test("rejects null against an inherited registry capability", () => { + test("accepts and preserves null against an inherited registry capability", () => { expect(validateConfigCandidate({ port: 10100, defaultProvider: "openai-apikey", @@ -420,7 +497,10 @@ describe("FastWire config and registry validation", () => { fastWire: null, }, }, - }).ok).toBe(false); + })).toMatchObject({ + ok: true, + config: { providers: { "openai-apikey": { fastWire: null } } }, + }); }); test("provider-level false closes an inherited registry capability", () => { diff --git a/tests/openai-chat-hardening.test.ts b/tests/openai-chat-hardening.test.ts index ab731ab35b..19b0e6be40 100644 --- a/tests/openai-chat-hardening.test.ts +++ b/tests/openai-chat-hardening.test.ts @@ -561,7 +561,7 @@ describe("openai-chat credential hardening", () => { expect(body.service_tier).toBe("priority"); }); - test("an exact model capability authorizes only that Chat model", () => { + test("an exact model capability authorizes canonical Fast only for that Chat model", () => { const exactOnly = provider({ modelSupportsServiceTier: { "test-model": true } }); const authorized = parsed(); authorized.options.serviceTier = "priority"; @@ -574,6 +574,11 @@ describe("openai-chat credential hardening", () => { expect(JSON.parse(createOpenAIChatAdapter(exactOnly).buildRequest(undeclared).body)) .not.toHaveProperty("service_tier"); + const foreign = parsed(); + foreign.options.serviceTier = "flex"; + expect(JSON.parse(createOpenAIChatAdapter(exactOnly).buildRequest(foreign).body)) + .not.toHaveProperty("service_tier"); + const providerDenied = provider({ supportsServiceTier: false, modelSupportsServiceTier: { "test-model": true }, @@ -582,14 +587,13 @@ describe("openai-chat credential hardening", () => { .not.toHaveProperty("service_tier"); }); - // `service_tier` is an OpenAI-specific extension and this adapter serves 66 registry - // providers, several of which reject unknown body fields. Forwarding it by default would - // turn a caller-supplied tier into an upstream 400 on those routes, so absence of the - // opt-in must mean the field is dropped — the same contract `prompt_cache_key` uses. - test("drops a caller-supplied service tier when the provider has not opted in", () => { + // Foreign `service_tier` values are OpenAI-specific extensions and this adapter serves 66 + // registry providers, several of which reject unknown body fields. Classified canonical Fast + // is handled separately by capability; an unclassified caller still needs this opt-in. + test("drops a foreign caller service tier when the provider has not opted in", () => { for (const p of [provider(), provider({ chatServiceTier: false })]) { const req = parsed(); - req.options.serviceTier = "priority"; + req.options.serviceTier = "flex"; const body = JSON.parse(createOpenAIChatAdapter(p).buildRequest(req).body); diff --git a/tests/service-tier-capability.test.ts b/tests/service-tier-capability.test.ts index ceaa46c2ad..c781208ea8 100644 --- a/tests/service-tier-capability.test.ts +++ b/tests/service-tier-capability.test.ts @@ -3,7 +3,7 @@ * for EVERY Responses provider; now a provider-level `supportsServiceTier` capability * gates it after the final route is settled (tri-state): canonical OpenAI providers * keep the fast-mode behavior (`true`), DeepSeek/Volcengine strip it (`false`), and - * unclassified custom providers preserve caller-supplied values untouched without + * unclassified custom Responses providers preserve caller-supplied values untouched without * ever receiving an injection (PR #860 family). */ import { afterEach, describe, expect, test } from "bun:test"; @@ -86,7 +86,7 @@ describe("service-tier capability is exact-model and provider-scoped", () => { expect(canForwardServiceTierForModel({ ...provider, supportsServiceTier: true, - }, "chat-model", "custom-relay")).toBe(false); + }, "chat-model", "custom-relay")).toBe(true); expect(canForwardServiceTierForModel({ ...provider, supportsServiceTier: true, @@ -176,7 +176,7 @@ describe("routing evidence uses the final model adapter", () => { expect(candidateCapabilityEvidence({ ...config, providers: { relay: relay({ chatServiceTier: false }) }, - }, "relay", "verified").serviceTier).toBe("unsupported"); + }, "relay", "verified").serviceTier).toBe("supported"); const mixedProvider = relay({ adapter: "openai-responses", @@ -185,7 +185,7 @@ describe("routing evidence uses the final model adapter", () => { }); const mixedConfig = { ...config, providers: { relay: mixedProvider } }; expect(candidateCapabilityEvidence(mixedConfig, "relay", "verified").serviceTier).toBe("supported"); - expect(candidateCapabilityEvidence(mixedConfig, "relay", "chat").serviceTier).toBe("unsupported"); + expect(candidateCapabilityEvidence(mixedConfig, "relay", "chat").serviceTier).toBe("supported"); const behavior = resolveProductionBehaviorValues( mixedConfig, @@ -194,7 +194,7 @@ describe("routing evidence uses the final model adapter", () => { mixedProvider, "service-tier-test-salt", ); - expect(behavior?.["responses.serviceTier"]?.value).toBe(false); + expect(behavior?.["responses.serviceTier"]?.value).toBe(true); }); });