clients: assert unknown events stay ignorable, guard action enums, de-time tests - #508
Merged
Merged
Conversation
…-time tests Test-only follow-up to #505 (shipped in 1.56.0). No changeset: nothing publishable changes, only the tests around it. The drift guard added in #505 has to encode BOTH tiers of the contract, or the obvious way to satisfy it is to make unknown types an error — which would break the other half. Per the stream_reasoning schema, a client that does not recognize an event MUST ignore it, so a frame from a server newer than this build has to be dropped quietly and leave the turn healthy. That behaviour was already correct in all four clients and is untouched here; it was simply untested, so nothing stopped a future "fix" from turning the catch-all into an error path. Each client now asserts an unrecognised type is neither surfaced to consumers nor fatal to the turn. Confirmed by making Go's dispatch loop fail the turn on unknown frames: the new test catches it. Go had no spec/actions guard, unlike the other three — and its ActionType constants had the same drift that hid submit_interaction. Added, derived from spec/actions/*.schema.json the same way. The dispatch assertions were timing-based, which is a flake risk on a box running at load 125. Go now emits every frame plus the terminal, drains the turn's channel to close, and asserts on the collected slice — no timers at all, in either the passing or the failing path, since the terminal always decodes and a dropped frame shows up as an absent entry rather than a hang. Python's and .NET's remaining waits are hang-detectors rather than assertions, so they move from 2s/5s to 30s: observed wall time for the Python file already ranged 0.9s to 3.7s under load. Verified with go test -race -count=10 and 5 consecutive Python runs at load 125. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Test-only follow-up to #505 (shipped in 1.56.0). No changeset — nothing publishable changes, only the tests around it.
#505 landed while I was writing these, so they arrive as a separate PR rather than as commits on that branch.
1. The guard has to encode BOTH tiers of the contract
The
stream_reasoningschema says "Clients that do not recognize this event MUST ignore it." That makes two rules, and satisfying one naively breaks the other:spec/events/must decode and be surfaced — what clients: decode interaction + preamble/reasoning frames, add submit_interaction #505's drift guard already asserts, failing the build when a new schema lands unimplemented.Rule 2 was already correct in all four clients and is untouched here. The problem was that it was untested: nothing stopped someone from "satisfying" rule 1 by turning the catch-all into an error path, which would regress every client the moment the server adds an 18th event. Each client now asserts an unrecognised type is neither surfaced to consumers nor fatal to the turn.
Verified negatively — making Go's dispatch loop
failAllon unknown frames instead ofcontinuemakesTestUnknownEventIsIgnoredNotFatalfail:2. Go was missing the
spec/actions/guardPython, .NET and TypeScript each got an action-side drift guard in #505; Go did not — and Go's
ActionTypeconstants had exactly the same drift that hidsubmit_interactionin the first place. Added, derived fromspec/actions/*.schema.jsonthe same way as the event guard.3. The dispatch assertions no longer race a clock
This box runs at load ~125 on 12 cores, so timing-based assertions are a real flake risk.
Go now emits every frame plus the terminal, drains the turn's channel to close, and asserts on the collected slice — no timers at all, in either the passing or the failing path. That works because the terminal
eventual_responsealways decodes (it was never part of this bug), so the drain always completes, and a dropped frame shows up as an absent entry in the slice rather than as a hang. The Go interaction tests went from ~4s of sleeping to 0.67s total.Python's and .NET's remaining waits are hang-detectors rather than assertions, so they move from 2s/5s to 30s. That margin is not theoretical: observed wall time for the Python file alone already ranged 0.9s–3.7s under load, so the old 2s net could have tripped on a slow scheduler rather than on a real bug.
Verified with
go test -race -count=10and 5 consecutive Python runs at load 125, all stable.Results
-race -count=10stableruffcleantsc --noEmitcleanAll four re-run against current
mainafter rebasing off the squash-merged #505.