fix(canary): watch untagged slab text; emit byte-exact req_bytes; harden billing-block tests - #274
Open
MaVo2010 wants to merge 4 commits into
Open
fix(canary): watch untagged slab text; emit byte-exact req_bytes; harden billing-block tests#274MaVo2010 wants to merge 4 commits into
MaVo2010 wants to merge 4 commits into
Conversation
unknownStaticTags catches a tag nobody has classified yet. churningStaticTags
catches the more expensive failure: a tag classified as static whose content
actually moves per turn, which re-renders the static slab PNG on every turn
and silently voids the image cache we built the whole prefix around.
Seven cases. Each uses its own first user message, because the observation
map is keyed by firstUserSha8 and lives at module scope: sharing a message
would inherit a foreign baseline and report phantom churn.
- first sighting stays silent (no baseline yet to compare against)
- content change within one session is reported
- a tag that settles again falls silent (an edge, not a latched verdict)
- a tag changing every turn is reported every turn; a tracker that drops
its baseline after a hit would flag every second turn and halve the
detection rate without anything looking broken
- observations are scoped per session, not global
- the table is bounded and forgets its oldest entries
- reordered same-named blocks count as churn, because the slab bytes
really do move even though the set of tags is unchanged
Suite 932 passed (52 files), typecheck clean; 925 before, so the new cases
are additive and no existing test changed behaviour.
observeStaticTagChurn only ever hashed content inside tag-shaped blocks, so plain prose in the static slab could change every turn in complete silence — re-rendering the slab PNG and voiding the image cache with nothing to show for it. Measured 2026-08-02 on live traffic: a two-character move in the untagged remainder invalidated 101,848 cached tokens while every tag hash stayed put. splitStaticDynamic now also returns the text outside every tag block, keyed under the reserved '#untagged'. The sniffer matches /[a-zA-Z][a-zA-Z0-9_-]*/ and so can never produce a key starting with '#', which makes shadowing by a real tag impossible. It is registered unconditionally, so the canary now runs for slabs carrying no tags at all — the exact shape that hid this, because staticTagContents was empty and the call site gated on size > 0. Three tests: prose that moves is reported, a tag-free slab is still watched, stable prose stays silent. Mutations: dropping the registration kills the first two; hashing the whole slab instead of the residue kills four of the pre-existing tag tests, since tag content would then leak into #untagged. One pre-existing test was tightened rather than weakened. The eviction test asserted that nothing at all was reported; with the residue watched it now legitimately reports #untagged, because removing 4100 '\n'-joined blocks leaves 4099 newlines the single-tag turns lack. It now asserts what it actually means: that the evicted tag is not named. Suite 935 passed (52 files), typecheck clean; 932 before.
info.serializedRequestBytes (set by proxy.ts:1282 from the final serialized upstream body) was measured but never written into TrackEvent, so the diagnostics-page headroom KPI had no size field to read. 28 real 413s in events.jsonl carried no size at all. Copy it through under req_bytes, matching the existing image_bytes convention. TransformInfo already declared serializedRequestBytes cleanly (transform.ts:580); no type changes needed there.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
13a32e3)The churn-observation canary only watched tagged static blocks, so churn in the untagged slab text could go undetected. Found while chasing a real mis-classification in production.
req_bytes(48f5656)Each event record now carries the serialized request size, measured on the exact bytes sent upstream instead of being recomputed later. Running in production on our proxy since 2026-08-11; it served as the primary size denominator for a month-long image-vs-text A/B evaluation.
1672cb3,da884e2)Cover
churningStaticTags(the mis-classification canary), adapt billing-block semantics to the upstream body-drop change, and add nonce-cache cases.Validation
main(8ba82b7),tsc --noEmitclean.