fix(router): imperativeVerbs was weighted zero — restore the dimension key - #53
Merged
Conversation
…n key The transpile that produced router_core/config.py from config.ts snake_cased config field names. `imperativeVerbs` is both a keyword-list field and one of the 15 dimension names, so its weight landed under `imperative_verbs` while rules.py emitted `imperativeVerbs`. `weights.get(name, 0)` then scored that dimension at zero on every request. Effect: build/deploy-shaped prompts were under-classified. 3 of 8 sampled imperative prompts — "Create and deploy the service", "Set up the config and deploy it", "Develop a CLI that generates reports" — classified SIMPLE where the correct score leaves them ambiguous, which the strategy defaults up to MEDIUM. The 24-shape cross-SDK check missed it because none of those prompts sat within 0.03 of a tier boundary; it still reports 24/24 after the fix. Guarded by two tests: the weight keys and the dimension names the classifier emits must be the same set, and the weight table must match config.ts at 18bf4ab verbatim. Folded into the unreleased 1.12.0 — no published artifact carries the bug.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Found while pulling live values out of the shipped config to update the docs.
The bug
router_core/config.pywas machine-transpiled from upstreamconfig.ts, snake_casing config field names.imperativeVerbsis both a keyword-list field and one of the 15 dimension names, so the transpiler renamed it in both places. The weight ended up underimperative_verbswhilerules.pyemitsimperativeVerbs, and the lookup isweights.get(dimension["name"], 0)— so the dimension scored zero on every request since 1.11.0.The config docstring even says dimension-weight keys stay camelCase; the KEY_MAP applied globally and caught this one anyway.
Impact
Build/deploy-shaped requests were under-classified. Sampling 8 imperative prompts, 3 changed tier:
The router is designed to fail upward under uncertainty; this made it fail downward for a whole class of prompt.
Why the parity check missed it
The 24-request cross-SDK comparison passes 24/24 both before and after — none of those prompts sat within 0.03 of a tier boundary. A sampled behavioural check cannot prove structural equivalence, which is why the new tests assert the structure directly.
Tests
config.tsat18bf4abverbatim.Cross-SDK parity re-verified after the fix: 24/24 identical.
Folded into the unreleased 1.12.0 — no published artifact carries this.