fix(lang-bias): preserve YAML bias: order so priority is deterministic

[inureyes]

Summary

A --lang-bias-config YAML file naming two or more CJK languages steered differently on every run. LangBiasYamlConfig::bias deserialized into a HashMap, and the resolve loop pushed that map's iteration order straight into LangBiasSet.ordered, which is the priority order TokenLanguageIndex::to_token_bias resolves conflicts with under first-language-wins. RandomState randomizes iteration order per map instance, so the priority order was a fresh random permutation on every load. Han is shared by ja, zh and ko, so every token they share got a different bias on every run, with no error and no warning. The schema example in the LangBiasYamlConfig doc comment is itself a three-CJK config, so copying the shipped example was enough to trigger it.

The same HashMap collapsed repeated keys during deserialization (serde_yaml resolves them last-wins with no diagnostic), which made the seen / DuplicateLanguageCode check in the resolve loop unreachable and let the YAML path silently accept input the --lang-bias parser has always rejected.

The accepted YAML schema does not change

bias: is still a plain mapping. Existing config files keep parsing unchanged, a sequence-shaped bias: block is still a parse error (pinned by yaml_bias_block_rejects_a_sequence), an absent or empty block still resolves to an empty LangBiasSet rather than an error, #[serde(deny_unknown_fields)] stays, and CLI --lang-bias still replaces the YAML set entirely.

Two behavior changes are visible to users, both recorded in CHANGELOG.md. A file that happened to rely on the random ordering now steers consistently toward its first-listed language, which is the documented intent. And a file repeating a language code now errors instead of silently taking the last occurrence.

Approach

Candidate 1 from the issue's implementation notes: a hand-written Deserialize collecting the block through MapAccess. bias becomes Option<BiasEntries>, where BiasEntries is a newtype over Vec<(String, BiasValueStr)>. Document order survives, and a repeated key arrives as a genuine second entry, so the existing seen check starts firing and the YAML path reaches CliError::DuplicateLanguageCode (the same variant, not a parallel validation path). No new dependency.

serde_yaml::Mapping was the alternative. It preserves order and rejects duplicates, but with serde_yaml's own error rather than the repo's, so the two entry points would still disagree on what a duplicate looks like. indexmap as the field type was rejected outright: it preserves order but resolves duplicates last-wins, so the dead check would have stayed dead.

The seen / ordered loop, BiasValueStr, parse_bias_f32, and the UnknownLanguageCode mapping with its "{code_str}: (from YAML)" context are all untouched. LangBiasSet and to_token_bias are untouched: they were already correct, only the YAML producer was wrong. Nothing outside src/lang_bias.rs names LangBiasYamlConfig, so the CLI generate path and the server model_worker path consume the resolved LangBiasSet exactly as before, and the languages field of the DEBUG lang_bias resolved event now prints the configured order.

Proof the regression tests are load-bearing

The four new tests were run against the pre-fix production path (the bias field reverted to Option<HashMap<String, BiasValueStr>>, tests unchanged). All four fail:

test lang_bias::tests::yaml_and_cli_paths_agree_on_multi_cjk_order ... FAILED
test lang_bias::tests::yaml_multi_cjk_bias_keeps_document_order_across_repeated_resolves ... FAILED
test lang_bias::tests::yaml_multi_cjk_first_language_wins_on_shared_han_tokens ... FAILED
test lang_bias::tests::yaml_duplicate_language_code_is_rejected ... FAILED

---- yaml_multi_cjk_bias_keeps_document_order_across_repeated_resolves stdout ----
assertion `left == right` failed: iteration 0: entry 0 should be Ja but the resolved order was [(Zh, -10.0), (Ko, 5.0), (Ja, -inf)]; YAML bias: order must be the priority order
  left: Zh
 right: Ja

---- yaml_multi_cjk_first_language_wins_on_shared_han_tokens stdout ----
assertion `left == right` failed: iteration 2: the shared Han token must take the first-listed language's bias (ja = -inf), resolved order was [(Zh, -10.0), (Ja, -inf), (Ko, 5.0)]
  left: Some(-10.0)
 right: Some(-inf)

---- yaml_and_cli_paths_agree_on_multi_cjk_order stdout ----
assertion `left == right` failed: iteration 0: the YAML path and the --lang-bias path must resolve equivalent input to the same LangBiasSet
  left: [(Ja, -inf), (Ko, 5.0), (Zh, -10.0)]
 right: [(Ja, -inf), (Zh, -10.0), (Ko, 5.0)]

---- yaml_duplicate_language_code_is_rejected stdout ----
a repeated language code in a YAML bias: block must be rejected: Some(LangBiasConfig { bias_set: LangBiasSet { ordered: [(Zh, -2.0), (Ja, -3.0)] }, ... })

test result: FAILED. 26 passed; 4 failed

Three distinct permutations of the same three-key file appeared inside that single process run, which is the per-map-instance randomization the issue measured. Each ordering test performs 32 full resolve() calls rather than one, because a single resolve over three languages comes out right often enough to prove nothing; the observed failures landed at iteration 0 and iteration 2.

What changed

• src/lang_bias.rs: new BiasEntries newtype over Vec<(String, BiasValueStr)> with a MapAccess-based Deserialize, IntoIterator, and as_slice / len / is_empty. The doc comment carries why a HashMap was wrong on both counts.
• src/lang_bias.rs: LangBiasYamlConfig::bias is now Option<BiasEntries>. The schema doc comment gains a paragraph stating that bias: order is the priority order and that a repeated code is rejected.
• src/lang_bias.rs: CliError::DuplicateLanguageCode's message no longer says the duplicate came from --lang-bias, since both entry points now raise it. The variant and its code field are unchanged.
• src/lang_bias.rs: the resolve loop is unchanged apart from a comment; it now receives entries in document order with repeats intact.
• src/lang_bias.rs tests: yaml_well_formed_parses asserts order instead of map membership, which is the blind spot that hid this. Six new tests: three looping order/parity/to_token_bias tests, a duplicate-rejection test, an empty-block test, and a sequence-rejection test pinning the schema.
• CHANGELOG.md: an ## [Unreleased] entry covering both user-visible changes.

Test plan

• cargo fmt --all -- --check (exit 0)
• cargo clippy --lib --tests --features cuda -- -D warnings (exit 0)
• cargo check --lib --tests --features cuda (exit 0)
• cargo check --bins --features cuda (exit 0)
• cargo test --profile test-fast --features cuda --lib lang_bias (exit 0, 30 passed / 0 failed)
• The same test run against the pre-fix bias field type: 26 passed / 4 failed, output above.

Not run here: the full make verify-test-cuda workspace gate.

Closes #1267

[inureyes]

make verify-test-cuda on GB10 (CUDA sm_121, Linux aarch64) at 6f934905: 8194 passed, 0 failed, 310 ignored, 101 suites, run to completion through the doc-test units.

That is +6 against main at fdc19666 (8188 passed), and the diff adds exactly 6 #[test] functions and removes none, so the totals reconcile.

cargo fmt --all -- --check, cargo clippy --lib --tests --features cuda -- -D warnings, cargo check --lib --tests --features cuda and cargo check --bins --features cuda are clean on the same tree.