UK spine: run age_tail first so age-conditioned imputations derive from the disaggregated surface; retire the NHS 85+ fold-in (#785)

[juaristi22]

Implements the PR half (P1–P6 code) of the approved #785 plan (repos/uk-785-age-reorder-plan.md, rev 2 of 2026-09-02, María's rulings folded in), and folds in #845. Originally stacked on #835; now based on main after #835 merged. Codex-implemented under the approved-plan protocol, reviewed section-by-section and re-verified independently. Three commits: the reorder with its instruments and derived surfaces; the NHS un-fold; the int64 age declaration (#845).

What this does

• age_tail moves from the spine's tail (index 26) to index 1, immediately after frs_spine, so every age-conditioned stage — ETB services' NHS allocation, both SPI QRF stages, salary sacrifice, the CGT carrier selection, and UK spine: benefit-unit-grain SPI UC reporter draw with pre-take-up screen (#832); exact capital sentinel (#833) #835's UC reporter redraw — sees the disaggregated 80–97 surface instead of the FRS top-code at 80. The draw key changes from person_source_id (which only exists once spi_support_channel stacks clones at stage 17) to the base-row person_id; because base rows copy person_id verbatim into person_source_id and every clone is a whole-table copy of a base row, the final age column is expected byte-identical to today's — so the ONS 85+ age-band targets cannot move and every downstream difference is a pure conditioning effect. A runtime guard refuses a frame that already carries clone provenance, so a future reorder back fails loudly.
• E6/E8 identity instruments invert to the stage_time_disaggregated basis (the min(age, 80) clamps go; the two regression tests that proved the clamps necessary now prove the opposite direction). The committed spine-i acceptance receipt keeps its historical stage_time_top_coded fields until the licensed re-mint.
• Acceptance binder (test_uk_spine_acceptance_receipt.py) becomes a pure function of the accepted roster applying the pending reviewed transformations in order — the two UK spine: condition SPI UC receipt on benefit-unit composition and add a UC-specific child count #832 insertions, then the Derive NHS allocation (and audit age-conditioned imputations) from the disaggregated age surface #785 move — each with an inner assert pinning the receipt state it expects, so each licensed re-mint must delete its own step and strict equality returns when the list is empty.
• Gate uk_stage_age_tail_targets moves transferred → assembled (the stage now precedes frs_brma); its achieved_weighted receipt now measures raw FRS-grossed 80+ mass before any support channel exists (stated in the gate notes and changelog).
• NHS 85–120 un-fold (second commit): etb_services no longer folds the donor table's 85–89 / 90–94 / 95+ bands into one cell — the fold existed only because top-coded recipients could never populate them. The top_band_fold_in manifest declaration is retired with it. Per-service realized totals are invariant to the fold when all cells are occupied; only the within-85+ rate structure changes (the L2 fold-delta receipt proves both on the same frame, no extra build).
• Receipts scaffolding (P1): experiments/785-age-tail-reorder-receipts.md (every number "to be measured at L2"), the committed blast-radius contract tools/uk_age_tail_reorder_payload_expectation.json, and the fail-closed tools/classify_uk_payload_diff.py over compare_uk_h5_payload.py's report. The contract is defined on the donor-excluded, row-aligned slice: the comparator compares column values only when row counts agree, and the reselected CGT band-donor set changes person/benunit counts, so structural surfaces can never be declared expected and any unequal table fails closed.

Why the premise changed (recorded so it gets signed, not discovered)

The issue's framing was "a few variables, a minor change". Exploration corrected it: ~3.59m weighted persons sit at exactly 80 and ~90% of them change value; the empty 85+ NHS cell today allocates ~£21.4bn of donor cost to nobody (realized total ≈ £180.6bn vs the £202bn anchor — inside the ±15% uk_aggregate_admin band, and the move lands it essentially on-anchor); SPI stage 2 uses age as a training feature on FRS base rows, so the whole SPI-channel fill surface churns (and stage 19/20 follow through their inputs); the CGT 270-donor selection reshuffles (the 255/270 precedent). WAS, LCFS, VAT, take-up, draws, BRMA, council tax, education, legacy proxies, student loans and every base-channel row of stages 19–20 are expected byte-equal — that expectation is the fail-closed half of the committed contract (R1).

Derived surfaces

sources.yaml authority (block moved, draw_key: person_id, notes rewritten as a quoted scalar — fixing the pre-existing #623 YAML-comment truncation of the parsed note) + lockstep source_stages.json; UK spec sha re-pin; coverage manifest regenerated; the three gate-battery digests plus the two spine certification-part digests in microcosm-data/contract.py and their test_contract.py mirrors; driver _STAGE_NAMES; roster/order tests (age_tail is frs_spine + 1, no later stage may output or rewrite age). Not touched, by design: spine_swap_signed_differences.json (re-cuts need licensed numbers), spine_candidate_acceptance.json, efrs_parity_known_gaps.json, WAS and uc_reporter_redraw models, US surfaces.

On the graph-executed spine (rebased over #835's review passes, then onto main)

#835 rebased onto main and gained the microcosm-graph package: the UK driver, binder, and roster tests now derive stage order from compile_graph(uk_spine_graph(spec)), a cell-ownership graph, and the spine executes through it. On that base the reorder is a declaration change rather than a list edit:

• uk_runtime/graph.py: age_tail consumes only age and gender (no clone provenance); because the root version owns age, its boundary opens by the same-version owner rule and every later age reader — every finite-consumes stage requests age as a context carrier — binds to the disaggregated cell. The compiled order equals the manifest order with age_tail second, verified by the existing declaration-order-invariance test.
• That early boundary means root cells are inherited rather than owned downstream, so frs_education_grant_split's rewrite of the root cell education_grants no longer opens a boundary and the open-surface frs_legacy_proxies reader formed a cycle; it joins the reader-isolation set exactly as uc_capital_coherence already did.
• Two consequences of age arriving as float64 early are closed at their source: frs_education casts age_started_or_accepted_current_education_or_training to int64 explicitly (the stage owns its output dtype; the export contract is unchanged), and tools/graph_uk_spine_fixture.py snapshots the captured root instead of aliasing the person table age_tail rewrites in place (at the tail, earlier stages had long since replaced that table). The charter-H2 parity fixture is re-pinned on the reordered graph; its captured root stays the top-coded surface.
• Digests (gate battery, spine certification part, UK bundle sha, coverage manifest) are cut on main — plan A8 is moot.

#845 folded in (third commit)

The CREATE node declared person.age float64 while the FRS root produces int64, so graph stages saw a float age from node 0 and the legacy plan an integer until age_tail rewrote it. The interface forbids a rewrite from changing its base's dtype (decl.py: "the base's declared dtype must equal dtype"), so #845's literal proposal — int64 root, float64 rewrite — is not expressible; the honest declaration is int64 end to end: the root cell and age_tail's rewrite are declared int64, and the transform preserves an integer input dtype (its bands and within-band offsets are integral; a float input, as in synthetic test frames, stays float). frs_education's explicit int64 cast stays as the stage's own contract. The only payload consequence is the exported age dtype (float64 → int64), declared as a dtype-only expected change through a new per-group surfaces allowlist in the classifier — a value difference on age remains unexpected, so Receipt 1's byte-identity claim keeps its fail-closed instrument. H2 fixture re-pinned (schema.json + uk_spine.json); policyengine-uk's loader casts to its float32 age as before.

Deviations from the plan

Codex's, all accepted on review: #830 absent from the supplied base (see A8); P4's funnel needed P5's binder to pass (order preserved); the two spine certification-part digests moved alongside the three named pins; coverage-manifest occurrence count is the generator's; sandbox needed a local uv cache and uv run python for the CI-group verifier. Reviewer's post-Codex tightenings: the donor-excluded-slice contract in the classifier/expectation (above); the top_band_fold_in declaration retired with the fold; the E8 proof test asserts the lifted person wins outright rather than comparing a function to itself.

Verification

Scoped UK + shared-machinery funnel (test_uk_*, country_spec, spec_engine, source_runtime, stochastic_assignment, gate_battery, gates, release_input_coverage, data test_contract) plus the whole microcosm-graph suite (charter-H2 UK spine parity included), ruff check ., and tools/ci_test_groups.py --verify — run independently after review, and again on the rebased tree. Expected pre-existing failures only: the two licensed-cache regeneration tests (test_uk_parity_reference reference arm, test_uk_release_input_coverage_manifest candidate arm — committed 2.89.0 evidence vs the 2.92.1 floor, PR #749's lane).

Out of scope / next (licensed lane, after #835's I5/I6 and the 2026-09-30 deferral disposition)

L1 before-side on the graph-executed post-#835 build of main (the legacy-executed spine-m would show the executor's own normalizations — StringDtype text columns, CREATE-time casts — as differences that are not this change's); L2 spine-n build on uk-publication-stack-785 with Receipt 1 (byte-identical age values; dtype now int64), Receipt 2 (payload classification on the donor-excluded slices), the fold-delta receipt, the identity ladder e4–e8 on the new basis and the gate battery; L3 parity re-cut (donor-selection-rng-entity-counts, spi-channel-qrf-incidence, salary-sacrifice-conversion-depth, uc-reporter-benefit-unit-redraw-incidence re-derived on this candidate — its refresh-lift sibling was retired by #835's third review pass — and a new NHS weighted-totals entry) and the acceptance re-mint; L4 delta-eval vs v18. This PR is held until those numbers exist.

Part of #785 (stays open for L1–L4). Closes #845. Parent lanes: #145 (WS-E), #796, epic #665.

🤖 Generated with Claude Code