UK spine: run age_tail first so age-conditioned imputations derive from the disaggregated surface; retire the NHS 85+ fold-in (#785)

changelog.d/uk-age-tail-reorder-785.changed.md

@@ -0,0 +1 @@
+Moved UK age-tail disaggregation immediately after the raw FRS spine and keyed its deterministic draw on base `person_id`, so every age-conditioned imputation now derives from the disaggregated age surface while later clones copy donor age; the age-tail gate now runs in the assembled phase and its `achieved_weighted` receipt measures raw FRS-grossed 80+ mass before support channels, the E6/E8 identity receipts pin the disaggregated basis, and NHS allocation uses the native 85–89, 90–94, and 95+ donor bands and `person.age` is declared int64 from the FRS root with age_tail rewriting it as int64, matching what the root produces (#785, #845).

experiments/785-age-tail-reorder-receipts.md

@@ -0,0 +1,128 @@
+# 785 — age-tail reorder receipts
+
+Committed, disclosure-safe checklist for the licensed #785 acceptance lane. The
+raw measurement scripts and receipts will live licensed-side under
+`data/ukds/acceptance/spine-n-785/`, not in this repository. Every published
+measurement below must name its artifact digest, PolicyEngine-UK version, period,
+and disclosure-control threshold.
+
+The before artifact is the **graph-executed 27-stage build of main after #835 merged**
+(its I5 re-mint candidate), not the legacy-executed spine-m: the graph executor's own
+normalizations (text columns stored as StringDtype, root cells cast at CREATE) would
+otherwise appear as payload differences that are not this change's. All before values
+below are **to be measured at L2** from that digest-pinned artifact; none are inferred
+from an earlier spine or from the planning estimates. The after artifact is the
+twin-verified **spine-n** build of this branch.
+
+## Inputs and provenance
+
+- spine-m H5 path and sha256: **to be measured at L2**
+- spine-m producing commit and stage-manifest digest: **to be measured at L2**
+- spine-n primary H5 path and sha256: **to be measured at L2**
+- spine-n twin H5 path and sha256: **to be measured at L2**
+- PolicyEngine-UK version and build period: **to be measured at L2**
+- minimum disclosure-control count: **to be measured at L2**
+
+## Receipt 1 — byte-identical final age
+
+- `person.age` value-surface absence from the payload diff: **to be measured at L2**
+- `person.age` dtype: int64 on spine-n (#845 declares the root cell int64 and age_tail
+  rewrites it as int64) versus the before artifact's dtype — the only licensed
+  surface on this column: **to be measured at L2**
+- sha256 over sorted shared-channel `(person_source_id, age)` pairs, spine-m:
+  **to be measured at L2**
+- sha256 over sorted shared-channel `(person_source_id, age)` pairs, spine-n:
+  **to be measured at L2**
+- equality verdict: **to be measured at L2**
+- capital-gains donor-set and entity-count deltas, reported separately from the
+  shared-channel identity proof: **to be measured at L2**
+
+## Receipt 2 — payload classification
+
+The comparator compares column values only when a table's row counts agree, and
+the reselected CGT band-donor households change the person and benunit counts.
+So first derive the **donor-excluded, row-aligned slices** of spine-m and spine-n
+(drop every household flagged `household_is_cgt_band_donor` together with its
+benefit units and persons, on both sides; the pre-donor block is positionally
+aligned because the donor rows are appended last), then run
+`tools/compare_uk_h5_payload.py` on the two slices and classify its JSON report
+with `tools/classify_uk_payload_diff.py` and
+`tools/uk_age_tail_reorder_payload_expectation.json`. Any row-count inequality on
+a slice is itself an unexpected difference. The committed classifier is the coarse
+payload allowlist; each scoped claim below must also be checked on the named row
+domain because the comparator deliberately reports counts, not unit-row
+identities. The donor block is compared separately under "CGT donor selection".
+
+- slice derivation script and both slice digests: **to be measured at L2**
+
+- comparator report sha256: **to be measured at L2**
+- classifier report sha256: **to be measured at L2**
+- `unexpected` count and verdict: **to be measured at L2**
+
+### SPI stage-1 row-locality
+
+- changed-column incidence on recipients whose stage-time age moved out of the
+  top-coded pile: **to be measured at L2**
+- byte equality outside that recipient domain: **to be measured at L2**
+
+### SPI stage-2 fill incidence
+
+- per-column SPI-channel changed-row incidence, including the reported-benefit
+  chain: **to be measured at L2**
+- any incidence shift beyond the signed-difference band: **to be measured at L2**
+
+### UC reporter and capital follow-through
+
+- stage-19 SPI reporter-set and amount movement, with base-channel byte equality
+  (`uc-reporter-benefit-unit-redraw-incidence` re-derived; the refresh-lift entry was
+  retired by #835's third review pass): **to be measured at L2**
+- stage-20 `frs_benunit_capital`, `uc_reported_capital`, and `would_claim_uc`
+  movement on the SPI reporter domain: **to be measured at L2**
+
+### Salary-sacrifice conditioning
+
+- `employee_pension_contributions` changed-row incidence: **to be measured at L2**
+- `pension_contributions_via_salary_sacrifice` changed-row incidence:
+  **to be measured at L2**
+
+### CGT donor selection and entity counts
+
+- before and after donor-set digests and overlap: **to be measured at L2**
+- household count and benunit/person count deltas: **to be measured at L2**
+- household-count pin verdict: **to be measured at L2**
+
+## NHS reorder totals
+
+Use the native NHS donor table on spine-m and spine-n and report each service
+separately. The before values come from spine-m, not an older planning receipt.
+
+- A&E realized total, before / after / anchor: **to be measured at L2**
+- admitted-patient realized total, before / after / anchor: **to be measured at L2**
+- outpatient realized total, before / after / anchor: **to be measured at L2**
+- combined realized total, before / after / anchor: **to be measured at L2**
+- `uk_aggregate_admin` verdict and margin: **to be measured at L2**
+
+## NHS fold-delta
+
+On the same spine-n frame, call `allocate_nhs_by_age_gender` once with the former
+85+ folded table and once with the native table.
+
+- per-service folded and native realized totals: **to be measured at L2**
+- exact totals-invariant verdict: **to be measured at L2**
+- redistribution across 85–89 / 90–94 / 95+ recipients: **to be measured at L2**
+
+## Identity ladder
+
+- E4: **to be measured at L2**
+- E5: **to be measured at L2**
+- E6 (`stage_time_disaggregated` NHS basis): **to be measured at L2**
+- E7: **to be measured at L2**
+- E8 (`stage_time_disaggregated` donor basis): **to be measured at L2**
+- permutation-stability verdict: **to be measured at L2**
+
+## Twin build
+
+- primary/twin payload comparison report sha256: **to be measured at L2**
+- payload-identical verdict: **to be measured at L2**
+- complete gate-battery verdict, including `uk_aggregate_admin`,
+  `uk_uc_capital_coherence`, and `uk_take_up_signal`: **to be measured at L2**

packages/microcosm-build/src/microcosm/build/uk/gates.json

@@ -376,15 +376,15 @@
     {
       "id": "uk_stage_age_tail_targets",
       "gate": "stage_health",
-      "phase": "transferred",
+      "phase": "assembled",
       "criticality": "release_blocking",
       "evidence_absent_blocks": true,
       "parameters": {
         "stage": "age_tail",
         "check": "age_tail_targets",
         "maximum_relative_deviation": 1.0
       },
-      "notes": "Stage-time achieved-versus-target gate over the age-tail disaggregation receipt."
+      "notes": "Stage-time achieved-versus-target gate over the age-tail disaggregation receipt. At its post-frs_spine position, achieved_weighted measures raw FRS-grossed 80+ mass before clone or zero-weight support channels exist."
     },
     {
       "id": "uk_release_input_coverage",

packages/microcosm-build/src/microcosm/build/uk/release_input_coverage_manifest.json

@@ -473,7 +473,7 @@
     "capital_gains"
    ],
    "source_manifest": "source_stages.json",
-   "source_manifest_sha256": "1128013c99cd3592c06adab25cd0efa7f92667765445e2c6370c6c867a4a3517",
+   "source_manifest_sha256": "2cf4b9513a59b1c50c3a658b98bc2a61e6fd5e54ec5f0fadc3467ae0004680ad",
    "source_vintages": {
     "source": "HMRC Capital Gains Tax statistics, July 2025, Table 2.1a",
     "survey": "HMRC Capital Gains Tax statistics Table 2.1a and Advani-Summers capital-gains incidence"
@@ -496,7 +496,7 @@
     "capital_gains"
    ],
    "source_manifest": "source_stages.json",
-   "source_manifest_sha256": "1128013c99cd3592c06adab25cd0efa7f92667765445e2c6370c6c867a4a3517",
+   "source_manifest_sha256": "2cf4b9513a59b1c50c3a658b98bc2a61e6fd5e54ec5f0fadc3467ae0004680ad",
    "source_vintages": {
     "source": "Advani and Summers (2020), Capital Gains and UK Inequality, CAGE Working Paper 465",
     "survey": "Family Resources Survey 2024-25, SPI synthetic support, and Advani-Summers capital-gains incidence"
@@ -525,7 +525,7 @@
    "required_mass_change_reason": "E5 source-stage transform preserves household rows and typed household weights; total household mass is conserved.",
    "rewrites": [],
    "source_manifest": "source_stages.json",
-   "source_manifest_sha256": "1128013c99cd3592c06adab25cd0efa7f92667765445e2c6370c6c867a4a3517",
+   "source_manifest_sha256": "2cf4b9513a59b1c50c3a658b98bc2a61e6fd5e54ec5f0fadc3467ae0004680ad",
    "source_vintages": {
     "source": "UK Data Service SN 8856 Effects of Taxes and Benefits household tab, DfT rail fare index, and public NHS activity/cost table.",
     "survey": "Effects of Taxes and Benefits 1977-2024 and NHS age-gender public table"
@@ -545,7 +545,7 @@
    "required_mass_change_reason": "E5 source-stage transform preserves household rows and typed household weights; total household mass is conserved.",
    "rewrites": [],
    "source_manifest": "source_stages.json",
-   "source_manifest_sha256": "1128013c99cd3592c06adab25cd0efa7f92667765445e2c6370c6c867a4a3517",
+   "source_manifest_sha256": "2cf4b9513a59b1c50c3a658b98bc2a61e6fd5e54ec5f0fadc3467ae0004680ad",
    "source_vintages": {
     "source": "UK Data Service SN 8856 Effects of Taxes and Benefits household tab and cited VAT anchor resource.",
     "survey": "Effects of Taxes and Benefits 1977-2024"
@@ -576,7 +576,7 @@
    "superseded_by": {
     "reason": "The FRS spine build executes hmrc_cgt_gains_spine, which applies the same HMRC Table 3 amounts redraw directly in source_stages.json before calibration.",
     "source_manifest": "source_stages.json",
-    "source_manifest_sha256": "1128013c99cd3592c06adab25cd0efa7f92667765445e2c6370c6c867a4a3517",
+    "source_manifest_sha256": "2cf4b9513a59b1c50c3a658b98bc2a61e6fd5e54ec5f0fadc3467ae0004680ad",
     "stage": "hmrc_cgt_gains_spine"
    }
   },
@@ -596,7 +596,7 @@
     "capital_gains"
    ],
    "source_manifest": "source_stages.json",
-   "source_manifest_sha256": "1128013c99cd3592c06adab25cd0efa7f92667765445e2c6370c6c867a4a3517",
+   "source_manifest_sha256": "2cf4b9513a59b1c50c3a658b98bc2a61e6fd5e54ec5f0fadc3467ae0004680ad",
    "source_vintages": {
     "hmrc_surface": "2023-24",
     "mapped_build_period": "2024"
@@ -610,7 +610,7 @@
    "base_candidate_tier": "frs",
    "calibration_permitted": false,
    "canonical_source_manifest": "source_stages.json",
-   "canonical_source_manifest_sha256": "1128013c99cd3592c06adab25cd0efa7f92667765445e2c6370c6c867a4a3517",
+   "canonical_source_manifest_sha256": "2cf4b9513a59b1c50c3a658b98bc2a61e6fd5e54ec5f0fadc3467ae0004680ad",
    "effective_mass_requirements": {
     "charitable_investment_gifts": {
      "mass_share_denominator": "all_person_effective_mass",
@@ -693,7 +693,7 @@
    "superseded_by": {
     "reason": "The FRS spine build executes hmrc_spi_income_spine, which supersedes the June retained-leaves/hmrc_spi_income pair inside source_stages.json.",
     "source_manifest": "source_stages.json",
-    "source_manifest_sha256": "1128013c99cd3592c06adab25cd0efa7f92667765445e2c6370c6c867a4a3517",
+    "source_manifest_sha256": "2cf4b9513a59b1c50c3a658b98bc2a61e6fd5e54ec5f0fadc3467ae0004680ad",
     "stage": "hmrc_spi_income_spine"
    }
   },
@@ -727,7 +727,7 @@
    "required_mass_change_reason": "E5 source-stage transform preserves household rows and typed household weights; total household mass is conserved.",
    "rewrites": [],
    "source_manifest": "source_stages.json",
-   "source_manifest_sha256": "1128013c99cd3592c06adab25cd0efa7f92667765445e2c6370c6c867a4a3517",
+   "source_manifest_sha256": "2cf4b9513a59b1c50c3a658b98bc2a61e6fd5e54ec5f0fadc3467ae0004680ad",
    "source_vintages": {
     "source": "UK Data Service SN 9468 Living Costs and Food Survey 2023-24 household/person tabs, NEED 2023 headline energy tables, Ofgem Q2 2026 unit rates, and WAS round-8 bridge donor.",
     "survey": "Living Costs and Food Survey 2023-24"
@@ -748,7 +748,7 @@
     "property_wealth"
    ],
    "source_manifest": "source_stages.json",
-   "source_manifest_sha256": "1128013c99cd3592c06adab25cd0efa7f92667765445e2c6370c6c867a4a3517",
+   "source_manifest_sha256": "2cf4b9513a59b1c50c3a658b98bc2a61e6fd5e54ec5f0fadc3467ae0004680ad",
    "source_vintages": {
     "source": "MHCLG dwellings and ONS UK House Price Index December 2025 regional average prices.",
     "survey": "Public regional property reference"
@@ -772,7 +772,7 @@
     "employee_pension_contributions"
    ],
    "source_manifest": "source_stages.json",
-   "source_manifest_sha256": "1128013c99cd3592c06adab25cd0efa7f92667765445e2c6370c6c867a4a3517",
+   "source_manifest_sha256": "2cf4b9513a59b1c50c3a658b98bc2a61e6fd5e54ec5f0fadc3467ae0004680ad",
    "source_vintages": {
     "source": "HMRC, Salary sacrifice reform for pension contributions effective from 6 April 2029",
     "survey": "Family Resources Survey 2024-25 salary-sacrifice respondents and HMRC salary-sacrifice reform analysis"
@@ -794,7 +794,7 @@
     "student_loan_plan"
    ],
    "source_manifest": "source_stages.json",
-   "source_manifest_sha256": "1128013c99cd3592c06adab25cd0efa7f92667765445e2c6370c6c867a4a3517",
+   "source_manifest_sha256": "2cf4b9513a59b1c50c3a658b98bc2a61e6fd5e54ec5f0fadc3467ae0004680ad",
    "source_vintages": {
     "source": "Explore Education Statistics Table 6a, Higher education total",
     "survey": "Family Resources Survey 2024-25 and Student Loans Company borrower forecasts for England"
@@ -827,7 +827,7 @@
    "required_mass_change_reason": "E5 source-stage transform preserves household rows and typed household weights; total household mass is conserved.",
    "rewrites": [],
    "source_manifest": "source_stages.json",
-   "source_manifest_sha256": "1128013c99cd3592c06adab25cd0efa7f92667765445e2c6370c6c867a4a3517",
+   "source_manifest_sha256": "2cf4b9513a59b1c50c3a658b98bc2a61e6fd5e54ec5f0fadc3467ae0004680ad",
    "source_vintages": {
     "source": "Office for National Statistics Wealth and Assets Survey, UK Data Service SN 7215, DOI 10.5255/UKDA-SN-7215-20; local licensed 2006-22 household tab.",
     "survey": "Wealth and Assets Survey round 8"