census_households: bind from the Chronicle feed and replace the inferred NI constituency mapping

[juaristi22]

Why

external:census_households/households is the only bound family in the UK local target surface whose
values do not come from Chronicle. The external: prefix is the cross-grain machinery's own marker
for no contract target id exists for this side
(_contract_target_id (packages/microcosm-build/src/microcosm/build/cross_grain.py:687)); the other
33 contract targets and 19,419 active references all compile from the pinned consumer feed
(chronicle-uk-6fb700e). These 1,011 cells — 650 constituencies, 361 local authorities — are instead
summed out of the OA ladder artifact and appended after the contract compile at
packages/microcosm-build/src/microcosm/build/uk_runtime/ledger_targets.py:1292.

Chronicle already publishes every one of those values, at exactly these grains, with codes matching
local_area_crosswalk.json one for one:

• Constituency (650), all on uk.local_geography.households.by_constituency.v1:
  ons/census2021_ts041_households_pcon24 (543 E + 32 W), nrs/census2022_households_ukpc24 (57 S),
  nisra/census2021_households_pcon24 (18 N).
• Local authority (361): ons/census2021_ts041_households_lad (296 E + 22 W) and
  nisra/census2021_households_lgd (11 N) on uk.local_geography.households.by_local_authority.v1;
  Scotland's 32 councils via the nrs.census2022.uv404.council.all_occupied_households record set on
  uk.local_geography.tenure.all_households.v1 (chronicle#253 asks for that one to be re-filed under
  the households spec — a convenience, not a dependency; a two-element record_set_spec_id list
  binds it today and is unambiguous, since that tenure spec has exactly one record set and the eight
  detail categories carry their own).

So this is a binding gap, not a facts gap. No Chronicle work is required to close it.

What the comparison shows

Ladder-summed values against the published Chronicle facts, area by area, all 1,011 cells:

• England (543) and Wales (32): mean absolute difference 7.4 and 6.4 households, max 29 and 15,
  nothing above 0.07%. Both legs use ONS's own OA21→PCON24 best-fit lookup, the same basis ONS builds
  its PCON24 census tables on, so we reproduce the publisher's method and differ only by cell-key
  perturbation.
• Scotland (57 constituencies / 32 councils): net −557 and −560 — the same shortfall at
  both grains, because both are our OA22 sum (2,508,715) against NRS's published totals (2,509,272
  and 2,509,275). NRS applies disclosure control independently at each output level. This is a level
  effect, not a mapping error: a mapping error appears at one grain and nets to zero.
• Northern Ireland (18 constituencies): net +4, but mean absolute 197.9, max 694
  (1.49% of Belfast East), with 11 of 18 constituencies off by more than 100 households. NI's own
  11 LGDs — same publisher, same census, same Data Zone counts — are off by a mean of 8.6, max 30.

That contrast is the diagnosis. Data Zones nest exactly into Super Data Zones → District Electoral
Areas → LGDs, and NISRA publishes lookups for that chain, so the LGD leg is clean. PARLCON24 is not
in the nesting chain and NISRA publishes no DZ2021→PARLCON24 lookup — the Flexible Table Builder
refuses to cross two geographies (rule dimension is not first), and the only published constituency
lookup is Small Area 2011 / SOA 2011 → PARLCON24. So microcosm infers one:
infer_ni_dz_constituencies_from_postcodes (packages/microcosm-build/src/microcosm/build/uk_runtime/geography_sources.py:743)
assigns each whole Data Zone to its modal constituency by unweighted postcode count — one
postcode, one vote, regardless of the households behind it.

The errors land in near-integer multiples of the mean Data Zone (203 households): +694 ≈ 3.4 DZ,
−556 ≈ 2.7 DZ, +444 ≈ 2.2 DZ. Total absolute error 3,562 households; since each misplaced Data Zone
creates equal and opposite errors in two constituencies, that is roughly 9 of 3,780 Data Zones
(0.24%), displacing about 1,780 households (0.23% of NI). NI's spine unit is the UK's coarsest —
203 households per Data Zone against 131 per OA in England and Wales and 54 in Scotland — so each
misplacement costs more.

The census_disclosure_control_noise adjudication records "E&W +105, Scotland −554, NI +3" and
accepts them as noise. Those reproduce as the net figures above, so the review measured this exact
comparison — but only the net, never the per-area dispersion, which is where the NI defect hides.

Increment 1 — bind from the feed

1. Declare the contract target. Add the household count as the 34th UK local contract target,
resolving across the three publisher concepts (ons.census2021_households,
nrs.census2022_households, nisra.census2021_households) and two census days (2021-03-21 for
England, Wales and Northern Ireland; 2022-03-20 for Scotland). Mixed vintage is a consumer concern by
the standing boundary and the compiler already records period holds per cell (ons.tenure.* holds
359 census-2021 cells to 2025 today), so the mechanism exists.

2. Replace the appended ladder rows. Drop the constituency_household_targets /
local_authority_household_targets loop at
packages/microcosm-build/src/microcosm/build/uk_runtime/ledger_targets.py:1292
and let the family compile through local_target_reference_membership.json like every other. The
cross-grain bridge national_household_composition_partition_vs_census_households
(packages/microcosm-build/src/microcosm/build/uk_runtime/ledger_targets.py:95)
changes its lower_side from external:census_households/households to the new contract: id;
_validate_uk_cross_grain_declarations will then find it in the contract, which it currently cannot.

3. Settle the uprating denominator. factor = ons.households_total / ladder_total is
29,003,000 / 28,060,832 = 1.0335759 today. On Chronicle facts the denominator becomes the
published sum: 28,061,271 at constituency grain and 28,061,277 at LA grain. The two grains
disagree by 6 households, so one denominator is chosen or the grains carry different factors. Either
gives ≈ 1.0335596, a 1.6e-5 relative change. Note the coupling: the same factor scales the census
tenure cells under the #762 A17 ruling, so it cannot be decided for households alone.

4. Retire pinned_in_ladder. The three ladder sources at
packages/microcosm-build/src/microcosm/build/uk_runtime/local_target_census.py:300
become pinned_in_ledger_facts carrying the Chronicle package ids. They are the only users of that
status, so the constant, its definition and its export go with them.

5. Replace the ladder-identity check. _build_bound_problem refuses when assignment and targets
come from different ladder objects
(tools/build_uk_rowwise_candidate.py:1627, same guard at
:1335). Targets and the sampler stop being the same array by design — that is the point — so the
invariant no longer holds. Replace it with the pairing discipline we actually want: record the ladder
sha beside the target provenance rather than assert object identity.

6. Re-word the fence. census_disclosure_control_noise (approved 2026-08-31 under microcosm#802,
re-worded 2026-09-02, expires 2026-11-30) should record per-area dispersion rather than net only,
and should state that the NI constituency leg's error is a mapping defect, not perturbation. Needs
María's ruling before renewal.

7. Re-measure. 1,011 targets move: up to 694 households in single NI constituencies, a systematic
−557 across Scotland, ≈ ±7 elsewhere. The national household total is untouched.

Increment 2 — replace the NI constituency inference

Independent of increment 1 and shippable after it. NISRA publishes population-weighted centroids
for Data Zones; assigning each DZ to the constituency containing its centroid is the standard
best-fit method and is almost certainly how NISRA built PARLCON24. Constituency boundaries are open
data. A cheaper interim step — weight the postcode vote by household count instead of counting
postcodes — uses only files the ladder tool already downloads.

Either way the work is scoreable: NISRA's published PARLCON24 household counts are the oracle.
The current mapping scores 3,562 total absolute error across 18 constituencies; a correct one should
score like England and Wales, on the order of ±10 per constituency. Iterate until it matches.

What each increment actually fixes

Increment 1 fixes the numbers. Once targets are the published figures, the solve moves ~694 households
of weight out of Belfast East into its neighbours, which is the right answer. Household
characteristics come from the FRS, not from the Data Zone, and the stage-two draw conditions on OA
population only, so row pools are effectively exchangeable within a constituency and calibration
absorbs the mapping error for every constituency-level estimate.

Increment 2 fixes the remaining consistency defect: for those ~9 Data Zones a household carries a
constituency label inconsistent with its own LGD, DEA, ward and Super Data Zone, all of which are
structurally correct. Anything crossing constituency with a different rung is wrong for 0.23% of NI,
and area-support diagnostics attribute those OAs to the wrong constituency.

A property we lose in increment 1, deliberately

Today the same ladder.households array is both the stage-one sampling weight (constituency drawn
within region proportional to it,
packages/microcosm-build/src/microcosm/build/uk_runtime/geography_ladder.py:453)
and the calibration target, so the family fits well whether or not the geography is right and its
residual is not evidence of anything. Binding published facts breaks the circle: the target becomes
an independent check on the assignment, and the NI residual becomes a real signal rather than a
guaranteed zero. Expect the family's fit to get worse in NI on first measure. That is the correct
outcome, not a regression — and it is what makes increment 2 scoreable.

Out of scope

• The OA ladder stays microcosm's geography-assignment artifact and keeps its OA-grain household
  counts as the stage-one sampling weight. Only target values move.
• No Chronicle change is required. chronicle#253 (re-file Scotland's council-area count under the
  households spec) is a selector convenience and blocks nothing.
• No cross-grain rule, precedence or bridge-membership change beyond the one lower_side rewrite.

🤖 Generated with Claude Code

[juaristi22]

Two corrections to the issue text, found while implementing (PR #900):

1. NISRA does publish a DZ2021→PARLCON24 lookup. "Data Zone and Super Data Zone lookups" v3
   (April 2025), sheet DZ2021_Admin_geog_lookup, 3,780 rows, columns DZ2021_code … PARLCON2024_code
   (sha256 8e2e1f6daaddd2f5b6887ccab1bf9d0bf179f6d0d2ba60ef8e0ebb495a3e1999). NISRA's own PARLCON24
   census tables are built by aggregating Data Zones with it (largest share of usual residents; NISRA,
   "Aggregation of Data Zones for 2024 Parliamentary Constituencies"). So increment 2 is a lookup swap,
   not a new best-fit method: the lookup reproduces the 18 published household counts to sum |Δ| 126,
   max 16, mean 7 — the same cell-key noise as England and Wales — against 3,562 / 694 for the
   postcode-mode inference. Neither the centroid method nor a household-weighted postcode vote is
   needed (164 Data Zones straddle a boundary, so no whole-DZ inference could beat the publisher's own
   assignment).
2. Exactly 10 Data Zones were misassigned (2,075 households), not "~9": N20000305, N20000483,
   N20000486, N20000585, N20000955, N20001038, N20001063, N20001937, N20002376, N20002971. Data Zone
   codes are N20… (Super Data Zones are N21…).

Also: the nrs.census2022.uv404.council.households record set from chronicle#253 landed in chronicle
ec7169b, so the PR re-pins the feed there and the new target selects the two households specs
directly rather than a record-set list.