Skip to content

UK region-grain activation: fan out the 18 two-level targets over the region tier and make region a cross-grain control (#905) - #906

Draft
juaristi22 wants to merge 10 commits into
uk-890-target-bindingfrom
uk-region-grain-905
Draft

UK region-grain activation: fan out the 18 two-level targets over the region tier and make region a cross-grain control (#905)#906
juaristi22 wants to merge 10 commits into
uk-890-target-bindingfrom
uk-region-grain-905

Conversation

@juaristi22

@juaristi22 juaristi22 commented Sep 11, 2026

Copy link
Copy Markdown
Collaborator

Activates region grain on the UK national calibration surface (closes #905; plan repos/uk-region-grain-activation-plan.md, María's rulings of 2026-09-11).

What changes

The 18 two-level (country + region) contract targets — the nine ons.population.age_*_by_region rows and the nine voa.council_tax_stock.band_a..h/total rows — no longer compile to one country row each. They fan out over the region tier: the nine English regions at Chronicle's region level plus Wales, Scotland and Northern Ireland at country level for the ONS family (108 cells), and the nine English regions for the VOA family (81 cells). The 18 country rows are retired; the twelve ONS cells of a band sum to the retired UK row to the unit (0–9: 7,553,013 at mid-2024).

Three pieces, shared code first (ruling R3):

  1. Fan-out (shared authoring). TargetReferenceAuthoringConfig.geography_fanout_by_target_id fans a target over a declared (level, geography_id) roster inside author_target_references: one reference per cell named and measured target_id@geography_id, selector = contract selector + the cell's geography, an unsigned absent cell refuses the run. The UK generator supplies the roster (UK_REGION_TIER in geography_ladder.py) and the per-cell metadata; the national rows fan over the nations only where the binding does not already pin a country (the VOA binding filters country == ENGLAND).
  2. Geography predicate (shared materializer). A reference may carry metadata.geography_predicate, a JSON predicate in the ordinary vocabulary appended to the contract binding's filters; references without it are untouched. The UK cells carry {"entity":"household","variable":"region","operator":"==","value":"LONDON","reduce":"any","map_to":<entity>}, resolved through the adapter's existing household_condition projection — no adapter change.
  3. Region as a cross-grain tier. UK_CROSS_GRAIN_RULE precedence becomes country > region > constituency > la; every tier code is its own leg (UK_CROSS_GRAIN_PARENT_GEOGRAPHY_LEGS), English constituencies and authorities resolve their leg through the crosswalk's new ladder-derived region_code_by_area, nations by GSS prefix. The three Chronicle-country nation rows are placed at the region grain by a cross_grain_grain stamp the surface builder reads (ruling R4). The shared operator now treats any winning row whose geography declares legs as a declared parent, so region controls get the same dedupe, bridge-completeness and receipt naming as country controls.

Registers and receipts

  • target_references.json / target_reference_membership.json: 595 active (424 − 18 + 189); the 406 untouched references regenerate byte-identically; membership records the fan-out roster per target under geography_pins.
  • local_area_crosswalk.json: region_code_by_area per level from the pinned ladder (bed3f13d…); every constituency and authority maps to exactly one tier code (London: 75 constituencies, 33 authorities).
  • Compile-parity receipts: the 108 incumbent region×age rows and 81 regional VOA rows now compare as matched through align_uk_national_registry_parity_fixture (also applied by the battery gate loader). 2025 fixture: fixture_only 269 → 80, ledger_only 56 → 38, calibration_drift 172 → 280 — the 108 ONS cells drift against the incumbent's rounded subnational projections (ruled rationale on each row); all 81 VOA cells match exactly.
  • uk_data_target_parity.json: cross_grain_council_tax_stock_england_regionported_national.
  • The two region_pin_unsupported signed exclusions were retired by Bind the Chronicle transport and energy facts on the UK national surface (#890 PR-T) #904.

Measurement (ruling R5)

Two tools/calibrate_uk_national_dataset.py runs on the spine-q national pool (cf1f9dda…, 52,846 households; feed c6f9361; epochs 1500, family_equal, lr 0.02): this branch's register (595 references, 545 after the packaged exclusions) against PR #904's (424, 374 after exclusions). Evidence: docs/evidence/uk-region-tier-905/calibration-comparison.json.

baseline (#904) this branch
targets after exclusions 374 545
final loss 0.0112 0.0128
within 10 % 95.7 % 96.5 %
effective sample size 10,350 10,469

Region cells: all 108 ONS cells within 5 % (median |error| 0.08 %, worst 0.5 %); 78 of 81 VOA cells within 5 % (median 0.09 %); the other three, voa.council_tax_stock.band_h in the North East, North West and Yorkshire and the Humber, sit at −100 % because the spine carries no band-H household in those regions (spine-m had 170 band-H households from 49 raw FRS households, none there — microcosm#762 A14, the class already signed for the local band-H family under council_tax_band_h_spine_support_absent). The terminal gate blocks on those three cells.

Common rows (356): fit unchanged within run noise — 332 → 329 within 5 %, 16 → 15 within 25 %. Two rows the gate also reports belong to the #890 / PR-S lane, not to this change: hmrc/self_employment_income_income_band_20_000_to_30_000 at the 25 % bound (+23.6 % → +25.0 %) and obr.fuel_duties_cars moving −14.9 % → −23.9 % inside its reviewed target-fit exclusion, which the gate now flags as stale.

Ruling needed (María): the three band-H region cells. Proposed: three reviewed measure exclusions in uk/calibration_measure_exclusions.jsonvoa.council_tax_stock.band_h@E12000001, @E12000002, @E12000003 — reason "no band-H household on the spine in this region (spine support absent, the local band-H class); binds when a spine vintage carries broader band-H support", tracking microcosm#796, adjudication microcosm#762 A14, windowed to 2026-11-26 with the other spine-support exclusions. Alternative: exclude all nine band-H region cells as one family. Not added without the go.

Tests

Targeted UK and shared suites green locally (471 register/parity tests, the ledger, council-tax, cross-grain, authoring, materialization, ladder, crosswalk and national-calibration files; tools/ci_test_groups.py --verify ok). Stacked on #904, so no PR CI until it re-points to main.

Notes for review

🤖 Generated with Claude Code

juaristi22 and others added 10 commits September 11, 2026 12:42
The national feed moves from ec7169b to c6f9361, the merge of chronicle #258
on top of #255: the transport and energy packages #890 binds (HMRC hydrocarbon
oils, OBR fuel duty by vehicle category, DESNZ pump prices, DfT licensed cars
by fuel type, Scotland/Wales/NI bus finance, ORR rail finance and fares index,
NEED consumption, Ofgem cap levels by period and region, ONS Consumer Trends
COICOP expenditure, NTS0313/NTS0621 bus-use frequency). 138,847 rows; the
consumer-fact schema digest moved with the new quarter and week period types.

No existing national reference resolves a different fact or value: 415
active, 7 deferred, 8 signed out, and the compile-parity receipts regenerate
unchanged; only the membership's feed label moves. The local census,
validation levels and local references keep their separately reviewed
ec7169b pin, so the local regeneration test now reads its own default file
(.codex-work/consumer_facts_uk_local.jsonl) instead of falling back to the
national one; the runbook records both locations.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Spine stages and target diagnostics need publisher values without reading the
feed at build time. tools/vendor_uk_ledger_facts.py copies the facts named by
uk/ledger_fact_vendor_selections.json into nine per-concern resources (DfT bus
receipts, support, fares index and NTS quintile trips; HMRC clearances, DESNZ
prices, OBR receipts by vehicle category and ONS population; DfT licensed cars
by fuel type; NEED consumption; Ofgem cap levels and benchmark consumption for
FY2024-25; NTS bus-use frequency; devolved bus finance; ORR rail finance; ONS
COICOP expenditure). Every row carries the fact keys, source digest and value
verbatim plus the national feed identity it was taken from; selectors use the
reference-selector vocabulary and are matched by the same function, so an
unknown field or a changed match count refuses. A test regenerates each
resource byte-identical from the pinned feed. This retires the hand-copied
chronicle_candidate anchors one concern at a time.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…de London (#890 B)

The national reference generator now pins a contract target whose only
declared level is region at region level with its E12 code, so the two
London bus rows match the region-stamped BUS05i facts (fares GBP 1,347.4m,
net support GBP 1,130.2m, year ending March 2025) instead of a country row
that never exists. The compiler refuses a region pin outside the nine English
region codes the geography ladder already carries and refuses a region-pinned
reference that resolves anything but a region-stamped fact, so no second
roster is needed.

England outside London is now a derived partition member: the England and
London rows are bound, Chronicle stamps the outside-London rows at
statistical_scope, and test_uk_bus_targets keeps asserting the publisher
partition reconciles within GBP 1,000. The UK rows stay signed out because
Wales publishes no fare receipts. Active references 415 -> 417; the
compile-parity receipts gain the two London rows as ledger-only entries.
Contract notes on the England rows now describe the compile-time BUS0415
alignment and the in-stage NTS0705a shaping rather than "pending #790".

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…index (#890 C)

BUS05ai receipts are published for the year ending March; the calibration
year is a calendar year. The national compile now transports each
dft_local_bus fare-receipts row by the ratio of the mean BUS0415 local bus
fares index at the four quarter-ends inside the calibration calendar year to
the mean at the four quarter-ends inside the receipts' April-to-March fiscal
year, per publisher series (England E92000001, London E12000007). On the
pinned feed that is x1.05696 for England (GBP 3,417.4m -> 3,612.0m) and x1.0
for London, whose fares were frozen. Net support is never aligned (#789
scope item 5). A missing quarter-end refuses the reference rather than
guessing.

The factor is declared on the spec metadata with the index concept, the
series geography, the months used, their source record ids, the two means,
the pre-alignment value and the adjudication (calendar-year basis, ruling
2026-09-10), so run manifests and receipts show the publisher facts the
transport stands on. The compile-parity receipts move by the same factors.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…evolved bus legs (#890 E1, G, F1)

Eight net-new national targets, all pinned Chronicle facts, in registry
scope with their no-incumbent-ancestor declarations:

- obr.fuel_duties_cars binds the OBR fuel duty receipts from cars (FY2025
  projection GBP 15.9bn) on the engine's cars-only fuel_duty. The
  all-road-users obr.fuel_duties row is signed out of the reference surface
  and its #757 calibration measure exclusion retires with it.
- orr.rail_government_support binds ORR table 7270 total government support
  (FY2024 GBP 21.6bn, Great Britain, so Northern Ireland households are
  filtered out of the measure) on rail_subsidy_spending, replacing the
  incumbent's post-calibration rail scaler.
- ons.household_energy_expenditure binds ONS Consumer Trends COICOP 04.5
  (CY2025 GBP 43.4bn) on domestic_energy_consumption, with the other-fuels
  scope difference noted.
- The devolved local-bus legs: Scotland passenger revenue (GBP 391m) and all
  government support (GBP 499m) from the 2025 Scottish Transport Statistics;
  Wales public support through local authorities (concessionary fares plus
  support to operators, GBP 131.5m gross) from the StatsWales outturn;
  Northern Ireland Translink bus receipts (GBP 150.1m, Ulsterbus plus
  Metro/Glider) from DfI; and Northern Ireland public transport support as a
  joint bus-plus-rail cell (PSO plus concessionary compensation, GBP 111.7m)
  from the NITHC accounts, measured on bus_subsidy_spending +
  rail_subsidy_spending. The UK net-support row is signed out as covered by
  the four country legs; the UK fares row stays out because Wales publishes
  no fare revenue.

Active references 417 -> 424; contract targets 236 -> 244 (registry scope
202 -> 210, unmapped declarations 19 -> 27). Compile-parity receipts gain
the new rows as ledger-only entries.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…eceipts (#890)

On the newest gated national spine (spine-p) with the PR-T surface, every new
row fits inside the 25% bound after calibration except obr.fuel_duties_cars at
-25.6% (from -46.6% before the solve). That row gets a dated reviewed
target-fit exclusion (approved 2026-09-11, expires 2026-10-11, naming the
baseline run) because both of its fixes are the spine half: the LCFS car-count
predictor and the donor litre-proxy uprating. The row stays bound; only the
release fence defers. A second run with the entry in force passes the battery.

The receipts record what the fit costs today: the London bus rows, the
Scotland and Northern Ireland legs and the ORR rail row are closed by weight
stretch on the households that carry the imputed columns (73-85% of London
bus mass and 71% of rail support mass on households stretched more than 5x),
which is the distortion the value-side stage work in PR-S removes; ONS
household energy expenditure already sits on the frame at design weights.
The ORR 7271 all-sources figure is recorded for the open 7270-versus-7271
rail ruling.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The 18 country+region contract targets (ONS population by age band, VOA
council-tax stock by band) compiled to one country row each because the
national generator pins every target to a single geography. They now fan
out over the twelve-area region tier declared in geography_ladder
(UK_REGION_TIER: the nine English regions at Chronicle's region level,
Wales, Scotland and Northern Ireland at country level), one reference per
cell named and measured target_id@geography_id, through a new shared
authoring option (geography_fanout_by_target_id + geography_fanout_metadata
on TargetReferenceAuthoringConfig). The VOA family fans over the English
regions only, because its binding already pins country == ENGLAND. An
unsigned absent cell refuses the run, as the area-grain authoring does.

The 18 single country rows are retired: the twelve ONS cells of a band sum
to the former UK row within the same publication (0-9: 7,553,013 at
mid-2024). Register: 595 active (424 - 18 + 189); the 406 untouched
references regenerate byte-identically; membership records the fan-out
roster per target under geography_pins. Contract binding notes describe our
mechanism instead of the incumbent's compute_regional_age.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A fanned-out reference shares its contract binding with every sibling
cell; what distinguishes the cells is the predicate the authoring stamps
into metadata.geography_predicate (a JSON object in the ordinary predicate
vocabulary). The shared materializer appends it to the binding's filters
before masking; references without the key are untouched, so no existing
country-level reference is re-scoped by its geography stamp, and a provider
binding refuses it because providers do not read filters.

The UK cells carry a household-region predicate (entity household,
variable region, reduce any, map_to the binding entity), resolved through
the adapter's existing household_condition projection: persons for the ONS
bands, households for the VOA bands. No adapter change. Tests cover the
shared mechanism on the stub adapter and the UK frame adapter at both
grains (81 VOA region cells, 12 ONS cells).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
UK_CROSS_GRAIN_RULE's precedence becomes country > region > constituency
> la. Every region-tier code is its own leg (UK_CROSS_GRAIN_PARENT_
GEOGRAPHY_LEGS; the K02/K03/E92 parents list the tier members they
contain), so a region cell is the control for the constituencies and
authorities inside it. English areas resolve their leg through the
crosswalk's new region_code_by_area, derived per level from the sha-pinned
OA ladder (every constituency and authority carries exactly one tier code;
London holds 75 constituencies and 33 authorities); Welsh, Scottish and
Northern Irish areas resolve by GSS prefix, and an English code outside
the crosswalk refuses instead of falling back to an England leg no control
covers. Leg licences follow the same function.

Chronicle stamps the Welsh, Scottish and Northern Irish population at
country; on the surface those rows are the same ITL1 tier as the nine
English regions and must not outrank them, so uk_local_target_surface
reads the cross_grain_grain stamp the generator writes and places them at
the region grain. The selector and ledger metadata keep Chronicle's level.
The shared operator now treats any winning row whose geography declares
legs as a declared parent, whatever grain it sits at, so region controls
get the same dedupe, bridge-completeness check and receipt naming as
country controls.

Compile parity: align_uk_national_registry_parity_fixture renames the
incumbent's ons/<region>_age_* and voa/council_tax/<REGION>/<band> rows to
the tier names (applied by the receipt tool and the battery gate loader),
so the 108 region x age rows and 81 regional VOA rows compare as matched.
2025 receipts: fixture_only 269 -> 80, ledger_only 56 -> 38 (the retired
country rows), calibration_drift 172 -> 280 (the ONS cells drift against
the incumbent's rounded subnational projections, with a ruled rationale on
each row; every VOA cell matches exactly). The data-target parity concern
cross_grain_council_tax_stock_england_region is ported.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Two tools/calibrate_uk_national_dataset.py runs on spine-q.h5 with the same
overrides (epochs 1500, family_equal, lr 0.02): this branch's 595-reference
register against PR #904's 424-reference register. All 108 ONS region cells
and 78 of 81 VOA cells fit within 5% (median |error| < 0.1%); the three
band-H cells with no band-H household on the spine sit at -100%, the class
already signed for the local band-H family. Common-row fit is unchanged
within run noise.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@juaristi22
juaristi22 force-pushed the uk-890-target-binding branch from 3fdede4 to 3bb2927 Compare September 11, 2026 19:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant