Bind the Chronicle transport and energy facts on the UK national surface (#890 PR-T) - #904
Bind the Chronicle transport and energy facts on the UK national surface (#890 PR-T)#904juaristi22 wants to merge 8 commits into
Conversation
|
Review at The binding work reproduces. The re-pin commit on its own moves only the membership feed label, so "no existing reference resolves a different fact or value" holds for it. All nine vendored resources carry the new feed identity with matching row counts. Every new resolved value rebuilds from the vendored rows: ONS 04.5 at £43.427bn, Scotland 391m and 499m, Wales 62.4m + 69.0m, DfI 49.6m + 100.5m, NITHC 61.8m + 49.9m, and the BUS0415 factors (England 1.056957928802589, London 1.0) from the four quarter-end means. All five measurement variables exist on the engine at household entity, the two-term Should-fix1. The first applied rescale on the UK surface is a runtime post-pass, not a declaration, and the surface now carries two values for one target. 2. The vendored register's 3. The reviewed exclusion's adjudication is untracked. 4. "Regenerates byte-identical under test" runs nowhere in CI. The workflow sets no pinned-feed path, so the vendored regeneration, the reference regeneration and the BUS0415 pinned-feed tests skip in CI as they do here. The local-surface regeneration test also now defaults to a new Questions
NitsReceipts Part F still carries the placeholder "(filled from … once the run completes)". The two BUS0415 windows share the March 2025 quarter-end, which the basis string declares. #903 overlapSame finding as posted there: five of the seven shared files auto-merge, two conflict ( Could not verifyRegeneration from the pinned feed (no feed here, skipped in CI too), the spine-p baseline runs and the acceptance evidence, and the plan-of-record rulings. Nothing blocks. 1 is the one that matters: it is the first value the UK surface changes rather than declares, and it should be visible on the reference, not only in the run manifest. 2 and 3 are register accuracy; 4 is a CI gap that predates this PR but bites harder now. |
Two tools/calibrate_uk_national_dataset.py runs on spine-q.h5 with the same overrides (epochs 1500, family_equal, lr 0.02): this branch's 595-reference register against PR #904's 424-reference register. All 108 ONS region cells and 78 of 81 VOA cells fit within 5% (median |error| < 0.1%); the three band-H cells with no band-H household on the spine sit at -100%, the class already signed for the local band-H family. Common-row fit is unchanged within run noise. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…fuel duty at FY2024-25 (#890) One commit because the pin rename touches every importer the other changes also touch. ## One Chronicle pin for both surfaces María's ruling on PR #904: the national and local target surfaces are one registry and must read one reviewed Chronicle artifact identity. The base already carried #900, which had moved both to ec7169b; round 1 of this PR re-pinned only the national side to c6f9361 and gave the local surface its own default feed file. This commit removes that split. - uk/national_chronicle_feed.json becomes uk/chronicle_feed.json (uk_runtime.chronicle_feed, load_uk_chronicle_feed, UKChronicleFeed, scope uk_calibration); every reader is renamed. - local_target_census restates the pin from that declaration instead of carrying its own digest; the local validation-level loader checks the register's source_feed against the same declaration. - The local census, local references and membership, and the local validation-level register regenerate from c6f9361: 20,430 active, 1,586 no-fact, 513 signed deferred, 1 signed compile error, every value unchanged; only the feed labels move. - The separate .codex-work/consumer_facts_uk_local.jsonl default and the CHRONICLE_UK_LOCAL_FACTS override are gone; both regeneration tests read the one pinned feed. docs/uk-chronicle-feed-repin.md describes the single re-pin procedure. ## Vahid's should-fixes and María's fiscal-year ruling Vahid's review of PR #904 and María's second ruling. Should-fix 1: the BUS0415 alignment is a declaration on the reference, not a runtime post-pass keyed on a family and concept string. The four dft.bus_fare_receipts.* contract rows declare uprating_index: dft.local_bus_fares_index; the shared authoring gains an uprating_appliers hook so the generator applies the UK applier while authoring and the membership records the applied value with index, factor and value_before_uprating on the hold; the runtime applies the same applier keyed on the declaration (apply_declared_uk_uprating; an index no applier implements refuses the compile). Membership, both parity receipts and the run manifest now carry one value (England 3,417,388,656 -> 3,612,036,036; London x1). Net support declares nothing and holds unchanged. Should-fix 2: the BUS0415 series is read from the vendored dft_bus_value_anchors.json (refused if its feed identity differs from the pin), so uk_runtime.ledger_targets is a real reader and the production factors reproduce in CI without the licensed feed. The vendor register separates consumers (present readers, test-checked by import and source) from planned_consumers (the spine half). María's ruling: fiscal-year sources bind the base data's FY2024-25 or CY2025, never a later fiscal-year projection. obr.fuel_duties_cars pins period_value 2024 and binds GBP 14.4bn (the April 2024 vintage's FY2024-25 figure) held to 2025, like the HMRC CGT rows, instead of the FY2025-26 GBP 15.9bn projection. On the spine-p baseline the row now solves to -16.7% (from -41.0%; the frame's calibrated cars duty is GBP 12.0bn either way), the round-2 run blocked on the dated exclusion being stale, and the target-fit register is back to empty as on main. The rest of the obr family still binds FY2025-26 projections by the inherited policy; that is flagged for María, not changed here. Also: the UK net-support row's reason id is replaced_by_heterogeneous_country_legs with each leg's scope spelled out; the production-2023 parity receipt drops the fuel row (a fact pinned at FY2024 is not at-or-before 2023); receipts Part G; UK spec digest re-pinned. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Round 2 at 1. Alignment as a declaration. The four 2. 3. Adjudication. Moot: the entry is retired. With the cars fuel duty bound at FY2024-25 (below) the row solves to −16.7%, the round-2 run blocked on the exclusion being stale, and the register is back to empty as on main (receipts Part G). The plan of record lives outside the tree by repo convention (plans are not committed), same as #903; its rulings are summarised in the body. 4. CI. The separate local feed default is gone: María's ruling is that the national and local surfaces share one pin, so Questions. 2023 factors (England 0.99368, London 0.99769, below one from the fare cap) and the Also in this round (María's ruling). The cars fuel duty binds at FY2024-25 ( #903 overlap. Agreed on the order; this PR rebases after #903 lands and Nits: the Part F placeholder is gone. Verification is in the body: |
The national feed moves from ec7169b to c6f9361, the merge of chronicle #258 on top of #255: the transport and energy packages #890 binds (HMRC hydrocarbon oils, OBR fuel duty by vehicle category, DESNZ pump prices, DfT licensed cars by fuel type, Scotland/Wales/NI bus finance, ORR rail finance and fares index, NEED consumption, Ofgem cap levels by period and region, ONS Consumer Trends COICOP expenditure, NTS0313/NTS0621 bus-use frequency). 138,847 rows; the consumer-fact schema digest moved with the new quarter and week period types. No existing national reference resolves a different fact or value: 415 active, 7 deferred, 8 signed out, and the compile-parity receipts regenerate unchanged; only the membership's feed label moves. The local census, validation levels and local references keep their separately reviewed ec7169b pin, so the local regeneration test now reads its own default file (.codex-work/consumer_facts_uk_local.jsonl) instead of falling back to the national one; the runbook records both locations. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Spine stages and target diagnostics need publisher values without reading the feed at build time. tools/vendor_uk_ledger_facts.py copies the facts named by uk/ledger_fact_vendor_selections.json into nine per-concern resources (DfT bus receipts, support, fares index and NTS quintile trips; HMRC clearances, DESNZ prices, OBR receipts by vehicle category and ONS population; DfT licensed cars by fuel type; NEED consumption; Ofgem cap levels and benchmark consumption for FY2024-25; NTS bus-use frequency; devolved bus finance; ORR rail finance; ONS COICOP expenditure). Every row carries the fact keys, source digest and value verbatim plus the national feed identity it was taken from; selectors use the reference-selector vocabulary and are matched by the same function, so an unknown field or a changed match count refuses. A test regenerates each resource byte-identical from the pinned feed. This retires the hand-copied chronicle_candidate anchors one concern at a time. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…de London (#890 B) The national reference generator now pins a contract target whose only declared level is region at region level with its E12 code, so the two London bus rows match the region-stamped BUS05i facts (fares GBP 1,347.4m, net support GBP 1,130.2m, year ending March 2025) instead of a country row that never exists. The compiler refuses a region pin outside the nine English region codes the geography ladder already carries and refuses a region-pinned reference that resolves anything but a region-stamped fact, so no second roster is needed. England outside London is now a derived partition member: the England and London rows are bound, Chronicle stamps the outside-London rows at statistical_scope, and test_uk_bus_targets keeps asserting the publisher partition reconciles within GBP 1,000. The UK rows stay signed out because Wales publishes no fare receipts. Active references 415 -> 417; the compile-parity receipts gain the two London rows as ledger-only entries. Contract notes on the England rows now describe the compile-time BUS0415 alignment and the in-stage NTS0705a shaping rather than "pending #790". Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…index (#890 C) BUS05ai receipts are published for the year ending March; the calibration year is a calendar year. The national compile now transports each dft_local_bus fare-receipts row by the ratio of the mean BUS0415 local bus fares index at the four quarter-ends inside the calibration calendar year to the mean at the four quarter-ends inside the receipts' April-to-March fiscal year, per publisher series (England E92000001, London E12000007). On the pinned feed that is x1.05696 for England (GBP 3,417.4m -> 3,612.0m) and x1.0 for London, whose fares were frozen. Net support is never aligned (#789 scope item 5). A missing quarter-end refuses the reference rather than guessing. The factor is declared on the spec metadata with the index concept, the series geography, the months used, their source record ids, the two means, the pre-alignment value and the adjudication (calendar-year basis, ruling 2026-09-10), so run manifests and receipts show the publisher facts the transport stands on. The compile-parity receipts move by the same factors. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…evolved bus legs (#890 E1, G, F1) Eight net-new national targets, all pinned Chronicle facts, in registry scope with their no-incumbent-ancestor declarations: - obr.fuel_duties_cars binds the OBR fuel duty receipts from cars (FY2025 projection GBP 15.9bn) on the engine's cars-only fuel_duty. The all-road-users obr.fuel_duties row is signed out of the reference surface and its #757 calibration measure exclusion retires with it. - orr.rail_government_support binds ORR table 7270 total government support (FY2024 GBP 21.6bn, Great Britain, so Northern Ireland households are filtered out of the measure) on rail_subsidy_spending, replacing the incumbent's post-calibration rail scaler. - ons.household_energy_expenditure binds ONS Consumer Trends COICOP 04.5 (CY2025 GBP 43.4bn) on domestic_energy_consumption, with the other-fuels scope difference noted. - The devolved local-bus legs: Scotland passenger revenue (GBP 391m) and all government support (GBP 499m) from the 2025 Scottish Transport Statistics; Wales public support through local authorities (concessionary fares plus support to operators, GBP 131.5m gross) from the StatsWales outturn; Northern Ireland Translink bus receipts (GBP 150.1m, Ulsterbus plus Metro/Glider) from DfI; and Northern Ireland public transport support as a joint bus-plus-rail cell (PSO plus concessionary compensation, GBP 111.7m) from the NITHC accounts, measured on bus_subsidy_spending + rail_subsidy_spending. The UK net-support row is signed out as covered by the four country legs; the UK fares row stays out because Wales publishes no fare revenue. Active references 417 -> 424; contract targets 236 -> 244 (registry scope 202 -> 210, unmapped declarations 19 -> 27). Compile-parity receipts gain the new rows as ledger-only entries. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…eceipts (#890) On the newest gated national spine (spine-p) with the PR-T surface, every new row fits inside the 25% bound after calibration except obr.fuel_duties_cars at -25.6% (from -46.6% before the solve). That row gets a dated reviewed target-fit exclusion (approved 2026-09-11, expires 2026-10-11, naming the baseline run) because both of its fixes are the spine half: the LCFS car-count predictor and the donor litre-proxy uprating. The row stays bound; only the release fence defers. A second run with the entry in force passes the battery. The receipts record what the fit costs today: the London bus rows, the Scotland and Northern Ireland legs and the ORR rail row are closed by weight stretch on the households that carry the imputed columns (73-85% of London bus mass and 71% of rail support mass on households stretched more than 5x), which is the distortion the value-side stage work in PR-S removes; ONS household energy expenditure already sits on the frame at design weights. The ORR 7271 all-sources figure is recorded for the open 7270-versus-7271 rail ruling. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…fuel duty at FY2024-25 (#890) One commit because the pin rename touches every importer the other changes also touch. María's ruling on PR #904: the national and local target surfaces are one registry and must read one reviewed Chronicle artifact identity. The base already carried #900, which had moved both to ec7169b; round 1 of this PR re-pinned only the national side to c6f9361 and gave the local surface its own default feed file. This commit removes that split. - uk/national_chronicle_feed.json becomes uk/chronicle_feed.json (uk_runtime.chronicle_feed, load_uk_chronicle_feed, UKChronicleFeed, scope uk_calibration); every reader is renamed. - local_target_census restates the pin from that declaration instead of carrying its own digest; the local validation-level loader checks the register's source_feed against the same declaration. - The local census, local references and membership, and the local validation-level register regenerate from c6f9361: 20,430 active, 1,586 no-fact, 513 signed deferred, 1 signed compile error, every value unchanged; only the feed labels move. - The separate .codex-work/consumer_facts_uk_local.jsonl default and the CHRONICLE_UK_LOCAL_FACTS override are gone; both regeneration tests read the one pinned feed. docs/uk-chronicle-feed-repin.md describes the single re-pin procedure. Vahid's review of PR #904 and María's second ruling. Should-fix 1: the BUS0415 alignment is a declaration on the reference, not a runtime post-pass keyed on a family and concept string. The four dft.bus_fare_receipts.* contract rows declare uprating_index: dft.local_bus_fares_index; the shared authoring gains an uprating_appliers hook so the generator applies the UK applier while authoring and the membership records the applied value with index, factor and value_before_uprating on the hold; the runtime applies the same applier keyed on the declaration (apply_declared_uk_uprating; an index no applier implements refuses the compile). Membership, both parity receipts and the run manifest now carry one value (England 3,417,388,656 -> 3,612,036,036; London x1). Net support declares nothing and holds unchanged. Should-fix 2: the BUS0415 series is read from the vendored dft_bus_value_anchors.json (refused if its feed identity differs from the pin), so uk_runtime.ledger_targets is a real reader and the production factors reproduce in CI without the licensed feed. The vendor register separates consumers (present readers, test-checked by import and source) from planned_consumers (the spine half). María's ruling: fiscal-year sources bind the base data's FY2024-25 or CY2025, never a later fiscal-year projection. obr.fuel_duties_cars pins period_value 2024 and binds GBP 14.4bn (the April 2024 vintage's FY2024-25 figure) held to 2025, like the HMRC CGT rows, instead of the FY2025-26 GBP 15.9bn projection. On the spine-p baseline the row now solves to -16.7% (from -41.0%; the frame's calibrated cars duty is GBP 12.0bn either way), the round-2 run blocked on the dated exclusion being stale, and the target-fit register is back to empty as on main. The rest of the obr family still binds FY2025-26 projections by the inherited policy; that is flagged for María, not changed here. Also: the UK net-support row's reason id is replaced_by_heterogeneous_country_legs with each leg's scope spelled out; the production-2023 parity receipt drops the fuel row (a fact pinned at FY2024 is not at-or-before 2023); receipts Part G; UK spec digest re-pinned. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
#904) Main merged the calibration target hierarchy (schema 8, #855) after round 2. The eight #890 targets now declare a catalog category and a label (four new providers: ORR, Welsh Government, DfI Northern Ireland, NITHC; six new categories; the cars fuel duty under obr.efo_receipts), and the committed references carry the contract-derived hierarchy lookups the generator emits for the 196 contract targets with an active reference. What the rebase exposes and this commit does not fix: main's hierarchy completion requires dimension labels on every selected Chronicle fact (chronicle#261), which no artifact carries, so on main and here alike the generators and compile_uk_target_registry return nothing against the pinned feed (0 compiled / 424 unsupported). The reference surfaces are the round-2 regenerations plus those lookups; the feed-backed regeneration and compile tests skip in CI and fail locally until #261 or the interim fallback that issue announces. Vendored resources still regenerate byte-identical. The nine English regions join Microcosm's UK geography label catalog: the London fact carries no geography name and the catalog had only country codes, so the region leg would be refused at compile once labels arrive; a test checks the national region roster is covered. Conflicts: measure-exclusion counts to the merged truth (47 entries, tranche 35); the UK spec digest re-pinned; the target-fit register and its tests are main's. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
3fdede4 to
3bb2927
Compare
Target-binding half of #890 (PR-T of the two-PR plan in
repos/uk-890-implementation-plan.md; the spine half, PR-S, follows on main). Everything here binds pinned Chronicle facts and lets the solver calibrate — the #790 ruling — and nothing in it changes an imputed value.Commits
c6f9361(chronicle Add state-level SNAP benefit hard targets from USDA FNS administrative data #255 + UK firm generator: net-VAT liability missing the 0.20 rate factor (inherited from firm-microsim-paper pre-fix) #258: 138,847 rows) as one declaration for both grains. Round 2 renameduk/national_chronicle_feed.jsontouk/chronicle_feed.jsonand made the local census, local references and local validation levels restate that pin instead of carrying their own (the base already includes Bind UK census households from Chronicle and map NI Data Zones with NISRA's published lookup (#887) #900, which had moved both toec7169b). No existing reference on either surface resolves a different fact or value: national 415/7/8 unchanged before the new bindings, local 20,430 active / 1,586 no-fact / 513 signed deferred / 1 signed compile error unchanged; only the feed labels move.tools/vendor_uk_ledger_facts.py+uk/ledger_fact_vendor_selections.jsoncopy the facts the stages and diagnostics need into nine per-concern resources, verbatim, with the feed identity; a test regenerates them byte-identical from the pin. This is the doctrine "fixtures of publisher facts come from Chronicle" applied to build resources and retires thechronicle_candidateanchors one concern at a time (PR-S consumes them).dft.bus_fare_receipts.*contract rows declareuprating_index: dft.local_bus_fares_index; the generator applies it while authoring (membership records the applied value withindex,factor,value_before_uprating) and the runtime applies the same applier keyed on the declaration, so membership, parity receipts and the run manifest carry one value (England 3,417,388,656 → 3,612,036,036, ×1.05696; London ×1.0). Net support declares no index and is never aligned. The index series is read from the vendoreddft_bus_value_anchors.json, refused if its feed identity differs from the pin, so the production factors reproduce in CI without the licensed feed. At target period 2023 (production parity receipt) the same alignment gives England ×0.99368 and London ×0.99769, below one because the fare cap held index growth inside calendar 2023 under the fiscal-year mean.obr.fuel_duties_cars(cars receipts) on the engine's cars-onlyfuel_duty, bound at FY2024-25 (period_value: 2024, GBP 14.4bn, held to 2025 like the CGT rows) per María's ruling that a fiscal-year source binds the base data's FY2024-25 or CY2025 and never a later projection; every row of the vehicle-type release is the April 2024 vintage, so this is that vintage's figure for the base year (its all-vehicle FY2024-25 total overshoots the March 2026 outturn by 1.4%).obr.fuel_dutiesis signed out and its Swap the microcosm-built UK spine into the national build and retire the incumbent feed #757 measure exclusion retired. Open for María: the rest of theobrfamily still binds FY2025-26 projections at period 2025 by the inherited uk-data policy;orr.rail_government_support(7270, GBP 21.6bn, GB);ons.household_energy_expenditure(COICOP 04.5, CY2025 GBP 43.4bn); Scotland passenger revenue and government support; Wales public support (local-authority route); NI Translink receipts; NI public transport support as a joint bus-plus-rail cell.Round 2 (Vahid's review)
test_childcare_and_bus_references_compile_with_declared_provenancenow pins the applied values and checks them against the membership.consumers): the register now separatesconsumers(present readers; a test imports each module and checks it names the file) fromplanned_consumers(PR-S). One resource has a present reader, the BUS0415 series.covered_by_country_legsis nowreplaced_by_heterogeneous_country_legswith the scope of each leg spelled out. Nits: the Part F placeholder is gone.Rebase onto main (after #903, #894, #855)
Main's schema-8 hierarchy now requires a catalog
category_idand alabelon every contract target: the eight #890 targets declare them (four new providers, six new categories, the cars fuel duty underobr.efo_receipts), and the committed references carry the contract-derived hierarchy lookups the generator emits. What the rebase exposes and does not fix: main's hierarchy completion needs dimension labels on every selected Chronicle fact (chronicle#261, María's), which no artifact carries, so on main and on this branch alike the generator andcompile_uk_target_registryreturn nothing against the pinned feed (0 / 424 unsupported). The reference surfaces here are the round-2 regenerations plus those lookups; the feed-backed regeneration tests skip in CI and are blocked locally until #261 or the interim Microcosm fallback that issue announces. The nine English regions join Microcosm's UK geography label catalog (UK_GEOGRAPHY_ID_TO_LABEL): the London fact carries no geography name and the catalog had only country codes, so the region leg would otherwise be refused at compile once labels arrive. Vendored resources still regenerate byte-identical. Receipts Part H.Surface
Contract 236 → 244 targets (registry 202 → 210); active references 415 → 424; 7 deferred; signed out 8 → 7. National runtime compile 424 / 0 unsupported on the pinned feed. Compile-parity receipts regenerate with the new rows as ledger-only entries.
Receipts
experiments/890-fuel-bus-energy-receipts.md; licensed evidence underdata/ukds/acceptance/890-fuel-bus/.Open rulings carried in the plan
Verification
Rebased head
3bb29277on main3094bfe8. Local lanes on that tree:uk2,089 tests, 4 failures, all four the chronicle#261-blocked feed-backed cases (test_committed_surfaces_regenerate_from_pinned_feed[national],[local],test_childcare_and_bus_references_compile_with_declared_provenance,test_bus0415_alignment_on_the_pinned_feed, which skip in CI and fail on main the same way), everything else green (the pre-existing eFRS parity failure is gone, retired on main by #903);shared-spec1,903 tests, 0 failures.tools/ci_test_groups.py --verify,census_uk_local_targets --check,census_uk_data_target_parity --check,vendor_uk_ledger_facts --checkandruffclean. Round 2 (pre-rebase,3fdede45) baseline runspr-t-round2-spine-pandpr-t-round2b-spine-pare in receipts Part G.Not in this PR
Every imputed value is unchanged: the car-count predictor, donor uprating, bus incidence, the bus rakes and the energy repricing are PR-S. No dense candidate run.
🤖 Generated with Claude Code