diff --git a/changelog.d/uk-net-new-stages-685.added.md b/changelog.d/uk-net-new-stages-685.added.md new file mode 100644 index 000000000..02dd81320 --- /dev/null +++ b/changelog.d/uk-net-new-stages-685.added.md @@ -0,0 +1,2 @@ +Add UK UC deduction attributes and the mortgage- and consumer-debt segment of +the WAS wealth chain, with source, support, graph, and release-gate contracts. Two generic operation kinds join the source vocabulary (`map_uniform_to_banded_rate`, `map_uniform_to_categorical`), and four benunit columns join the reviewed UK export surface (`uc_deduction_random_draw`, `uc_deduction_type_random_draw`, `uc_latent_deduction_rate`, `uc_deduction_combination`). diff --git a/experiments/685-net-new-stages-receipts.md b/experiments/685-net-new-stages-receipts.md new file mode 100644 index 000000000..436d0060a --- /dev/null +++ b/experiments/685-net-new-stages-receipts.md @@ -0,0 +1,167 @@ +# #685 net-new UK stages receipts + +## Part A — WAS debt source audit + +The I1 audit used the manifest-pinned Wealth and Assets Survey round-8 +household tab (15,128 rows; licensed source bytes remain outside the +repository). It found no missing or negative values in `HMortGR8`: 10,560 +rows were zero and 4,568 were positive. +`consumer_debt`, defined as `HFINWR8_SUM - HFINWNTR8_exSLC_Sum`, had no +missing or negative values: 7,999 rows were zero and 7,129 were positive. The +declared zero clip is therefore a no-op on this vintage while remaining part +of the source contract. + +`Ten1R8` confirmed the mortgage-tenure predictor. Mortgage-positive shares +were 6.9% for code 1 (8,135 rows), 99.9% for code 2 (3,862), 100% for code 3 +(64), 2.4% for code 4 (2,908), 10.8% for code 5 (157), and 0% for code 6 +(one row); the single `-8` sentinel maps to false. The donor indicator is +therefore `Ten1R8 in {2, 3}`, paired with recipient +`tenure_type == OWNED_WITH_MORTGAGE`. + +Using `R8xshhwgt`, the donor mortgage-positive share was 0.3226 and mean +mortgage debt was £46,915 per household. The donor consumer-debt-positive +share was 0.5296 and mean consumer debt was £3,623 per household. + +### Realized spine (I5, twin A, design weights) + +Measured on the stage's own cleaned donor frame and on the twin-A spine +(`debt_receipt_twin_a.json`, licensed side): + +| Column | Donor share > 0 | Spine share > 0 | Donor mean / hh | Spine mean / hh | Spine / donor | +|---|---|---|---|---|---| +| `mortgage_debt` | 0.3226 | 0.3069 | £46,915 | £55,944 | 1.19 | +| `consumer_debt` | 0.5296 | 0.5263 | £3,623 | £4,727 | 1.30 | + +Incidence is donor-faithful: among mortgage-tenure households the spine's +mortgage-positive share is 0.9992 against the donor's 0.9991, and among the +rest 0.0548 against 0.0418. Levels sit above the donor: the conditional +mortgage among mortgage-tenure households is £170,943 against the donor's +£143,996 (1.19). The segment conditions on E5's imputed property and +financial-wealth columns, whose own levels the E5 acceptance recorded above +the donor means, so the debt levels inherit that surface; there is no +incumbent comparison (uk-data#426 never merged). Recorded as a signed +observation, not a defect. E5's fourteen wealth columns are byte-equal +between the control build and the E9 twins (attribution, Part E). + +## Part B — UC deduction attribute contract + +The `uc_deduction_attributes` stage writes four benefit-unit columns after +`uc_capital_coherence`: two identity-keyed draws rounded to float32 and +clamped below one, a region-conditioned pre-cap latent deduction rate, and a +PolicyEngine-UK deduction-combination enum member name. It assigns latent +attributes to every benefit unit; the nonzero latent-rate share is not a UC +caseload statistic. The committed DWP distribution resource mirrors the +PolicyEngine-UK 2.92.1 parameter tree, and hermetic and engine-bearing tests +pin the resource, mapping, enum names, permutation invariance, and held-draw +round trip. + +### Realization (stage receipt, twin A) + +The `latent_attribute_realization` stage-health gate passed on both twins: +30 cells (13 regions, 11 bands, 7 combinations), 0 coherence violations, +worst deviation 0.63 of its four-sigma band. The gate holds the unweighted +realization; the first licensed run had gated the weighted share against an +unweighted binomial band and was refused on one band cell by weight variance +alone (design effect 1.4 among assigned units), which is why the receipt now +carries both figures. + +### Engine round-trip (policyengine-uk 2.92.1 on the rebased twin A, main `47c74225` with #835; `roundtrip_uc_deductions_e9r_twin_a.json`) + +At 2024 (25% cap) and 2025 (15% cap), row for row across all 61,211 benefit +units: `uc_has_deduction` equals `would_claim_uc & award > 0 & latent > 0` +with 0 disagreements; `combination != NONE` iff `latent > 0` with 0 +disagreements; the engine's fallback recomputed from the held draws equals +the held attributes on every claimant with a positive award (8,303 at 2024, +8,232 at 2025; rate max abs diff 0.0, 0 combination mismatches); no +non-claimant carries a positive `uc_deductions`. 24,781 non-claimant units +carry a positive latent rate, the documented latent semantics. + +Per-household statistics among UC recipients (`universal_credit > 0`), +compared with pe-uk's validation table and DWP: + +| Statistic | 2024 | 2025 | pe-uk | DWP | +|---|---|---|---|---| +| Mean monthly deduction among deducting | £66.81 | £50.66 | £66 / £50 | £67–68 / £51–54 | +| At-cap share of recipients | 0.145 | 0.263 | 0.134 / 0.264 | 0.13–0.14 / 0.21 | +| Above-cap share of recipients | 0.027 | 0.027 | 0.018 | 0.02 | + +Incidence among recipients must be read on unique rows: CGT clone and +band-donor households (50.1% of recipient rows) carry the completed +attributes copied from their source rows and double-count draws. On the +4,141 (2024) and 4,107 (2025) unique recipient rows the unweighted incidence +is 0.4844 and 0.4858 against a region-mix expectation of 0.470, z = 1.84 and +2.00 (within three sigma); the weighted figure is 0.4914 and 0.4929 with a +design effect of 1.56. The pre-rebase twins (before #835's reporter redraw) gave z = 2.28 and +2.40 on the same statistic; the six E9 columns are byte-identical between the two runs. The per-unit share over all 30,455 unique benefit +units is 0.470, as declared. + +Reported, not accepted (they scale with the UC caseload gap, #701 and +uk-data#452): 2.50m weighted deducting benefit units and £1.52bn a year of +deductions at 2025. + +### Identity (`verify_uk_identity_stability.py --check e9`, twin A) + +PASS: the four columns recomputed from `benunit_id` and region in original +and permuted row order are identical and equal the stored columns on all +30,455 unique benefit units; 30,756 copied rows excluded and counted. + +## Part C — bus-fare closure on the landed E6 stages + +The #685 bus construction requirement is already present in E6; this +increment adds no bus-stage code. + +| Output | Stage | Manifest operation | Support clip | Bounds | Export allow-list | Coverage (family status) | +|---|---|---|---|---|---|---| +| `bus_fare_spending` | `lcfs_consumption` | `fit_weighted_qrf_chain` | Declared; stage-health high/low allowances both zero | `[0, 9000]` | `household.bus_fare_spending` | family status: `required_at_build` | +| `bus_subsidy_spending` | `etb_services` | `fit_weighted_qrf_chain` | Declared; stage-health high/low allowances both zero | `[0, 20000]` | `household.bus_subsidy_spending` | family status: `required_at_build` | + +## Part D — licensed bus measurement hand-off + +Measured on twin A at design weights (`part_d_bus_totals_twin_a.json`, +`part_d_fare_gradient_twin_a.json`; identical on spine-l, since E6 is +unchanged). England is the publisher's support; the UK rows have no fact. + +| Geography | Fare spine £m | Fare fact £m | Ratio | Support spine £m | Support fact £m | Ratio | +|---|---|---|---|---|---|---| +| UK | 2,182.0 | — | — | 2,548.1 | — | — | +| England | 1,909.4 | 3,417.4 | 0.56 | 2,140.1 | 3,024.9 | 0.71 | +| London | 377.8 | 1,347.4 | 0.28 | 311.6 | 1,130.2 | 0.28 | +| England outside London | 1,531.5 | 2,070.0 | 0.74 | 1,828.5 | 1,894.7 | 0.97 | + +Weighted nonzero shares: fares 0.131 (UK), subsidy 0.544 (UK). + +Fare gradient by household income quintile (England, ratio to the England +mean fare) against NTS0705a local-bus trips per person (ratio to the quintile +mean): Q1 1.05 vs 1.68, Q2 0.85 vs 1.32, Q3 0.90 vs 0.88, Q4 0.97 vs 0.65, +Q5 1.23 vs 0.46. The LCFS-imputed spending gradient runs the wrong way +against NTS trips, which is the shape the incumbent's post-calibration +reshape corrected; the disposition is #790's, the England-support targets +are #789's. + +## Part E — twins, attribution, parity, battery (I5) + +- **Twins**: two full licensed builds of the rebased branch (`e9r-twin-a`, + `e9r-twin-b`, on main `47c74225` with #835 and #844) are payload-identical + (`compare_uk_h5_payload.py`, `payload_identical: true`); both run 28 stages, + 113,649 / 61,211 / 52,846 rows, household mass 29,247,433.0, and pass the + spine battery 15 of 15 (`uk_stage_was_wealth_support` now checks 15 columns + with zero clipped rows). The pre-rebase pair (`e9-twin-a`, `e9-twin-b`, + 27 stages, main `90293e0a` plus the #844 fixes) was payload-identical too, + and the six E9 columns are byte-identical across the two pairs. +- **Attribution against the control** (`control-47c74225`, built from that + main with the same inputs): the only columns present on the twins and not + on the control are the six E9 columns; no common column differs in any + entity; E5's fourteen columns are byte-equal + (`attribution_control47c74225_vs_e9r_twin_a.json`; the payload tool's + per-table `columns_only_right` lists exactly the same six). +- **Strict parity** against the pinned eFRS reference: the twin and the + control produce the identical set of 15 unsigned beyond-band columns + (`bus_fare_spending`, `bus_subsidy_spending`, `corporate_wealth`, + `diesel_spending`, `education_consumption`, `employment_sector`, + `household_furnishings_consumption`, `main_residence_value`, + `other_residential_property_value`, `petrol_spending`, + `private_pension_wealth`, `restaurants_and_hotels_consumption`, `savings`, + `sic_industry_division`, `transport_consumption`). They pre-date E9 and + belong to the 2.92.1 recognition re-record (PR #749's lane); E9 adds no + unsigned difference, and its six columns match the six `net_new_column` + entries. diff --git a/packages/microcosm-build/src/microcosm/build/source_manifest.py b/packages/microcosm-build/src/microcosm/build/source_manifest.py index dc7c766db..e7681bdbe 100644 --- a/packages/microcosm-build/src/microcosm/build/source_manifest.py +++ b/packages/microcosm-build/src/microcosm/build/source_manifest.py @@ -123,6 +123,8 @@ "iterative_proportional_fit", "map_columns", "map_coded_amounts", + "map_uniform_to_banded_rate", + "map_uniform_to_categorical", "materialize_hmrc_income_bands_fail_closed", "materialize_rules_engine_predictors", "rank_preserving_allocation", diff --git a/packages/microcosm-build/src/microcosm/build/spec_engine/schema/sources.schema.json b/packages/microcosm-build/src/microcosm/build/spec_engine/schema/sources.schema.json index c9f3ad80f..aa43f8ec7 100644 --- a/packages/microcosm-build/src/microcosm/build/spec_engine/schema/sources.schema.json +++ b/packages/microcosm-build/src/microcosm/build/spec_engine/schema/sources.schema.json @@ -5759,6 +5759,122 @@ } } }, + { + "type": "object", + "additionalProperties": false, + "required": [ + "kind", + "output", + "draw", + "draw_dtype", + "resource", + "distribution", + "incidence_modifier", + "none_value" + ], + "properties": { + "kind": { + "const": "map_uniform_to_banded_rate" + }, + "output": { + "type": "string" + }, + "draw": { + "type": "string" + }, + "draw_dtype": { + "type": "string", + "enum": [ + "float32" + ] + }, + "resource": { + "type": "string" + }, + "distribution": { + "type": "string" + }, + "incidence_modifier": { + "type": "object", + "additionalProperties": false, + "required": [ + "column", + "entity", + "table" + ], + "properties": { + "column": { + "type": "string" + }, + "entity": { + "type": "string" + }, + "table": { + "type": "string" + } + } + }, + "none_value": { + "type": "number" + } + } + }, + { + "type": "object", + "additionalProperties": false, + "required": [ + "kind", + "output", + "draw", + "draw_dtype", + "resource", + "distribution", + "gate", + "none_value" + ], + "properties": { + "kind": { + "const": "map_uniform_to_categorical" + }, + "output": { + "type": "string" + }, + "draw": { + "type": "string" + }, + "draw_dtype": { + "type": "string", + "enum": [ + "float32" + ] + }, + "resource": { + "type": "string" + }, + "distribution": { + "type": "string" + }, + "gate": { + "type": "object", + "additionalProperties": false, + "required": [ + "column", + "positive" + ], + "properties": { + "column": { + "type": "string" + }, + "positive": { + "type": "boolean" + } + } + }, + "none_value": { + "type": "string" + } + } + }, { "type": "object", "additionalProperties": false, @@ -6915,6 +7031,12 @@ "type": "string" } }, + "debt_segment_predictors": { + "type": "array", + "items": { + "type": "string" + } + }, "region_remap": { "type": "object", "minProperties": 1, diff --git a/packages/microcosm-build/src/microcosm/build/uk/country_package.json b/packages/microcosm-build/src/microcosm/build/uk/country_package.json index 275589254..b3e38b70a 100644 --- a/packages/microcosm-build/src/microcosm/build/uk/country_package.json +++ b/packages/microcosm-build/src/microcosm/build/uk/country_package.json @@ -132,6 +132,11 @@ "kind": "legacy_json", "schema_id": "legacy_json" }, + { + "path": "dwp_uc_deduction_distributions.json", + "kind": "legacy_json", + "schema_id": "legacy_json" + }, { "path": "nhs_consumption_by_age_gender.json", "kind": "legacy_json", @@ -237,6 +242,11 @@ "kind": "legacy_json", "schema_id": "legacy_json" }, + { + "path": "uc_deduction_support_bounds.json", + "kind": "legacy_json", + "schema_id": "legacy_json" + }, { "path": "local_binding_adjudications.json", "kind": "legacy_json", diff --git a/packages/microcosm-build/src/microcosm/build/uk/dwp_uc_deduction_distributions.json b/packages/microcosm-build/src/microcosm/build/uk/dwp_uc_deduction_distributions.json new file mode 100644 index 000000000..811d1078f --- /dev/null +++ b/packages/microcosm-build/src/microcosm/build/uk/dwp_uc_deduction_distributions.json @@ -0,0 +1,68 @@ +{ + "version": 1, + "country": "uk", + "policy": "Pinned estimation distributions for identity-keyed Universal Credit deduction attributes. The mappings mirror PolicyEngine-UK 2.92.1 exactly; statutory caps and award floors remain engine-owned.", + "source": { + "publisher": "Department for Work and Pensions", + "publication_url": "https://www.gov.uk/government/statistics/universal-credit-quarterly-statistics-29-april-2013-to-12-february-2026/universal-credit-deductions-statistics-march-2025-to-february-2026", + "ods_url": "https://assets.publishing.service.gov.uk/media/69fb3ea22a6137e93226b7ce/universal-credit-deductions-march-2025-to-february-2026.ods", + "ods_sha256": "307ec8fa49a1f1e23db3c59f1282e16609de03444e501824e204b4151e2e5c9b", + "chronicle_package_id": "dwp-uc-deductions-march-2025-february-2026", + "chronicle_candidate": true, + "observation_windows": { + "table_2": "March to May 2025 average, before the Fair Repayment Rate", + "table_5": "March to May 2025 average, before the Fair Repayment Rate", + "table_6": "February 2026 constituency rows" + } + }, + "latent_rate_distribution": { + "parameter_path": "gov.simulation.uc_deductions.latent_rate_distribution", + "calibration_cap": { + "value": 0.25, + "parameter_path": "gov.simulation.uc_deductions.calibration_cap" + }, + "bands": [ + {"name": "UNDER_5", "lower": 0.0, "upper": 0.05, "share": 0.047}, + {"name": "AT_5", "lower": 0.05, "upper": 0.05, "share": 0.03}, + {"name": "FIVE_TO_10", "lower": 0.05, "upper": 0.1, "share": 0.07}, + {"name": "AT_10", "lower": 0.1, "upper": 0.1, "share": 0.01}, + {"name": "TEN_TO_15", "lower": 0.1, "upper": 0.15, "share": 0.04}, + {"name": "AT_15", "lower": 0.15, "upper": 0.15, "share": 0.047}, + {"name": "FIFTEEN_TO_20", "lower": 0.15, "upper": 0.2, "share": 0.03}, + {"name": "AT_20", "lower": 0.2, "upper": 0.2, "share": 0.01}, + {"name": "TWENTY_TO_25", "lower": 0.2, "upper": 0.25, "share": 0.03}, + {"name": "AT_25", "lower": 0.25, "upper": 0.25, "share": 0.133}, + {"name": "OVER_25", "lower": 0.25, "upper": 0.3, "share": 0.02} + ] + }, + "type_combination": { + "parameter_path": "gov.simulation.uc_deductions.type_combination", + "shares": [ + {"name": "ADVANCE_ONLY", "share": 0.327}, + {"name": "THIRD_PARTY_ONLY", "share": 0.06}, + {"name": "GOVERNMENT_ONLY", "share": 0.227}, + {"name": "ADVANCE_AND_GOVERNMENT", "share": 0.18}, + {"name": "ADVANCE_AND_THIRD_PARTY", "share": 0.1}, + {"name": "THIRD_PARTY_AND_GOVERNMENT", "share": 0.03}, + {"name": "ALL_THREE", "share": 0.08} + ] + }, + "region_incidence_factor": { + "parameter_path": "gov.simulation.uc_deductions.region_incidence_factor", + "factors": { + "UNKNOWN": 1.0, + "NORTH_EAST": 1.12, + "NORTH_WEST": 1.09, + "YORKSHIRE": 1.05, + "EAST_MIDLANDS": 0.97, + "WEST_MIDLANDS": 1.0, + "EAST_OF_ENGLAND": 0.93, + "LONDON": 0.94, + "SOUTH_EAST": 0.93, + "SOUTH_WEST": 0.9, + "WALES": 1.05, + "SCOTLAND": 1.09, + "NORTHERN_IRELAND": 1.0 + } + } +} diff --git a/packages/microcosm-build/src/microcosm/build/uk/gates.json b/packages/microcosm-build/src/microcosm/build/uk/gates.json index d290835b5..88cf87605 100644 --- a/packages/microcosm-build/src/microcosm/build/uk/gates.json +++ b/packages/microcosm-build/src/microcosm/build/uk/gates.json @@ -89,9 +89,11 @@ "check": "support_clip", "columns": [ "cash_isa", + "consumer_debt", "corporate_wealth", "gross_financial_wealth", "main_residence_value", + "mortgage_debt", "net_financial_wealth", "non_residential_property_value", "num_vehicles", @@ -104,9 +106,11 @@ ], "max_clipped_low_rows_by_column": { "cash_isa": 0, + "consumer_debt": 0, "corporate_wealth": 0, "gross_financial_wealth": 0, "main_residence_value": 0, + "mortgage_debt": 0, "net_financial_wealth": 0, "non_residential_property_value": 0, "num_vehicles": 0, @@ -119,9 +123,11 @@ }, "max_clipped_high_rows_by_column": { "cash_isa": 0, + "consumer_debt": 0, "corporate_wealth": 0, "gross_financial_wealth": 0, "main_residence_value": 0, + "mortgage_debt": 0, "net_financial_wealth": 0, "non_residential_property_value": 0, "num_vehicles": 0, @@ -135,6 +141,18 @@ }, "notes": "Stage-time support-clip health gate over the WAS wealth receipt; raw FRS mapping stages have no imputation semantics and intentionally have no empty health gate. Allowances pinned at the receipted spine-e baseline (2026-08-26): zero rows clipped on every non-exempt column, so any clipping is a real signal against a clean baseline; a missing allowance fails closed rather than skipping the comparison." }, + { + "id": "uk_stage_uc_deduction_attributes", + "gate": "stage_health", + "phase": "transferred", + "criticality": "release_blocking", + "evidence_absent_blocks": true, + "parameters": { + "stage": "uc_deduction_attributes", + "check": "latent_attribute_realization" + }, + "notes": "Stage-time realization check over region incidence, latent-rate bands, normalized deduction-type combinations, and the exact latent/type/incidence coherence identity. Tolerances are derived from each receipt cell's realized row count; missing evidence blocks." + }, { "id": "uk_stage_lcfs_consumption_support", "gate": "stage_health", @@ -499,7 +517,8 @@ "was_wealth_support_bounds.json", "lcfs_consumption_support_bounds.json", "etb_vat_support_bounds.json", - "etb_services_support_bounds.json" + "etb_services_support_bounds.json", + "uc_deduction_support_bounds.json" ] }, "notes": "Every WAS/LCFS/ETB imputed output covered by support clipping must remain inside committed disclosure-safe donor support bounds. Exact donor ranges are used in-run for clipping; this release gate uses outward-rounded reviewed bounds so unit-record donor minima and maxima are never committed." @@ -555,6 +574,10 @@ "allowed_extra_columns": [ "benunit.child_benefit_opts_out", "benunit.frs_benunit_capital", + "benunit.uc_deduction_combination", + "benunit.uc_deduction_random_draw", + "benunit.uc_deduction_type_random_draw", + "benunit.uc_latent_deduction_rate", "benunit.uc_reported_capital", "household.bus_fare_spending", "household.bus_subsidy_spending", @@ -637,6 +660,18 @@ }, "notes": "The household BRMA assignment must remain inside the PolicyEngine-UK brma enum domain. Evaluated at the assembled boundary, right after frs_brma writes the column: enum membership is a property of the values themselves, so checking it there is identical to checking it at the end and refuses fourteen stages earlier. Take-up signal stays terminal by contrast, because its weighted shares are not the same statistic before and after calibration." }, + { + "id": "uk_uc_deduction_combination_enum_domain", + "gate": "enum_domain", + "phase": "terminal", + "criticality": "release_blocking", + "parameters": { + "columns": [ + "uc_deduction_combination" + ] + }, + "notes": "The benefit-unit UC deduction combination must remain inside PolicyEngine-UK's UCDeductionCombination enum domain." + }, { "id": "uk_student_loan_plan_enum_domain", "gate": "enum_domain", diff --git a/packages/microcosm-build/src/microcosm/build/uk/release_input_coverage_manifest.json b/packages/microcosm-build/src/microcosm/build/uk/release_input_coverage_manifest.json index cc4d78e88..5ed25a380 100644 --- a/packages/microcosm-build/src/microcosm/build/uk/release_input_coverage_manifest.json +++ b/packages/microcosm-build/src/microcosm/build/uk/release_input_coverage_manifest.json @@ -473,7 +473,7 @@ "capital_gains" ], "source_manifest": "source_stages.json", - "source_manifest_sha256": "1128013c99cd3592c06adab25cd0efa7f92667765445e2c6370c6c867a4a3517", + "source_manifest_sha256": "ee3fabfd7958017b71c534fd81070df288f7d4bc87163a0f568708aa73439af0", "source_vintages": { "source": "HMRC Capital Gains Tax statistics, July 2025, Table 2.1a", "survey": "HMRC Capital Gains Tax statistics Table 2.1a and Advani-Summers capital-gains incidence" @@ -496,7 +496,7 @@ "capital_gains" ], "source_manifest": "source_stages.json", - "source_manifest_sha256": "1128013c99cd3592c06adab25cd0efa7f92667765445e2c6370c6c867a4a3517", + "source_manifest_sha256": "ee3fabfd7958017b71c534fd81070df288f7d4bc87163a0f568708aa73439af0", "source_vintages": { "source": "Advani and Summers (2020), Capital Gains and UK Inequality, CAGE Working Paper 465", "survey": "Family Resources Survey 2024-25, SPI synthetic support, and Advani-Summers capital-gains incidence" @@ -525,7 +525,7 @@ "required_mass_change_reason": "E5 source-stage transform preserves household rows and typed household weights; total household mass is conserved.", "rewrites": [], "source_manifest": "source_stages.json", - "source_manifest_sha256": "1128013c99cd3592c06adab25cd0efa7f92667765445e2c6370c6c867a4a3517", + "source_manifest_sha256": "ee3fabfd7958017b71c534fd81070df288f7d4bc87163a0f568708aa73439af0", "source_vintages": { "source": "UK Data Service SN 8856 Effects of Taxes and Benefits household tab, DfT rail fare index, and public NHS activity/cost table.", "survey": "Effects of Taxes and Benefits 1977-2024 and NHS age-gender public table" @@ -545,7 +545,7 @@ "required_mass_change_reason": "E5 source-stage transform preserves household rows and typed household weights; total household mass is conserved.", "rewrites": [], "source_manifest": "source_stages.json", - "source_manifest_sha256": "1128013c99cd3592c06adab25cd0efa7f92667765445e2c6370c6c867a4a3517", + "source_manifest_sha256": "ee3fabfd7958017b71c534fd81070df288f7d4bc87163a0f568708aa73439af0", "source_vintages": { "source": "UK Data Service SN 8856 Effects of Taxes and Benefits household tab and cited VAT anchor resource.", "survey": "Effects of Taxes and Benefits 1977-2024" @@ -576,7 +576,7 @@ "superseded_by": { "reason": "The FRS spine build executes hmrc_cgt_gains_spine, which applies the same HMRC Table 3 amounts redraw directly in source_stages.json before calibration.", "source_manifest": "source_stages.json", - "source_manifest_sha256": "1128013c99cd3592c06adab25cd0efa7f92667765445e2c6370c6c867a4a3517", + "source_manifest_sha256": "ee3fabfd7958017b71c534fd81070df288f7d4bc87163a0f568708aa73439af0", "stage": "hmrc_cgt_gains_spine" } }, @@ -596,7 +596,7 @@ "capital_gains" ], "source_manifest": "source_stages.json", - "source_manifest_sha256": "1128013c99cd3592c06adab25cd0efa7f92667765445e2c6370c6c867a4a3517", + "source_manifest_sha256": "ee3fabfd7958017b71c534fd81070df288f7d4bc87163a0f568708aa73439af0", "source_vintages": { "hmrc_surface": "2023-24", "mapped_build_period": "2024" @@ -610,7 +610,7 @@ "base_candidate_tier": "frs", "calibration_permitted": false, "canonical_source_manifest": "source_stages.json", - "canonical_source_manifest_sha256": "1128013c99cd3592c06adab25cd0efa7f92667765445e2c6370c6c867a4a3517", + "canonical_source_manifest_sha256": "ee3fabfd7958017b71c534fd81070df288f7d4bc87163a0f568708aa73439af0", "effective_mass_requirements": { "charitable_investment_gifts": { "mass_share_denominator": "all_person_effective_mass", @@ -693,7 +693,7 @@ "superseded_by": { "reason": "The FRS spine build executes hmrc_spi_income_spine, which supersedes the June retained-leaves/hmrc_spi_income pair inside source_stages.json.", "source_manifest": "source_stages.json", - "source_manifest_sha256": "1128013c99cd3592c06adab25cd0efa7f92667765445e2c6370c6c867a4a3517", + "source_manifest_sha256": "ee3fabfd7958017b71c534fd81070df288f7d4bc87163a0f568708aa73439af0", "stage": "hmrc_spi_income_spine" } }, @@ -727,7 +727,7 @@ "required_mass_change_reason": "E5 source-stage transform preserves household rows and typed household weights; total household mass is conserved.", "rewrites": [], "source_manifest": "source_stages.json", - "source_manifest_sha256": "1128013c99cd3592c06adab25cd0efa7f92667765445e2c6370c6c867a4a3517", + "source_manifest_sha256": "ee3fabfd7958017b71c534fd81070df288f7d4bc87163a0f568708aa73439af0", "source_vintages": { "source": "UK Data Service SN 9468 Living Costs and Food Survey 2023-24 household/person tabs, NEED 2023 headline energy tables, Ofgem Q2 2026 unit rates, and WAS round-8 bridge donor.", "survey": "Living Costs and Food Survey 2023-24" @@ -748,7 +748,7 @@ "property_wealth" ], "source_manifest": "source_stages.json", - "source_manifest_sha256": "1128013c99cd3592c06adab25cd0efa7f92667765445e2c6370c6c867a4a3517", + "source_manifest_sha256": "ee3fabfd7958017b71c534fd81070df288f7d4bc87163a0f568708aa73439af0", "source_vintages": { "source": "MHCLG dwellings and ONS UK House Price Index December 2025 regional average prices.", "survey": "Public regional property reference" @@ -772,7 +772,7 @@ "employee_pension_contributions" ], "source_manifest": "source_stages.json", - "source_manifest_sha256": "1128013c99cd3592c06adab25cd0efa7f92667765445e2c6370c6c867a4a3517", + "source_manifest_sha256": "ee3fabfd7958017b71c534fd81070df288f7d4bc87163a0f568708aa73439af0", "source_vintages": { "source": "HMRC, Salary sacrifice reform for pension contributions effective from 6 April 2029", "survey": "Family Resources Survey 2024-25 salary-sacrifice respondents and HMRC salary-sacrifice reform analysis" @@ -794,7 +794,7 @@ "student_loan_plan" ], "source_manifest": "source_stages.json", - "source_manifest_sha256": "1128013c99cd3592c06adab25cd0efa7f92667765445e2c6370c6c867a4a3517", + "source_manifest_sha256": "ee3fabfd7958017b71c534fd81070df288f7d4bc87163a0f568708aa73439af0", "source_vintages": { "source": "Explore Education Statistics Table 6a, Higher education total", "survey": "Family Resources Survey 2024-25 and Student Loans Company borrower forecasts for England" @@ -822,12 +822,14 @@ "num_vehicles", "cash_isa", "stocks_and_shares_isa", - "student_loan_balance" + "student_loan_balance", + "mortgage_debt", + "consumer_debt" ], "required_mass_change_reason": "E5 source-stage transform preserves household rows and typed household weights; total household mass is conserved.", "rewrites": [], "source_manifest": "source_stages.json", - "source_manifest_sha256": "1128013c99cd3592c06adab25cd0efa7f92667765445e2c6370c6c867a4a3517", + "source_manifest_sha256": "ee3fabfd7958017b71c534fd81070df288f7d4bc87163a0f568708aa73439af0", "source_vintages": { "source": "Office for National Statistics Wealth and Assets Survey, UK Data Service SN 7215, DOI 10.5255/UKDA-SN-7215-20; local licensed 2006-22 household tab.", "survey": "Wealth and Assets Survey round 8" diff --git a/packages/microcosm-build/src/microcosm/build/uk/source_stages.json b/packages/microcosm-build/src/microcosm/build/uk/source_stages.json index c403cf81b..0e2a89b29 100644 --- a/packages/microcosm-build/src/microcosm/build/uk/source_stages.json +++ b/packages/microcosm-build/src/microcosm/build/uk/source_stages.json @@ -900,6 +900,9 @@ "property_wealth": "DVPropertyR8", "gross_financial_wealth": "HFINWR8_SUM", "net_financial_wealth": "HFINWNTR8_Sum", + "net_financial_wealth_exsl": "HFINWNTR8_exSLC_Sum", + "mortgage_debt": "HMortGR8", + "tenure_code": "Ten1R8", "main_residence_value": "DVhvalueR8", "other_residential_property_value": "DVHseValR8_sum", "non_residential_property_value": "DVBlDValR8_sum", @@ -913,6 +916,8 @@ "stocks_and_shares_isa": "DVIISAVR8_aggr", "cash_isa": "DVCISAVR8_aggr", "student_loan_balance": "Tot_LosR8_aggr - Tot_los_exc_SLCR8_aggr", + "consumer_debt": "max(HFINWR8_SUM - HFINWNTR8_exSLC_Sum, 0)", + "has_mortgage_tenure": "Ten1R8 in {2, 3}", "is_renting": "DVPriRntR8 == 1", "region": "GORR8 via incumbent REGIONS map" } @@ -959,6 +964,9 @@ "integer_outputs": [ "num_vehicles" ], + "debt_segment_predictors": [ + "has_mortgage_tenure" + ], "chain_order": [ "owned_land", "property_wealth", @@ -974,7 +982,9 @@ "savings", "num_vehicles", "student_loan_balance", - "cash_isa" + "cash_isa", + "mortgage_debt", + "consumer_debt" ] }, { @@ -1021,7 +1031,9 @@ "num_vehicles", "cash_isa", "stocks_and_shares_isa", - "student_loan_balance" + "student_loan_balance", + "mortgage_debt", + "consumer_debt" ], "nonnegative_outputs": [ "owned_land", @@ -1036,9 +1048,11 @@ "num_vehicles", "cash_isa", "stocks_and_shares_isa", - "student_loan_balance" + "student_loan_balance", + "mortgage_debt", + "consumer_debt" ], - "notes": "Ports incumbent WAS round-8 wealth imputation with signed E5 differences: exact lower-case column matching replaces the fuzzy r/w fallback; cash ISA uses DVCISAVR8_aggr and stocks-and-shares ISA uses DVIISAVR8_aggr; private pension wealth (WAS total private pension wealth less the current defined-benefit component; DB-type components at the SCAPE rate, money-purchase components at reported fund values) is drawn as its own output private_pension_wealth and no longer folded into corporate_wealth, which keeps only the share-like holdings - pension rights are disregarded capital in every means test (UC Regs 2013 Sch 10 para 10 and the HB/JSA/ESA/IS/SPC parallels; uk-data#452) - and the downstream chain segment conditions on both components; each chain segment starts its own fit and draw RNG streams from a child seed derived deterministically from the declared seed (start_chain respawns from the model seed on every call, so one shared model coupled the k-th target of every segment); corporate_wealth folds stocks-and-shares ISA after drawing corporate_wealth_excl_isa; recipient Northern Ireland regions are mapped to Wales for prediction only; student_loan_balance is allocated by household id rather than the incumbent positional off-by-one. Engine predictors materialize at their native entity and person/benunit values are summed to household, reproducing the incumbent map_to=household semantics; region is one-hot encoded jointly across donor and recipient (the incumbent's dummy encoding), with unmapped donor GOR codes becoming all-zero dummy rows. The WAS and FRS predictor definitions are not fully like-for-like and are ported as-is; raw WAS missing values are blanket-filled with zero; UKDS negative sentinel codes (-9/-8/-7/-6) are recoded to zero for the nonnegative-domain columns the licensed audit found carrying them (vcarnr8: 2 rows; HBedRmR8: 95.8 percent - the bedrooms question is effectively unasked in the WAS household file, predictor-quality revisit registered on microcosm#145) - a signed difference vs the incumbent, which trains on raw sentinels; DVPriRntR8's -9 is structural not-applicable so the is_renting mapping is unchanged; genuinely negative domains are never recoded." + "notes": "Ports incumbent WAS round-8 wealth imputation with signed E5 differences: exact lower-case column matching replaces the fuzzy r/w fallback; cash ISA uses DVCISAVR8_aggr and stocks-and-shares ISA uses DVIISAVR8_aggr; private pension wealth (WAS total private pension wealth less the current defined-benefit component; DB-type components at the SCAPE rate, money-purchase components at reported fund values) is drawn as its own output private_pension_wealth and no longer folded into corporate_wealth, which keeps only the share-like holdings - pension rights are disregarded capital in every means test (UC Regs 2013 Sch 10 para 10 and the HB/JSA/ESA/IS/SPC parallels; uk-data#452) - and the downstream chain segment conditions on both components; each chain segment starts its own fit and draw RNG streams from a child seed derived deterministically from the declared seed (start_chain respawns from the model seed on every call, so one shared model coupled the k-th target of every segment); corporate_wealth folds stocks-and-shares ISA after drawing corporate_wealth_excl_isa; recipient Northern Ireland regions are mapped to Wales for prediction only; student_loan_balance is allocated by household id rather than the incumbent positional off-by-one. The debt segment (fourth chain segment, fourth child seed) reads WAS round-8 derived variables HMortGR8 (total outstanding mortgage debt), HFINWNTR8_exSLC_Sum (net financial wealth excluding student loans; gross less it is the non-student-loan financial liabilities) and Ten1R8 (tenure; codes 2 and 3 are buying with a mortgage and part own/part rent, confirmed by the I1 audit of the pinned tab); UKDS SN 7215 documentation is the codebook. Engine predictors materialize at their native entity and person/benunit values are summed to household, reproducing the incumbent map_to=household semantics; region is one-hot encoded jointly across donor and recipient (the incumbent's dummy encoding), with unmapped donor GOR codes becoming all-zero dummy rows. The WAS and FRS predictor definitions are not fully like-for-like and are ported as-is; raw WAS missing values are blanket-filled with zero; UKDS negative sentinel codes (-9/-8/-7/-6) are recoded to zero for the nonnegative-domain columns the licensed audit found carrying them (vcarnr8: 2 rows; HBedRmR8: 95.8 percent - the bedrooms question is effectively unasked in the WAS household file, predictor-quality revisit registered on microcosm#145) - a signed difference vs the incumbent, which trains on raw sentinels; DVPriRntR8's -9 is structural not-applicable so the is_renting mapping is unchanged; genuinely negative domains are never recoded." }, { "stage": "regional_property_uprating", @@ -2526,6 +2540,71 @@ ], "notes": "Runs after the last universal_credit_reported writer and before CGT cloning. Redraws only SPI post-fill reporters from weighted base-FRS reporter capital within dependent-children band by couple-status cells, then applies the monotone reported-UC OR refresh." }, + { + "stage": "uc_deduction_attributes", + "survey": "DWP Universal Credit deductions statistics, March 2025 to February 2026", + "source": "Department for Work and Pensions supplementary Tables 2, 5 and 6, sha-pinned by Chronicle package dwp-uc-deductions-march-2025-february-2026.", + "grain": "benunit", + "artifacts": [ + { + "role": "deduction_distribution", + "resource": "dwp_uc_deduction_distributions.json", + "kind": "public_aggregate_reference", + "format": "json" + } + ], + "operations": [ + { + "kind": "assign_uniform_draw", + "output": "uc_deduction_random_draw", + "seed": 0 + }, + { + "kind": "assign_uniform_draw", + "output": "uc_deduction_type_random_draw", + "seed": 0 + }, + { + "kind": "map_uniform_to_banded_rate", + "output": "uc_latent_deduction_rate", + "draw": "uc_deduction_random_draw", + "draw_dtype": "float32", + "resource": "dwp_uc_deduction_distributions.json", + "distribution": "latent_rate_distribution", + "incidence_modifier": { + "column": "region", + "entity": "household", + "table": "region_incidence_factor" + }, + "none_value": 0.0 + }, + { + "kind": "map_uniform_to_categorical", + "output": "uc_deduction_combination", + "draw": "uc_deduction_type_random_draw", + "draw_dtype": "float32", + "resource": "dwp_uc_deduction_distributions.json", + "distribution": "type_combination", + "gate": { + "column": "uc_latent_deduction_rate", + "positive": true + }, + "none_value": "NONE" + } + ], + "outputs": [ + "uc_deduction_random_draw", + "uc_deduction_type_random_draw", + "uc_latent_deduction_rate", + "uc_deduction_combination" + ], + "nonnegative_outputs": [ + "uc_deduction_random_draw", + "uc_deduction_type_random_draw", + "uc_latent_deduction_rate" + ], + "notes": "Assigns region-conditioned latent deduction demand to every benefit unit from identity-keyed seed-0 draws, rounded and clamped to the PolicyEngine-UK float32 input contract before applying the engine's exact band and normalized-combination mappings. The attributes are award-independent; law and claimant gating remain in the engine. Clone and donor-stack stages copy the completed attributes byte-for-byte. The persisted draws are microcosm's identity-keyed blake2b uniforms, not PolicyEngine-UK's splitmix64 fallback hash; the engine reads the persisted columns and derives the same flag, rate and combination from them, so the two draw columns must stay on the export surface (allow-listed and test-pinned)." + }, { "stage": "cgt_incidence_clone", "survey": "Family Resources Survey 2024-25, SPI synthetic support, and Advani-Summers capital-gains incidence", diff --git a/packages/microcosm-build/src/microcosm/build/uk/spec/sources.yaml b/packages/microcosm-build/src/microcosm/build/uk/spec/sources.yaml index ca30dbece..4a3394974 100644 --- a/packages/microcosm-build/src/microcosm/build/uk/spec/sources.yaml +++ b/packages/microcosm-build/src/microcosm/build/uk/spec/sources.yaml @@ -718,6 +718,9 @@ stages: property_wealth: DVPropertyR8 gross_financial_wealth: HFINWR8_SUM net_financial_wealth: HFINWNTR8_Sum + net_financial_wealth_exsl: HFINWNTR8_exSLC_Sum + mortgage_debt: HMortGR8 + tenure_code: Ten1R8 main_residence_value: DVhvalueR8 other_residential_property_value: DVHseValR8_sum non_residential_property_value: DVBlDValR8_sum @@ -730,6 +733,8 @@ stages: stocks_and_shares_isa: DVIISAVR8_aggr cash_isa: DVCISAVR8_aggr student_loan_balance: Tot_LosR8_aggr - Tot_los_exc_SLCR8_aggr + consumer_debt: max(HFINWR8_SUM - HFINWNTR8_exSLC_Sum, 0) + has_mortgage_tenure: Ten1R8 in {2, 3} is_renting: DVPriRntR8 == 1 region: GORR8 via incumbent REGIONS map - kind: materialize_rules_engine_predictors @@ -766,6 +771,8 @@ stages: NORTHERN_IRELAND: WALES integer_outputs: - num_vehicles + debt_segment_predictors: + - has_mortgage_tenure chain_order: - owned_land - property_wealth @@ -782,6 +789,8 @@ stages: - num_vehicles - student_loan_balance - cash_isa + - mortgage_debt + - consumer_debt - kind: fold_into output: corporate_wealth inputs: @@ -817,6 +826,8 @@ stages: - cash_isa - stocks_and_shares_isa - student_loan_balance + - mortgage_debt + - consumer_debt nonnegative_outputs: - owned_land - property_wealth @@ -831,7 +842,9 @@ stages: - cash_isa - stocks_and_shares_isa - student_loan_balance - notes: 'Ports incumbent WAS round-8 wealth imputation with signed E5 differences: exact lower-case column matching replaces the fuzzy r/w fallback; cash ISA uses DVCISAVR8_aggr and stocks-and-shares ISA uses DVIISAVR8_aggr; private pension wealth (WAS total private pension wealth less the current defined-benefit component; DB-type components at the SCAPE rate, money-purchase components at reported fund values) is drawn as its own output private_pension_wealth and no longer folded into corporate_wealth, which keeps only the share-like holdings - pension rights are disregarded capital in every means test (UC Regs 2013 Sch 10 para 10 and the HB/JSA/ESA/IS/SPC parallels; uk-data#452) - and the downstream chain segment conditions on both components; each chain segment starts its own fit and draw RNG streams from a child seed derived deterministically from the declared seed (start_chain respawns from the model seed on every call, so one shared model coupled the k-th target of every segment); corporate_wealth folds stocks-and-shares ISA after drawing corporate_wealth_excl_isa; recipient Northern Ireland regions are mapped to Wales for prediction only; student_loan_balance is allocated by household id rather than the incumbent positional off-by-one. Engine predictors materialize at their native entity and person/benunit values are summed to household, reproducing the incumbent map_to=household semantics; region is one-hot encoded jointly across donor and recipient (the incumbent''s dummy encoding), with unmapped donor GOR codes becoming all-zero dummy rows. The WAS and FRS predictor definitions are not fully like-for-like and are ported as-is; raw WAS missing values are blanket-filled with zero; UKDS negative sentinel codes (-9/-8/-7/-6) are recoded to zero + - mortgage_debt + - consumer_debt + notes: 'Ports incumbent WAS round-8 wealth imputation with signed E5 differences: exact lower-case column matching replaces the fuzzy r/w fallback; cash ISA uses DVCISAVR8_aggr and stocks-and-shares ISA uses DVIISAVR8_aggr; private pension wealth (WAS total private pension wealth less the current defined-benefit component; DB-type components at the SCAPE rate, money-purchase components at reported fund values) is drawn as its own output private_pension_wealth and no longer folded into corporate_wealth, which keeps only the share-like holdings - pension rights are disregarded capital in every means test (UC Regs 2013 Sch 10 para 10 and the HB/JSA/ESA/IS/SPC parallels; uk-data#452) - and the downstream chain segment conditions on both components; each chain segment starts its own fit and draw RNG streams from a child seed derived deterministically from the declared seed (start_chain respawns from the model seed on every call, so one shared model coupled the k-th target of every segment); corporate_wealth folds stocks-and-shares ISA after drawing corporate_wealth_excl_isa; recipient Northern Ireland regions are mapped to Wales for prediction only; student_loan_balance is allocated by household id rather than the incumbent positional off-by-one. The debt segment (fourth chain segment, fourth child seed) reads WAS round-8 derived variables HMortGR8 (total outstanding mortgage debt), HFINWNTR8_exSLC_Sum (net financial wealth excluding student loans; gross less it is the non-student-loan financial liabilities) and Ten1R8 (tenure; codes 2 and 3 are buying with a mortgage and part own/part rent, confirmed by the I1 audit of the pinned tab); UKDS SN 7215 documentation is the codebook. Engine predictors materialize at their native entity and person/benunit values are summed to household, reproducing the incumbent map_to=household semantics; region is one-hot encoded jointly across donor and recipient (the incumbent''s dummy encoding), with unmapped donor GOR codes becoming all-zero dummy rows. The WAS and FRS predictor definitions are not fully like-for-like and are ported as-is; raw WAS missing values are blanket-filled with zero; UKDS negative sentinel codes (-9/-8/-7/-6) are recoded to zero for the nonnegative-domain columns the licensed audit found carrying them (vcarnr8: 2 rows; HBedRmR8: 95.8 percent - the bedrooms question is effectively unasked in the WAS household file, predictor-quality revisit registered on microcosm#145) - a signed difference vs the incumbent, which trains on raw sentinels; DVPriRntR8''s -9 is structural not-applicable so the is_renting mapping is unchanged; genuinely negative domains are never recoded.' - stage: regional_property_uprating survey: Public regional property reference @@ -2013,6 +2026,53 @@ stages: - frs_benunit_capital - would_claim_uc notes: Runs after the last universal_credit_reported writer and before CGT cloning. Redraws only SPI post-fill reporters from weighted base-FRS reporter capital within dependent-children band by couple-status cells, then applies the monotone reported-UC OR refresh. +- stage: uc_deduction_attributes + survey: DWP Universal Credit deductions statistics, March 2025 to February 2026 + source: Department for Work and Pensions supplementary Tables 2, 5 and 6, sha-pinned by Chronicle package dwp-uc-deductions-march-2025-february-2026. + grain: benunit + artifacts: + - role: deduction_distribution + resource: dwp_uc_deduction_distributions.json + kind: public_aggregate_reference + format: json + operations: + - kind: assign_uniform_draw + output: uc_deduction_random_draw + seed: 0 + - kind: assign_uniform_draw + output: uc_deduction_type_random_draw + seed: 0 + - kind: map_uniform_to_banded_rate + output: uc_latent_deduction_rate + draw: uc_deduction_random_draw + draw_dtype: float32 + resource: dwp_uc_deduction_distributions.json + distribution: latent_rate_distribution + incidence_modifier: + column: region + entity: household + table: region_incidence_factor + none_value: 0.0 + - kind: map_uniform_to_categorical + output: uc_deduction_combination + draw: uc_deduction_type_random_draw + draw_dtype: float32 + resource: dwp_uc_deduction_distributions.json + distribution: type_combination + gate: + column: uc_latent_deduction_rate + positive: true + none_value: NONE + outputs: + - uc_deduction_random_draw + - uc_deduction_type_random_draw + - uc_latent_deduction_rate + - uc_deduction_combination + nonnegative_outputs: + - uc_deduction_random_draw + - uc_deduction_type_random_draw + - uc_latent_deduction_rate + notes: Assigns region-conditioned latent deduction demand to every benefit unit from identity-keyed seed-0 draws, rounded and clamped to the PolicyEngine-UK float32 input contract before applying the engine's exact band and normalized-combination mappings. The attributes are award-independent; law and claimant gating remain in the engine. Clone and donor-stack stages copy the completed attributes byte-for-byte. The persisted draws are microcosm's identity-keyed blake2b uniforms, not PolicyEngine-UK's splitmix64 fallback hash; the engine reads the persisted columns and derives the same flag, rate and combination from them, so the two draw columns must stay on the export surface (allow-listed and test-pinned). - stage: cgt_incidence_clone survey: Family Resources Survey 2024-25, SPI synthetic support, and Advani-Summers capital-gains incidence source: Advani and Summers (2020), Capital Gains and UK Inequality, CAGE Working Paper 465 diff --git a/packages/microcosm-build/src/microcosm/build/uk/spine_swap_signed_differences.json b/packages/microcosm-build/src/microcosm/build/uk/spine_swap_signed_differences.json index 176e4b5b0..16c3365f4 100644 --- a/packages/microcosm-build/src/microcosm/build/uk/spine_swap_signed_differences.json +++ b/packages/microcosm-build/src/microcosm/build/uk/spine_swap_signed_differences.json @@ -1,6 +1,6 @@ { "schema_version": 1, - "scope_note": "Adjudicated intentional differences between the microcosm-built UK spine and the frozen enhanced-FRS incumbent (#686). The whole-spine comparison treats any difference beyond the #723 acceptance band that is not signed here as a defect, so every entry is scoped to the exact surface and columns where the difference is expected to appear: a too-broad entry would sign a real defect. Entries are permanent adjudications and carry no expiry; time-limited per-gate suppressions belong in input_mass_reviewed_exclusions.json, qrf_tail_reviewed_exclusions.json or degenerate_reviewed_exclusions.json instead, and an entry here points at one through its evidence field when both descend from the same adjudication. The twenty-nine beyond-band divergences measured or structurally expected against the re-pinned 1.56.16 reference are signed below, each scoped to the divergence actually observed rather than carried across on its prose classification, and each deliberately split so that no entry covers both columns where the spine is closer to its donor and columns where the incumbent is: the direction of the evidence is part of what is being signed. Donor figures quoted as magnitude evidence are survey-weighted shares and population means computed on each stage's own committed cleaning function over its own pinned donor tab, which is the convention that reproduces the E6 acceptance receipt's education figure exactly. The incumbent side of every figure is measured on the pinned 1.56.16 artifact itself (sha256 e433e532), not on a local rebuild: an earlier pass read the 1.56.14 published artifact, whose shares differ by up to 0.0045 and which flipped one verdict.", + "scope_note": "Adjudicated intentional differences between the microcosm-built UK spine and the frozen enhanced-FRS incumbent (#686). The whole-spine comparison treats any difference beyond the #723 acceptance band that is not signed here as a defect, so every entry is scoped to the exact surface and columns where the difference is expected to appear: a too-broad entry would sign a real defect. Entries are permanent adjudications and carry no expiry; time-limited per-gate suppressions belong in input_mass_reviewed_exclusions.json, qrf_tail_reviewed_exclusions.json or degenerate_reviewed_exclusions.json instead, and an entry here points at one through its evidence field when both descend from the same adjudication. The thirty-five beyond-band divergences measured or structurally expected against the re-pinned 1.56.16 reference are signed below, each scoped to the divergence actually observed rather than carried across on its prose classification, and each deliberately split so that no entry covers both columns where the spine is closer to its donor and columns where the incumbent is: the direction of the evidence is part of what is being signed. Donor figures quoted as magnitude evidence are survey-weighted shares and population means computed on each stage's own committed cleaning function over its own pinned donor tab, which is the convention that reproduces the E6 acceptance receipt's education figure exactly. The incumbent side of every figure is measured on the pinned 1.56.16 artifact itself (sha256 e433e532), not on a local rebuild: an earlier pass read the 1.56.14 published artifact, whose shares differ by up to 0.0045 and which flipped one verdict.", "differences": [ { "id": "scottish-water-incumbent-nan-zeroing", @@ -536,6 +536,156 @@ }, "magnitude_provenance": "I1 dry-run bound from experiments/832-uc-reporter-receipts.md Part C: 4,428 unchanged base-channel reporter persons plus between zero and 3,299 one-person SPI reporter landings among 113,649 people implies share in [0.038962, 0.067990]. I5 actual (Part F, spine-m): 5,730 nonzero reporter records, share 0.050418, delta 0.006941 below the incumbent, inside the signed 0.0184 floor-distance; the above side of the structural range is unreached by the built artifact." } + }, + { + "id": "mortgage-debt-net-new-column", + "class": "net_new_column", + "scope": { + "surface": "nonzero_shares", + "columns": [ + "mortgage_debt" + ], + "entities": [ + "household" + ] + }, + "expectation": "column_missing_in_reference", + "magnitude_evidence": "The fourth WAS wealth-chain segment now persists mortgage_debt from the HMortGR8 donor target. The pinned enhanced-FRS reference has no column with that name, so this is a structural candidate-only export. The I1 audit of the pinned round-8 donor measured a survey-weighted positive share of 0.3226 and a population mean of GBP 46,915 per household.", + "evidence": "experiments/685-net-new-stages-receipts.md#part-a--was-debt-source-audit", + "adjudicator": "juaristi22", + "adjudicated_on": "2026-09-02", + "quantitative": { + "structural": { + "expected_columns": [ + "mortgage_debt" + ] + } + } + }, + { + "id": "consumer-debt-net-new-column", + "class": "net_new_column", + "scope": { + "surface": "nonzero_shares", + "columns": [ + "consumer_debt" + ], + "entities": [ + "household" + ] + }, + "expectation": "column_missing_in_reference", + "magnitude_evidence": "The fourth WAS wealth-chain segment now persists consumer_debt, defined as gross financial wealth less financial wealth net of all liabilities except student loans, clipped at zero. The pinned enhanced-FRS reference has no column with that name, so this is a structural candidate-only export. The I1 audit of the pinned round-8 donor measured a survey-weighted positive share of 0.5296 and a population mean of GBP 3,623 per household.", + "evidence": "experiments/685-net-new-stages-receipts.md#part-a--was-debt-source-audit", + "adjudicator": "juaristi22", + "adjudicated_on": "2026-09-02", + "quantitative": { + "structural": { + "expected_columns": [ + "consumer_debt" + ] + } + } + }, + { + "id": "uc-deduction-random-draw-net-new-column", + "class": "net_new_column", + "scope": { + "surface": "nonzero_shares", + "columns": [ + "uc_deduction_random_draw" + ], + "entities": [ + "benunit" + ] + }, + "expectation": "column_missing_in_reference", + "magnitude_evidence": "The UC deduction-attribute stage persists its identity-keyed float32-rounded incidence draw so PolicyEngine-UK 2.92.1 derives the same diagnostic flag as the held latent attributes. The pinned enhanced-FRS reference has no column with that name. Realized licensed-spine figures are pending — I5 licensed acceptance.", + "evidence": "experiments/685-net-new-stages-receipts.md#part-b--uc-deduction-attribute-contract", + "adjudicator": "juaristi22", + "adjudicated_on": "2026-09-02", + "quantitative": { + "structural": { + "expected_columns": [ + "uc_deduction_random_draw" + ] + } + } + }, + { + "id": "uc-deduction-type-random-draw-net-new-column", + "class": "net_new_column", + "scope": { + "surface": "nonzero_shares", + "columns": [ + "uc_deduction_type_random_draw" + ], + "entities": [ + "benunit" + ] + }, + "expectation": "column_missing_in_reference", + "magnitude_evidence": "The UC deduction-attribute stage persists its independent identity-keyed float32-rounded type draw so PolicyEngine-UK 2.92.1 derives the same combination as the build. The pinned enhanced-FRS reference has no column with that name. Realized licensed-spine figures are pending — I5 licensed acceptance.", + "evidence": "experiments/685-net-new-stages-receipts.md#part-b--uc-deduction-attribute-contract", + "adjudicator": "juaristi22", + "adjudicated_on": "2026-09-02", + "quantitative": { + "structural": { + "expected_columns": [ + "uc_deduction_type_random_draw" + ] + } + } + }, + { + "id": "uc-latent-deduction-rate-net-new-column", + "class": "net_new_column", + "scope": { + "surface": "nonzero_shares", + "columns": [ + "uc_latent_deduction_rate" + ], + "entities": [ + "benunit" + ] + }, + "expectation": "column_missing_in_reference", + "magnitude_evidence": "The UC deduction-attribute stage persists region-conditioned, pre-cap latent deduction demand on every benefit unit from the pinned DWP distribution. The pinned enhanced-FRS reference has no column with that name; its nonzero share is a latent-attribute statistic, not a claimant-incidence statistic. Realized licensed-spine figures are pending — I5 licensed acceptance.", + "evidence": "experiments/685-net-new-stages-receipts.md#part-b--uc-deduction-attribute-contract", + "adjudicator": "juaristi22", + "adjudicated_on": "2026-09-02", + "quantitative": { + "structural": { + "expected_columns": [ + "uc_latent_deduction_rate" + ] + } + } + }, + { + "id": "uc-deduction-combination-net-new-column", + "class": "net_new_column", + "scope": { + "surface": "nonzero_shares", + "columns": [ + "uc_deduction_combination" + ], + "entities": [ + "benunit" + ] + }, + "expectation": "column_missing_in_reference", + "magnitude_evidence": "The UC deduction-attribute stage persists the PolicyEngine-UK deduction-combination enum member name, with NONE exactly where latent demand is zero. The pinned enhanced-FRS reference has no column with that name. Realized licensed-spine figures are pending — I5 licensed acceptance.", + "evidence": "experiments/685-net-new-stages-receipts.md#part-b--uc-deduction-attribute-contract", + "adjudicator": "juaristi22", + "adjudicated_on": "2026-09-02", + "quantitative": { + "structural": { + "expected_columns": [ + "uc_deduction_combination" + ] + } + } } ] } diff --git a/packages/microcosm-build/src/microcosm/build/uk/uc_deduction_support_bounds.json b/packages/microcosm-build/src/microcosm/build/uk/uc_deduction_support_bounds.json new file mode 100644 index 000000000..7a1616e30 --- /dev/null +++ b/packages/microcosm-build/src/microcosm/build/uk/uc_deduction_support_bounds.json @@ -0,0 +1,10 @@ +{ + "version": 1, + "country": "uk", + "policy": "Declared mathematical support for the persisted Universal Credit deduction attributes; these bounds are not donor-derived.", + "bounds": { + "uc_deduction_random_draw": [0.0, 1.0], + "uc_deduction_type_random_draw": [0.0, 1.0], + "uc_latent_deduction_rate": [0.0, 0.3] + } +} diff --git a/packages/microcosm-build/src/microcosm/build/uk/was_wealth_support_bounds.json b/packages/microcosm-build/src/microcosm/build/uk/was_wealth_support_bounds.json index 1cc3475e7..2b9bc0f9b 100644 --- a/packages/microcosm-build/src/microcosm/build/uk/was_wealth_support_bounds.json +++ b/packages/microcosm-build/src/microcosm/build/uk/was_wealth_support_bounds.json @@ -65,6 +65,14 @@ "student_loan_balance": [ 0.0, 200000 + ], + "mortgage_debt": [ + 0.0, + 4000000 + ], + "consumer_debt": [ + 0.0, + 400000 ] }, "chronicle": [ diff --git a/packages/microcosm-build/src/microcosm/build/uk_runtime/battery_bindings.py b/packages/microcosm-build/src/microcosm/build/uk_runtime/battery_bindings.py index 7cb2002a3..e528b6ba3 100644 --- a/packages/microcosm-build/src/microcosm/build/uk_runtime/battery_bindings.py +++ b/packages/microcosm-build/src/microcosm/build/uk_runtime/battery_bindings.py @@ -506,6 +506,7 @@ def _evaluate_support( "lcfs_consumption_support_bounds.json", "etb_vat_support_bounds.json", "etb_services_support_bounds.json", + "uc_deduction_support_bounds.json", } if set(resource_names) - allowed: raise ValueError( diff --git a/packages/microcosm-build/src/microcosm/build/uk_runtime/calibration_run.py b/packages/microcosm-build/src/microcosm/build/uk_runtime/calibration_run.py index 6864a9eff..522383c77 100644 --- a/packages/microcosm-build/src/microcosm/build/uk_runtime/calibration_run.py +++ b/packages/microcosm-build/src/microcosm/build/uk_runtime/calibration_run.py @@ -107,6 +107,7 @@ class UKCalibrationRunResult: UK_SPINE_GATE_SCOPE = ( "uk_stage_was_wealth_support", + "uk_stage_uc_deduction_attributes", "uk_stage_lcfs_consumption_support", "uk_stage_etb_vat_support", "uk_stage_etb_services_support", @@ -139,6 +140,7 @@ class UKCalibrationRunResult: "uk_degenerate_release_surface", "uk_nonnegative_columns", "uk_uc_capital_coherence", + "uk_uc_deduction_combination_enum_domain", "uk_support", "uk_aggregate_admin", "uk_export_surface", diff --git a/packages/microcosm-build/src/microcosm/build/uk_runtime/graph.py b/packages/microcosm-build/src/microcosm/build/uk_runtime/graph.py index ccc530ddb..dfc2156ca 100644 --- a/packages/microcosm-build/src/microcosm/build/uk_runtime/graph.py +++ b/packages/microcosm-build/src/microcosm/build/uk_runtime/graph.py @@ -194,6 +194,7 @@ ("benunit", "would_claim_uc"), } ), + "uc_deduction_attributes": frozenset({("household", "region")}), "cgt_incidence_clone": None, "cgt_band_donors": None, "hmrc_cgt_gains_spine": frozenset( @@ -445,6 +446,7 @@ def _cells( ), _Cell("household", "num_vehicles", "int64"), *_cells("household", ("cash_isa", "stocks_and_shares_isa")), + *_cells("household", ("mortgage_debt", "consumer_debt")), _Cell("person", "student_loan_balance", "float64"), ), "regional_property_uprating": _cells( @@ -532,6 +534,12 @@ def _cells( _Cell("benunit", "frs_benunit_capital", "float64"), _Cell("benunit", "would_claim_uc", "bool"), ), + "uc_deduction_attributes": ( + _Cell("benunit", "uc_deduction_random_draw", "float64"), + _Cell("benunit", "uc_deduction_type_random_draw", "float64"), + _Cell("benunit", "uc_latent_deduction_rate", "float64"), + _Cell("benunit", "uc_deduction_combination", "string"), + ), "cgt_incidence_clone": ( _Cell("household", "household_is_capital_gains_clone", "bool"), _Cell("person", "capital_gains", "float64"), diff --git a/packages/microcosm-build/src/microcosm/build/uk_runtime/graph_kernels.py b/packages/microcosm-build/src/microcosm/build/uk_runtime/graph_kernels.py index 9321a3c81..540b46544 100644 --- a/packages/microcosm-build/src/microcosm/build/uk_runtime/graph_kernels.py +++ b/packages/microcosm-build/src/microcosm/build/uk_runtime/graph_kernels.py @@ -82,6 +82,7 @@ "hmrc_spi_income_spine": "spi_spine", "uc_reporter_redraw": "uc_reporter_redraw", "uc_capital_coherence": "uc_capital_coherence", + "uc_deduction_attributes": "uc_deduction_attributes", "cgt_incidence_clone": "cgt_structure", "cgt_band_donors": "cgt_structure", "hmrc_cgt_gains_spine": "cgt_imputation", @@ -275,7 +276,7 @@ def records(name: str) -> list[Mapping[str, object]]: def _fixture_descriptor( source: Path, ) -> tuple[Mapping[str, object], dict[str, SourceStageSpec]]: - """Parse the H2 bundle's descriptor and its 26 stage specs, keyed by name.""" + """Parse the H2 bundle's descriptor and its 27 stage specs, keyed by name.""" from microcosm.build.source_manifest import SourceStageSpec @@ -299,7 +300,7 @@ def _fixture_descriptor( missing = sorted(set(_STAGE_MODULES) - set(stages)) extra = sorted(set(stages) - set(_STAGE_MODULES)) raise ValueError( - "UK parity fixture must describe the current 26-stage spine " + "UK parity fixture must describe the current 28-stage spine " f"(missing={missing}, extra={extra})." ) return descriptor, stages @@ -339,6 +340,7 @@ def _fixture_implementations(source: Path) -> Mapping[str, object]: from .student_loans import UKStudentLoansStageTransform from .take_up_contract import load_uk_take_up_contract from .uc_capital_coherence import UKUCCapitalCoherenceStageTransform + from .uc_deduction_attributes import UKUCDeductionAttributesStageTransform from .uc_reporter_redraw import UKUCReporterRedrawStageTransform from .was_wealth import UKWASWealthStageTransform @@ -454,6 +456,9 @@ def _fixture_implementations(source: Path) -> Mapping[str, object]: "uc_capital_coherence": UKUCCapitalCoherenceStageTransform( stage=stages["uc_capital_coherence"] ), + "uc_deduction_attributes": UKUCDeductionAttributesStageTransform( + stage=stages["uc_deduction_attributes"] + ), "cgt_incidence_clone": UKCGTIncidenceCloneStageTransform( stage=stages["cgt_incidence_clone"] ), diff --git a/packages/microcosm-build/src/microcosm/build/uk_runtime/source_runtime.py b/packages/microcosm-build/src/microcosm/build/uk_runtime/source_runtime.py index a86e89fa3..92f097bba 100644 --- a/packages/microcosm-build/src/microcosm/build/uk_runtime/source_runtime.py +++ b/packages/microcosm-build/src/microcosm/build/uk_runtime/source_runtime.py @@ -69,6 +69,7 @@ def uk_stage_implementations( frs_hmrc_spine_leaves_transform: Callable[[Frame], Frame] | None = None, spi_support_channel_transform: Callable[[Frame], Frame] | None = None, hmrc_spi_income_spine_transform: Callable[[Frame], Frame] | None = None, + uc_deduction_attributes_transform: Callable[[Frame], Frame] | None = None, cgt_incidence_clone_transform: Callable[[Frame], Frame] | None = None, cgt_band_donors_transform: Callable[[Frame], Frame] | None = None, hmrc_cgt_gains_spine_transform: Callable[[Frame], Frame] | None = None, @@ -101,6 +102,7 @@ def uk_stage_implementations( "frs_hmrc_spine_leaves": frs_hmrc_spine_leaves_transform, "spi_support_channel": spi_support_channel_transform, "hmrc_spi_income_spine": hmrc_spi_income_spine_transform, + "uc_deduction_attributes": uc_deduction_attributes_transform, "cgt_incidence_clone": cgt_incidence_clone_transform, "cgt_band_donors": cgt_band_donors_transform, "hmrc_cgt_gains_spine": hmrc_cgt_gains_spine_transform, diff --git a/packages/microcosm-build/src/microcosm/build/uk_runtime/stage_health.py b/packages/microcosm-build/src/microcosm/build/uk_runtime/stage_health.py index a22e4a38b..3a91e0f74 100644 --- a/packages/microcosm-build/src/microcosm/build/uk_runtime/stage_health.py +++ b/packages/microcosm-build/src/microcosm/build/uk_runtime/stage_health.py @@ -3,6 +3,7 @@ from __future__ import annotations import json +import math from collections.abc import Mapping from importlib.resources import files @@ -51,6 +52,8 @@ def uk_stage_health_gate( return _cgt_band_donor_support_gate(stage, evidence, parameters) if check == "cgt_imputation_summary": return _cgt_imputation_summary_gate(stage, evidence, parameters) + if check == "latent_attribute_realization": + return _latent_attribute_realization_gate(stage, evidence) return GateResult( name="stage_health", passed=False, @@ -429,3 +432,110 @@ def _cgt_imputation_summary_gate( failures.append(f"{stage}: {key} is negative.") details = {"band_rows": len(rows), "taxpayer_mass": evidence.get("taxpayer_mass")} return _fail(stage, check, failures, details) if failures else _pass(stage, check, details) + + +def _latent_attribute_realization_gate( + stage: str, + evidence: Mapping[str, object], +) -> GateResult: + """Check a latent-attribute stage's realization receipt. + + The receipt carries, per cell, the declared ``target`` share, the + **unweighted** ``realized`` share over the ``rows`` behind it, and the + producer's ``tolerance``. Identity-keyed draws give every unit the same + probability regardless of its weight, so the unweighted share is the + statistic that tests the mechanism; the weighted share (``realized_weighted``, + informational) carries the frame's weight variance on top and is what the + engine round-trip compares with the publisher. The gate owns the pass rule: + it recomputes the binomial band from ``target`` and ``rows`` at + :data:`LATENT_ATTRIBUTE_SIGMA` (floored at one row's worth, capped at one) + and holds the cell to the tighter of the producer's figure and its own, so + a widened producer tolerance cannot pass silently. + """ + + check = "latent_attribute_realization" + failures: list[str] = [] + raw_coherence = evidence.get("coherence_violation_count") + if not isinstance(raw_coherence, int) or isinstance(raw_coherence, bool): + failures.append(f"{stage}: coherence_violation_count is missing.") + coherence = None + else: + coherence = int(raw_coherence) + if coherence != 0: + failures.append( + f"{stage}: coherence_violation_count is {coherence}, expected 0." + ) + cells_checked = 0 + worst_ratio = 0.0 + for block_name in ( + "incidence_by_region", + "latent_rate_bands", + "combination_shares", + ): + block = _mapping(evidence.get(block_name), label=f"{stage}.{block_name}") + if not block: + failures.append(f"{stage}: {block_name} is empty.") + for name, raw_row in block.items(): + if not isinstance(raw_row, Mapping): + failures.append(f"{stage}: {block_name}.{name} is not an object.") + continue + label = f"{stage}.{block_name}.{name}" + target = _finite_number(raw_row.get("target"), label=f"{label}.target") + realized = _finite_number( + raw_row.get("realized"), label=f"{label}.realized" + ) + declared_tolerance = _finite_number( + raw_row.get("tolerance"), label=f"{label}.tolerance" + ) + rows = int(raw_row.get("rows", 0)) + cells_checked += 1 + if not 0.0 <= target <= 1.0 or not 0.0 <= realized <= 1.0: + failures.append(f"{label}: target and realized must be shares.") + if rows <= 0 or not 0.0 <= declared_tolerance <= 1.0: + failures.append(f"{label}: has invalid rows/tolerance.") + continue + tolerance = min( + declared_tolerance, _binomial_tolerance(target=target, rows=rows) + ) + deviation = abs(realized - target) + ratio = deviation / tolerance if tolerance > 0.0 else float("inf") + worst_ratio = max(worst_ratio, ratio) + if deviation > tolerance: + failures.append( + f"{label}: deviation {deviation} exceeds {tolerance} " + f"(declared {declared_tolerance})." + ) + details = { + "cells_checked": cells_checked, + "coherence_violation_count": coherence, + "worst_tolerance_ratio": worst_ratio, + } + return ( + _fail(stage, check, failures, details) + if failures + else _pass(stage, check, details) + ) + + +#: Sigma multiple for the per-cell binomial band. A latent-attribute receipt +#: holds ~30 cells (regions, bands, combinations) per build; three sigma would +#: false-alarm on roughly one build in twelve, four sigma on one in five hundred. +LATENT_ATTRIBUTE_SIGMA = 4.0 + + +def latent_attribute_tolerance(*, target: float, rows: int) -> float: + """Binomial band for a share realized over ``rows`` identity-keyed draws.""" + + if rows <= 0: + return 1.0 + return min( + 1.0, + max( + 1.0 / rows, + LATENT_ATTRIBUTE_SIGMA * math.sqrt(target * (1.0 - target) / rows), + ), + ) + + +def _binomial_tolerance(*, target: float, rows: int) -> float: + return latent_attribute_tolerance(target=target, rows=rows) diff --git a/packages/microcosm-build/src/microcosm/build/uk_runtime/terminal_gates.py b/packages/microcosm-build/src/microcosm/build/uk_runtime/terminal_gates.py index d835d5cf5..cb086f903 100644 --- a/packages/microcosm-build/src/microcosm/build/uk_runtime/terminal_gates.py +++ b/packages/microcosm-build/src/microcosm/build/uk_runtime/terminal_gates.py @@ -161,6 +161,10 @@ def __post_init__(self) -> None: UK_ALLOWED_EXTRA_EXPORT_COLUMNS: tuple[str, ...] = ( "benunit.child_benefit_opts_out", "benunit.frs_benunit_capital", + "benunit.uc_deduction_combination", + "benunit.uc_deduction_random_draw", + "benunit.uc_deduction_type_random_draw", + "benunit.uc_latent_deduction_rate", "benunit.uc_reported_capital", "household.bus_fare_spending", "household.bus_subsidy_spending", diff --git a/packages/microcosm-build/src/microcosm/build/uk_runtime/uc_deduction_attributes.py b/packages/microcosm-build/src/microcosm/build/uk_runtime/uc_deduction_attributes.py new file mode 100644 index 000000000..8e3d7007f --- /dev/null +++ b/packages/microcosm-build/src/microcosm/build/uk_runtime/uc_deduction_attributes.py @@ -0,0 +1,617 @@ +"""Late UK Universal Credit deduction-attribute assignment stage. + +The two persisted draw columns are microcosm's identity-keyed uniforms (the +E4 primitive: blake2b over ``seed:salt:benunit_id``), not PolicyEngine-UK's +own fallback hash (``splitmix64_uniform(benunit_id, salt)``). The engine reads +the persisted columns when a dataset carries them and only hashes when it +does not, so the two never need to agree; what the design does rest on is the +draw columns staying on the export surface, which the allow-list and +``test_uk_uc_deduction_attributes.py`` pin. +""" + +from __future__ import annotations + +import json +from collections.abc import Mapping, Sequence +from dataclasses import dataclass, field +from importlib.resources import files +from typing import Any + +import numpy as np +import pandas as pd + +from microcosm.build.source_manifest import SourceStageSpec +from microcosm.build.stochastic_assignment import stable_identity_uniforms +from microcosm.build.uk_runtime.cgt_structure import ( + _assert_closed_world_operations, +) +from microcosm.build.uk_runtime.frs_brma import _benunit_household_map +from microcosm.build.uk_runtime.national_frame import ( + uk_household_weight_kind, + uk_national_frame, + uk_time_period, + validate_uk_national_frame, +) +from microcosm.build.uk_runtime.stage_health import latent_attribute_tolerance +from microcosm.frame import Frame + +UC_DEDUCTION_STAGE_NAME = "uc_deduction_attributes" +UC_DEDUCTION_RESOURCE = "dwp_uc_deduction_distributions.json" +UC_DEDUCTION_OUTPUT_COLUMNS = ( + "uc_deduction_random_draw", + "uc_deduction_type_random_draw", + "uc_latent_deduction_rate", + "uc_deduction_combination", +) +UC_DEDUCTION_NONNEGATIVE_OUTPUT_COLUMNS = UC_DEDUCTION_OUTPUT_COLUMNS[:3] +UK_UC_DEDUCTION_ATTRIBUTES_DECLARED_SEEDS = { + "uc_deduction_random_draw": 0, + "uc_deduction_type_random_draw": 0, +} +UC_DEDUCTION_COMBINATIONS = ( + "ADVANCE_ONLY", + "THIRD_PARTY_ONLY", + "GOVERNMENT_ONLY", + "ADVANCE_AND_GOVERNMENT", + "ADVANCE_AND_THIRD_PARTY", + "THIRD_PARTY_AND_GOVERNMENT", + "ALL_THREE", +) +UC_DEDUCTION_BANDS = ( + "UNDER_5", + "AT_5", + "FIVE_TO_10", + "AT_10", + "TEN_TO_15", + "AT_15", + "FIFTEEN_TO_20", + "AT_20", + "TWENTY_TO_25", + "AT_25", + "OVER_25", +) +UC_DEDUCTION_REGIONS = frozenset( + { + "UNKNOWN", + "NORTH_EAST", + "NORTH_WEST", + "YORKSHIRE", + "EAST_MIDLANDS", + "WEST_MIDLANDS", + "EAST_OF_ENGLAND", + "LONDON", + "SOUTH_EAST", + "SOUTH_WEST", + "WALES", + "SCOTLAND", + "NORTHERN_IRELAND", + } +) +FLOAT32_UNIFORM_MAX = float(1.0 - 2.0**-24) + + +def load_uc_deduction_distributions() -> Mapping[str, Any]: + """Load the pinned DWP UC deduction estimation resource.""" + + return json.loads( + files("microcosm.build.uk") + .joinpath(UC_DEDUCTION_RESOURCE) + .read_text(encoding="utf-8") + ) + + +@dataclass(frozen=True) +class UKUCDeductionAttributesResult: + """Transformed frame and executed-effect realization receipt.""" + + frame: Frame + incidence_by_region: Mapping[str, Mapping[str, float | int]] + latent_rate_bands: Mapping[str, Mapping[str, float | int]] + combination_shares: Mapping[str, Mapping[str, float | int]] + coherence_violation_count: int + + def evidence(self) -> dict[str, object]: + """Return the JSON-safe stage receipt consumed by stage health.""" + + return { + "stage": UC_DEDUCTION_STAGE_NAME, + "declared_seeds": dict(UK_UC_DEDUCTION_ATTRIBUTES_DECLARED_SEEDS), + "draw_dtype": "float32", + "float32_uniform_max": FLOAT32_UNIFORM_MAX, + "incidence_by_region": dict(self.incidence_by_region), + "latent_rate_bands": dict(self.latent_rate_bands), + "combination_shares": dict(self.combination_shares), + "coherence_violation_count": self.coherence_violation_count, + } + + +@dataclass(frozen=True) +class UKUCDeductionAttributesStageTransform: + """Assign held UC deduction draws, latent rates, and combinations.""" + + stage: SourceStageSpec + resource: Mapping[str, Any] | None = None + last_result: UKUCDeductionAttributesResult | None = field(default=None, init=False) + + def __call__(self, frame: Frame) -> Frame: + resource = self.resource or load_uc_deduction_distributions() + _assert_stage_parameters(self.stage) + validate_uc_deduction_resource(resource) + result = assign_uc_deduction_attributes(frame, resource=resource) + object.__setattr__(self, "last_result", result) + return result.frame + + @staticmethod + def output_columns() -> tuple[str, ...]: + return UC_DEDUCTION_OUTPUT_COLUMNS + + def checkpoint_metadata(self) -> dict[str, object]: + if self.last_result is None: + raise RuntimeError("checkpoint metadata requires a completed stage run.") + return {"evidence": self.last_result.evidence()} + + +def assign_uc_deduction_attributes( + frame: Frame, + *, + resource: Mapping[str, Any], +) -> UKUCDeductionAttributesResult: + """Assign latent deduction demand to every benefit unit by region.""" + + validate_uk_national_frame(frame) + validate_uc_deduction_resource(resource) + person = frame.table("person").copy() + benunit = frame.table("benunit").copy() + household = frame.table("household").copy() + _require_columns( + person, + ("person_benunit_id", "person_household_id"), + label="person", + ) + _require_columns(benunit, ("benunit_id",), label="benunit") + _require_columns(household, ("household_id", "region"), label="household") + + household_by_benunit = benunit["benunit_id"].map(_benunit_household_map(person)) + if household_by_benunit.isna().any(): + raise ValueError( + "UC deduction attributes require every benunit in a household." + ) + region_by_household = household.set_index("household_id")["region"] + regions = household_by_benunit.map(region_by_household).map(_enum_name) + if regions.isna().any(): + raise ValueError("UC deduction attributes require a region for every benunit.") + unknown_regions = sorted(set(regions) - UC_DEDUCTION_REGIONS) + if unknown_regions: + raise ValueError( + f"UC deduction attributes found unknown region name(s): {unknown_regions}." + ) + + ids = benunit["benunit_id"].to_numpy() + # Identity-keyed blake2b uniforms, rounded to the engine's float32 input + # contract. These are not the engine's splitmix64 fallback draws: the + # engine reads these persisted columns and derives the same flag, rate and + # combination from them (module docstring). + incidence_draws = _identity_float32_uniforms( + ids, + seed=UK_UC_DEDUCTION_ATTRIBUTES_DECLARED_SEEDS["uc_deduction_random_draw"], + salt="uc_deduction_random_draw", + ) + type_draws = _identity_float32_uniforms( + ids, + seed=UK_UC_DEDUCTION_ATTRIBUTES_DECLARED_SEEDS["uc_deduction_type_random_draw"], + salt="uc_deduction_type_random_draw", + ) + rate_mapping = _banded_rate_mapping( + incidence_draws, + regions.to_numpy(dtype=object), + resource, + ) + combinations = map_uniform_to_categorical( + type_draws, + gate=rate_mapping.rates > 0.0, + resource=resource, + ) + combination_assigned = combinations != "NONE" + coherence_violations = ~((rate_mapping.rates > 0.0) == combination_assigned) | ~( + combination_assigned == rate_mapping.assigned + ) + if coherence_violations.any(): + raise ValueError( + "UC deduction attribute coherence failed: latent > 0, combination != " + "NONE, and the region-adjusted incidence draw must agree exactly; " + f"found {int(coherence_violations.sum())} violation(s)." + ) + + benunit["uc_deduction_random_draw"] = incidence_draws + benunit["uc_deduction_type_random_draw"] = type_draws + benunit["uc_latent_deduction_rate"] = rate_mapping.rates + benunit["uc_deduction_combination"] = combinations + result_frame = uk_national_frame( + person=person, + benunit=benunit, + household=household, + time_period=uk_time_period(frame), + weight_kind=uk_household_weight_kind(frame), + household_weights=frame.weights_for("household").values, + mass_log=frame.mass_log, + ) + validate_uk_national_frame(result_frame) + + weights_by_household = pd.Series( + frame.weights_for("household").values, + index=household["household_id"], + ) + benunit_weights = household_by_benunit.map(weights_by_household).to_numpy( + dtype=float + ) + return UKUCDeductionAttributesResult( + frame=result_frame, + incidence_by_region=_incidence_receipt( + regions.to_numpy(dtype=object), + assigned=rate_mapping.assigned, + adjusted_incidence=rate_mapping.adjusted_incidence, + weights=benunit_weights, + ), + latent_rate_bands=_categorical_receipt( + names=UC_DEDUCTION_BANDS, + assignments=rate_mapping.band_names, + selected=rate_mapping.assigned, + target_weights=_latent_shares(resource), + weights=benunit_weights, + ), + combination_shares=_categorical_receipt( + names=UC_DEDUCTION_COMBINATIONS, + assignments=combinations, + selected=rate_mapping.assigned, + target_weights=_combination_shares(resource), + weights=benunit_weights, + ), + coherence_violation_count=int(coherence_violations.sum()), + ) + + +@dataclass(frozen=True) +class _BandedRateMapping: + rates: np.ndarray + assigned: np.ndarray + adjusted_incidence: np.ndarray + band_names: np.ndarray + + +def map_uniform_to_banded_rate( + draws: Sequence[float] | np.ndarray, + regions: Sequence[object] | np.ndarray, + resource: Mapping[str, Any], +) -> np.ndarray: + """Mirror PolicyEngine-UK's banded latent-rate inverse CDF.""" + + return _banded_rate_mapping(draws, regions, resource).rates + + +def _banded_rate_mapping( + draws: Sequence[float] | np.ndarray, + regions: Sequence[object] | np.ndarray, + resource: Mapping[str, Any], +) -> _BandedRateMapping: + draws = np.asarray(draws, dtype=np.float64) + regions = np.asarray([_enum_name(value) for value in regions], dtype=object) + if draws.shape != regions.shape: + raise ValueError("UC deduction rate draws and regions must align.") + _validate_uniforms(draws) + bands = resource["latent_rate_distribution"]["bands"] + shares = np.asarray([float(row["share"]) for row in bands], dtype=np.float64) + lower = np.asarray([float(row["lower"]) for row in bands], dtype=np.float64) + upper = np.asarray([float(row["upper"]) for row in bands], dtype=np.float64) + # PolicyEngine-UK sums the eleven shares twice with different arithmetic: + # ``uc_has_deduction`` adds the parameters left to right in Python, while + # ``uc_latent_deduction_rate`` takes ``np.array(shares).sum()`` (pairwise). + # The two can differ in the last ulp, so each threshold is formed the way + # the engine forms it. + flag_incidence = 0.0 + for share in shares.tolist(): + flag_incidence += share + total_share = shares.sum() + factors = resource["region_incidence_factor"]["factors"] + try: + region_factors = np.asarray( + [float(factors[str(region)]) for region in regions], dtype=np.float64 + ) + except KeyError as exc: + raise ValueError( + f"UC deduction resource has no factor for {exc.args[0]!r}." + ) from exc + incidence_for_flag = np.clip(flag_incidence * region_factors, 0.0, 1.0) + adjusted_incidence = np.clip(total_share * region_factors, 1e-9, 1.0) + assigned = draws < incidence_for_flag + quantile = np.clip(draws / adjusted_incidence, 0.0, 1.0 - 1e-9) + position = quantile * total_share + cumulative = np.cumsum(shares) + band = np.searchsorted(cumulative, position, side="right") + band = np.clip(band, 0, len(shares) - 1) + cumulative_before = cumulative[band] - shares[band] + within = (position - cumulative_before) / np.maximum(shares[band], 1e-12) + rates = lower[band] + (upper[band] - lower[band]) * within + rates = np.where(assigned, rates, 0.0) + names = np.asarray([str(row["name"]) for row in bands], dtype=object)[band] + return _BandedRateMapping( + rates=np.asarray(rates, dtype=np.float64), + assigned=np.asarray(assigned, dtype=bool), + adjusted_incidence=np.asarray(incidence_for_flag, dtype=np.float64), + band_names=names, + ) + + +def map_uniform_to_categorical( + draws: Sequence[float] | np.ndarray, + *, + gate: Sequence[bool] | np.ndarray, + resource: Mapping[str, Any], +) -> np.ndarray: + """Mirror PolicyEngine-UK's normalized categorical inverse CDF.""" + + draws = np.asarray(draws, dtype=np.float64) + gate = np.asarray(gate, dtype=bool) + if draws.shape != gate.shape: + raise ValueError("UC deduction type draws and gates must align.") + _validate_uniforms(draws) + rows = resource["type_combination"]["shares"] + shares = np.asarray([float(row["share"]) for row in rows], dtype=np.float64) + total = shares.sum() + if total <= 0.0: + return np.full(draws.shape, "NONE", dtype=object) + cumulative = np.cumsum(shares / total) + index = np.searchsorted( + cumulative, + np.clip(draws, 0.0, 1.0 - 1e-9), + side="right", + ) + index = np.clip(index, 0, len(shares) - 1) + names = np.asarray([str(row["name"]) for row in rows], dtype=object) + return np.where(gate, names[index], "NONE").astype(object) + + +def validate_uc_deduction_resource(resource: Mapping[str, Any]) -> None: + """Refuse malformed or semantically drifted deduction distributions.""" + + if resource.get("version") != 1 or resource.get("country") != "uk": + raise ValueError("UC deduction resource must declare version 1 and country uk.") + latent = _mapping( + resource.get("latent_rate_distribution"), + label="latent_rate_distribution", + ) + if latent.get("parameter_path") != ( + "gov.simulation.uc_deductions.latent_rate_distribution" + ): + raise ValueError("UC deduction latent-rate parameter path drifted.") + cap = _mapping(latent.get("calibration_cap"), label="calibration_cap") + if cap.get("parameter_path") != "gov.simulation.uc_deductions.calibration_cap": + raise ValueError("UC deduction calibration-cap parameter path drifted.") + if float(cap.get("value", -1.0)) != 0.25: + raise ValueError("UC deduction calibration cap must be 0.25.") + bands = latent.get("bands") + if not isinstance(bands, list) or tuple(row.get("name") for row in bands) != ( + UC_DEDUCTION_BANDS + ): + raise ValueError("UC deduction latent-rate band order drifted.") + previous_upper = 0.0 + for row in bands: + lower = _finite_number(row.get("lower"), label=f"{row.get('name')}.lower") + upper = _finite_number(row.get("upper"), label=f"{row.get('name')}.upper") + share = _finite_number(row.get("share"), label=f"{row.get('name')}.share") + if lower != previous_upper or not 0.0 <= lower <= upper <= 0.30 or share <= 0.0: + raise ValueError("UC deduction latent-rate bands are not well formed.") + previous_upper = upper + if not np.isclose(sum(_latent_shares(resource)), 0.467, rtol=0.0, atol=1e-12): + raise ValueError("UC deduction latent-rate shares must sum to 0.467.") + + combinations = _mapping(resource.get("type_combination"), label="type_combination") + if combinations.get("parameter_path") != ( + "gov.simulation.uc_deductions.type_combination" + ): + raise ValueError("UC deduction combination parameter path drifted.") + rows = combinations.get("shares") + if not isinstance(rows, list) or tuple(row.get("name") for row in rows) != ( + UC_DEDUCTION_COMBINATIONS + ): + raise ValueError("UC deduction combination order drifted.") + shares = _combination_shares(resource) + if any(not np.isfinite(value) or value <= 0.0 for value in shares): + raise ValueError("UC deduction combination shares must be finite and positive.") + if not np.isclose(sum(shares), 1.004, rtol=0.0, atol=1e-12): + raise ValueError( + "UC deduction combination shares must retain published sum 1.004." + ) + + region = _mapping( + resource.get("region_incidence_factor"), + label="region_incidence_factor", + ) + if region.get("parameter_path") != ( + "gov.simulation.uc_deductions.region_incidence_factor" + ): + raise ValueError("UC deduction region-factor parameter path drifted.") + factors = _mapping(region.get("factors"), label="region_incidence_factor.factors") + if set(factors) != UC_DEDUCTION_REGIONS: + raise ValueError("UC deduction region-factor domain drifted.") + if any( + not np.isfinite(float(value)) or float(value) <= 0.0 + for value in factors.values() + ): + raise ValueError("UC deduction region factors must be finite and positive.") + + +def _identity_float32_uniforms( + ids: Sequence[object] | np.ndarray, + *, + seed: int, + salt: str, +) -> np.ndarray: + draws = stable_identity_uniforms(ids, seed=seed, salt=salt) + rounded = np.asarray(draws, dtype=np.float32) + return np.minimum(rounded, np.float32(FLOAT32_UNIFORM_MAX)).astype(np.float64) + + +def _incidence_receipt( + regions: np.ndarray, + *, + assigned: np.ndarray, + adjusted_incidence: np.ndarray, + weights: np.ndarray, +) -> dict[str, Mapping[str, float | int]]: + receipt: dict[str, Mapping[str, float | int]] = {} + for region in sorted(set(regions)): + cell = regions == region + target = float(adjusted_incidence[cell][0]) + receipt[str(region)] = _realization_row( + target=target, + selected=assigned[cell], + weights=weights[cell], + ) + return receipt + + +def _categorical_receipt( + *, + names: Sequence[str], + assignments: np.ndarray, + selected: np.ndarray, + target_weights: Sequence[float], + weights: np.ndarray, +) -> dict[str, Mapping[str, float | int]]: + targets = np.asarray(target_weights, dtype=np.float64) + targets = targets / targets.sum() + receipt: dict[str, Mapping[str, float | int]] = {} + for name, target in zip(names, targets, strict=True): + receipt[str(name)] = _realization_row( + target=float(target), + selected=(assignments == name), + weights=weights, + population=selected, + ) + return receipt + + +def _realization_row( + *, + target: float, + selected: np.ndarray, + weights: np.ndarray, + population: np.ndarray | None = None, +) -> Mapping[str, float | int]: + """One receipt cell: the unweighted realization is gated, the weighted one reported. + + ``population`` restricts the denominator (the assigned units for band and + combination cells); ``selected`` marks the units counted in the numerator. + Identity-keyed draws give every unit the same probability whatever its + weight, so the unweighted share is the mechanism's own statistic and takes + the binomial band over ``rows``; the weighted share adds the frame's weight + variance (its effective sample size is reported alongside) and is the + figure the engine round-trip compares with the publisher. + """ + + if population is None: + population = np.ones(selected.shape, dtype=bool) + numerator = selected & population + rows = int(population.sum()) + realized = float(numerator.sum()) / rows if rows > 0 else 0.0 + weight_total = float(weights[population].sum()) + realized_weighted = ( + float(weights[numerator].sum()) / weight_total if weight_total > 0.0 else 0.0 + ) + squares = float((weights[population] ** 2).sum()) + effective_rows = (weight_total**2 / squares) if squares > 0.0 else 0.0 + return { + "target": target, + "realized": realized, + "tolerance": latent_attribute_tolerance(target=target, rows=rows), + "rows": rows, + "realized_weighted": realized_weighted, + "weighted_effective_rows": effective_rows, + } + + +def _latent_shares(resource: Mapping[str, Any]) -> tuple[float, ...]: + return tuple( + float(row["share"]) for row in resource["latent_rate_distribution"]["bands"] + ) + + +def _combination_shares(resource: Mapping[str, Any]) -> tuple[float, ...]: + return tuple(float(row["share"]) for row in resource["type_combination"]["shares"]) + + +def _assert_stage_parameters(stage: SourceStageSpec) -> None: + _assert_closed_world_operations( + stage, + ( + ( + "assign_uniform_draw", + {"output": "uc_deduction_random_draw", "seed": 0}, + ), + ( + "assign_uniform_draw", + {"output": "uc_deduction_type_random_draw", "seed": 0}, + ), + ( + "map_uniform_to_banded_rate", + { + "output": "uc_latent_deduction_rate", + "draw": "uc_deduction_random_draw", + "draw_dtype": "float32", + "resource": UC_DEDUCTION_RESOURCE, + "distribution": "latent_rate_distribution", + "incidence_modifier": { + "column": "region", + "entity": "household", + "table": "region_incidence_factor", + }, + "none_value": 0.0, + }, + ), + ( + "map_uniform_to_categorical", + { + "output": "uc_deduction_combination", + "draw": "uc_deduction_type_random_draw", + "draw_dtype": "float32", + "resource": UC_DEDUCTION_RESOURCE, + "distribution": "type_combination", + "gate": { + "column": "uc_latent_deduction_rate", + "positive": True, + }, + "none_value": "NONE", + }, + ), + ), + ) + + +def _validate_uniforms(draws: np.ndarray) -> None: + if not np.isfinite(draws).all() or (draws < 0.0).any() or (draws >= 1.0).any(): + raise ValueError("UC deduction draws must be finite values in [0, 1).") + + +def _mapping(value: object, *, label: str) -> Mapping[str, Any]: + if not isinstance(value, Mapping): + raise ValueError(f"{label} must be an object.") + return value + + +def _finite_number(value: object, *, label: str) -> float: + if not isinstance(value, int | float) or not np.isfinite(float(value)): + raise ValueError(f"{label} must be finite, got {value!r}.") + return float(value) + + +def _require_columns( + table: pd.DataFrame, columns: Sequence[str], *, label: str +) -> None: + missing = sorted(set(columns) - set(table.columns)) + if missing: + raise ValueError(f"UC deduction attributes {label} columns missing: {missing}.") + + +def _enum_name(value: object) -> str: + name = getattr(value, "name", None) + return str(name if name is not None else value) diff --git a/packages/microcosm-build/src/microcosm/build/uk_runtime/was_wealth.py b/packages/microcosm-build/src/microcosm/build/uk_runtime/was_wealth.py index 1a3879f7f..59b88f689 100644 --- a/packages/microcosm-build/src/microcosm/build/uk_runtime/was_wealth.py +++ b/packages/microcosm-build/src/microcosm/build/uk_runtime/was_wealth.py @@ -55,6 +55,11 @@ "capital_income", "is_renting", ) +#: Extra base predictor of the debt chain segment only (segments 1-3 are +#: unchanged so E5's fourteen columns stay byte-equal); declared in the manifest +#: as ``debt_segment_predictors`` and drift-asserted at run time. +UK_WAS_DEBT_SEGMENT_PREDICTORS = ("has_mortgage_tenure",) +UK_WAS_DEBT_OUTPUT_COLUMNS = ("mortgage_debt", "consumer_debt") UK_WAS_WEALTH_OUTPUT_COLUMNS = ( "owned_land", "property_wealth", @@ -70,6 +75,8 @@ "cash_isa", "stocks_and_shares_isa", "student_loan_balance", + "mortgage_debt", + "consumer_debt", ) UK_WAS_WEALTH_HOUSEHOLD_OUTPUT_COLUMNS = tuple( column @@ -146,7 +153,21 @@ "DVPriRntR8": "private_rent_code", "CTAmtR8": "council_tax", "HFINWNTR8_Sum": "net_financial_wealth", + # WAS round-8 household derived variables (UKDS SN 7215, DOI + # 10.5255/UKDA-SN-7215-20; round-8 user guide and derived-variable + # specification). HMortGR8 is total outstanding mortgage debt on the + # household's property (uk-data#426's mortgage measure); HFINWNTR8_exSLC_Sum + # is net financial wealth excluding student loans, so gross less it is the + # non-student-loan financial liabilities used for consumer_debt; Ten1R8 is + # the tenure code (1 own outright, 2 buying with mortgage, 3 part own/part + # rent, 4 rent, 5 rent free, 6 other). The I1 audit on the pinned tab + # confirmed the tenure reading empirically: mortgage debt is positive in + # 99.9% of code-2 and 100% of code-3 rows against 6.9% / 2.4% / 10.8% for + # codes 1 / 4 / 5 (experiments/685-net-new-stages-receipts.md, Part A). + "HFINWNTR8_exSLC_Sum": "net_financial_wealth_exsl", "HFINWR8_SUM": "gross_financial_wealth", + "HMortGR8": "mortgage_debt", + "Ten1R8": "tenure_code", "DVhvalueR8": "main_residence_value", "DVHseValR8_sum": "other_residential_property_value", "DVBlDValR8_sum": "non_residential_property_value", @@ -212,6 +233,7 @@ def __call__(self, frame: Frame) -> Frame: ) ) ) + _assert_debt_segment_predictors(self.stage) household_predictors = recipient_predictors(frame, self.engine) imputation = impute_was_wealth( donor, @@ -307,12 +329,17 @@ def clean_was_household_table(raw: pd.DataFrame) -> pd.DataFrame: cleaned["student_loan_balance"] = ( cleaned["total_loans"] - cleaned["total_loans_exc_slc"] ) + cleaned["consumer_debt"] = ( + cleaned["gross_financial_wealth"] - cleaned["net_financial_wealth_exsl"] + ).clip(lower=0) + cleaned["has_mortgage_tenure"] = cleaned["tenure_code"].isin({2, 3}) cleaned["region"] = cleaned["region_code"].map(REGIONS) return cleaned[ [ *UK_WAS_WEALTH_PREDICTORS, "weight", "corporate_wealth_excl_isa", + "has_mortgage_tenure", *UK_WAS_WEALTH_HOUSEHOLD_OUTPUT_COLUMNS, "student_loan_balance", ] @@ -378,7 +405,12 @@ def recipient_predictors(frame: Frame, engine: object) -> pd.DataFrame: result[predictor] = household[predictor].to_numpy() result["region"] = result["region"].map(_enum_name).replace(REGION_REMAP) result["is_renting"] = result["is_renting"].astype(bool) - return result.loc[:, UK_WAS_WEALTH_PREDICTORS] + if "tenure_type" not in household.columns: + raise KeyError("recipient household table is missing 'tenure_type'.") + result["has_mortgage_tenure"] = ( + household["tenure_type"].map(_enum_name) == "OWNED_WITH_MORTGAGE" + ) + return result.loc[:, (*UK_WAS_WEALTH_PREDICTORS, "has_mortgage_tenure")] @dataclass(frozen=True) @@ -391,7 +423,7 @@ class UKWASWealthImputationResult: segment_seeds: tuple[int, ...] = () -def was_wealth_segment_seeds(seed: int, segments: int = 3) -> tuple[int, ...]: +def was_wealth_segment_seeds(seed: int, segments: int = 4) -> tuple[int, ...]: """Derive one independent RNG root per chain segment from the stage seed. :meth:`RegimeGatedQRF.start_chain` spawns its fit and draw streams from @@ -410,17 +442,32 @@ def was_wealth_segment_seeds(seed: int, segments: int = 3) -> tuple[int, ...]: ) +UK_WAS_CHAIN_SEGMENTS = 4 + + def impute_was_wealth( donor: pd.DataFrame, recipient_predictor_frame: pd.DataFrame, *, seed: int, n_estimators: int, + segments: int = UK_WAS_CHAIN_SEGMENTS, ) -> UKWASWealthImputationResult: - """Fit segmented checkpointed QRF chains and draw WAS wealth outputs.""" + """Fit segmented checkpointed QRF chains and draw WAS wealth outputs. + + ``segments`` runs only the first *k* chain segments (a test seam: the + fourth, debt, segment draws from the fourth child seed, so the first three + segments' draws are identical whether or not it runs, and the E5 columns + stay byte-equal by construction). + """ from microcosm.fit import RegimeGatedQRF + if not 1 <= int(segments) <= UK_WAS_CHAIN_SEGMENTS: + raise ValueError( + f"segments must be between 1 and {UK_WAS_CHAIN_SEGMENTS}, got {segments!r}." + ) + donor_encoded, recipient_encoded, encoded_predictors = encode_qrf_predictor_pair( donor, recipient_predictor_frame ) @@ -464,6 +511,8 @@ def run_segment(base_predictors: Sequence[str], targets: Sequence[str]) -> None: base = encoded_predictors run_segment(base, ("owned_land", "property_wealth")) + if segments == 1: + return _partial_result(raw, fit_records, segment_seeds) donor_encoded["corporate_wealth"] = donor_encoded["corporate_wealth"].astype(float) donor_encoded["private_pension_wealth"] = donor_encoded[ "private_pension_wealth" @@ -484,6 +533,8 @@ def run_segment(base_predictors: Sequence[str], targets: Sequence[str]) -> None: raw["corporate_wealth"] = ( raw["corporate_wealth_excl_isa"] + raw["stocks_and_shares_isa"] ) + if segments == 2: + return _partial_result(raw, fit_records, segment_seeds) recipient_encoded["private_pension_wealth"] = raw["private_pension_wealth"] recipient_encoded["corporate_wealth"] = raw["corporate_wealth"] # Downstream targets condition on both components, carrying the @@ -508,6 +559,19 @@ def run_segment(base_predictors: Sequence[str], targets: Sequence[str]) -> None: "cash_isa", ), ) + if segments == 3: + return _partial_result(raw, fit_records, segment_seeds) + prior_outputs = tuple( + column + for column in UK_WAS_WEALTH_OUTPUT_COLUMNS + if column not in UK_WAS_DEBT_OUTPUT_COLUMNS + ) + for output in prior_outputs: + recipient_encoded[output] = raw[output] + run_segment( + (*base, *UK_WAS_DEBT_SEGMENT_PREDICTORS, *prior_outputs), + UK_WAS_DEBT_OUTPUT_COLUMNS, + ) return UKWASWealthImputationResult( draws=raw.loc[:, UK_WAS_WEALTH_OUTPUT_COLUMNS], fit_weight_records=tuple(fit_records), @@ -515,6 +579,19 @@ def run_segment(base_predictors: Sequence[str], targets: Sequence[str]) -> None: ) +def _partial_result( + raw: pd.DataFrame, + fit_records: list[FitWeightRecord], + segment_seeds: tuple[int, ...], +) -> UKWASWealthImputationResult: + produced = [column for column in UK_WAS_WEALTH_OUTPUT_COLUMNS if column in raw] + return UKWASWealthImputationResult( + draws=raw.loc[:, produced], + fit_weight_records=tuple(fit_records), + segment_seeds=segment_seeds, + ) + + def encode_qrf_predictor_pair( donor: pd.DataFrame, recipient: pd.DataFrame, @@ -547,6 +624,9 @@ def _encode(table: pd.DataFrame, block: pd.DataFrame) -> pd.DataFrame: encoded = table.drop(columns=["region"]).copy() if "is_renting" in encoded.columns: encoded["is_renting"] = encoded["is_renting"].astype(bool).astype(float) + for column in UK_WAS_DEBT_SEGMENT_PREDICTORS: + if column in encoded.columns: + encoded[column] = encoded[column].astype(bool).astype(float) for column in encoded.columns: if column == "weight": continue @@ -659,6 +739,20 @@ def _donor_artifact(stage: SourceStageSpec) -> Mapping[str, Any]: ) +def _assert_debt_segment_predictors(stage: SourceStageSpec) -> None: + """The chain op must declare exactly the debt segment's extra predictors.""" + + operation = next( + op for op in stage.operations if op.kind == "fit_weighted_qrf_chain" + ) + declared = tuple(operation.parameters.get("debt_segment_predictors", ())) + if declared != UK_WAS_DEBT_SEGMENT_PREDICTORS: + raise ValueError( + "was_wealth debt_segment_predictors drifted: manifest declares " + f"{declared!r}, runtime uses {UK_WAS_DEBT_SEGMENT_PREDICTORS!r}." + ) + + def _qrf_n_estimators(stage: SourceStageSpec) -> int: for operation in stage.operations: if operation.kind == "fit_weighted_qrf_chain": diff --git a/packages/microcosm-build/tests/test_country_spec.py b/packages/microcosm-build/tests/test_country_spec.py index 28359b1be..7eabedef4 100644 --- a/packages/microcosm-build/tests/test_country_spec.py +++ b/packages/microcosm-build/tests/test_country_spec.py @@ -922,6 +922,7 @@ def test_spi_spine_adds_no_country_package_resources(self) -> None: "lcfs_consumption_anchors.json", "etb_policy_anchors.json", "etb_services_anchors.json", + "dwp_uc_deduction_distributions.json", "nhs_consumption_by_age_gender.json", "ons_age_tail_band_populations.json", "lcfs_consumption_support_bounds.json", @@ -943,6 +944,7 @@ def test_spi_spine_adds_no_country_package_resources(self) -> None: "registry_parity_fixture_2025.json", "local_registry_parity_fixture_2025.json", "was_wealth_support_bounds.json", + "uc_deduction_support_bounds.json", "local_binding_adjudications.json", "uk_local_target_census.json", "uk_data_target_parity.json", @@ -957,13 +959,14 @@ def test_spi_spine_adds_no_country_package_resources(self) -> None: "local_target_reference_membership.json", ) - def test_uk_source_manifest_loads_twenty_nine_stages(self) -> None: + def test_uk_source_manifest_loads_thirty_stages(self) -> None: spec = load_country_spec("uk") assert spec.sources is not None - # 27 spine stages (uc_reporter_redraw is the newest, #832) plus the + # 28 spine stages (uc_reporter_redraw #832, then uc_deduction_attributes + # #685 as the newest) plus the # two certified-pair stages the June path still uses. - assert len(spec.sources.stages) == 29 + assert len(spec.sources.stages) == 30 class TestExistingPackagesGeneralize: @@ -1009,6 +1012,7 @@ def test_uk_package_loads(self) -> None: "lcfs_consumption_anchors.json", "etb_policy_anchors.json", "etb_services_anchors.json", + "dwp_uc_deduction_distributions.json", "nhs_consumption_by_age_gender.json", "ons_age_tail_band_populations.json", "lcfs_consumption_support_bounds.json", @@ -1030,6 +1034,7 @@ def test_uk_package_loads(self) -> None: "registry_parity_fixture_2025.json", "local_registry_parity_fixture_2025.json", "was_wealth_support_bounds.json", + "uc_deduction_support_bounds.json", "local_binding_adjudications.json", "uk_local_target_census.json", "uk_data_target_parity.json", @@ -1280,6 +1285,7 @@ def test_declares_the_full_june_battery(self, manifest) -> None: "uk_ledger_compile_parity_local_incumbent_2025", "uk_target_surface_local_default_2025", "uk_stage_was_wealth_support", + "uk_stage_uc_deduction_attributes", "uk_stage_lcfs_consumption_support", "uk_stage_etb_vat_support", "uk_stage_etb_services_support", @@ -1305,6 +1311,7 @@ def test_declares_the_full_june_battery(self, manifest) -> None: "uk_export_surface", "uk_take_up_signal", "uk_brma_enum_domain", + "uk_uc_deduction_combination_enum_domain", "uk_student_loan_plan_enum_domain", "uk_calibration_reference_coverage", "uk_target_surface", @@ -1370,6 +1377,7 @@ def test_strict_absent_evidence_entries_are_declared(self, manifest) -> None: flagged = [g.id for g in manifest.gates if g.evidence_absent_blocks] assert flagged == [ "uk_stage_was_wealth_support", + "uk_stage_uc_deduction_attributes", "uk_stage_lcfs_consumption_support", "uk_stage_etb_vat_support", "uk_stage_etb_services_support", @@ -1416,6 +1424,7 @@ def test_thresholds_match_the_schema4_manifest(self, manifest) -> None: "lcfs_consumption_support_bounds.json", "etb_vat_support_bounds.json", "etb_services_support_bounds.json", + "uc_deduction_support_bounds.json", ) aggregate = params["uk_aggregate_admin"] assert aggregate["default_rtol"] == 0.15 diff --git a/packages/microcosm-build/tests/test_spec_engine_country_bundles.py b/packages/microcosm-build/tests/test_spec_engine_country_bundles.py index 225af1c0d..d37d934d0 100644 --- a/packages/microcosm-build/tests/test_spec_engine_country_bundles.py +++ b/packages/microcosm-build/tests/test_spec_engine_country_bundles.py @@ -55,7 +55,7 @@ ), ( "uk", - "8e6f5861362e1e23b650c1d2937995bebee573bb9f1cab51fc8e1fef2cef8a9c", + "aa6064bbdf576df88793ff99d706a21af9c8430863fcd3df13856877062c079c", { "benunit.benunit_id", "household.household_id", diff --git a/packages/microcosm-build/tests/test_uk_battery_bindings.py b/packages/microcosm-build/tests/test_uk_battery_bindings.py index b35577176..86d923c3b 100644 --- a/packages/microcosm-build/tests/test_uk_battery_bindings.py +++ b/packages/microcosm-build/tests/test_uk_battery_bindings.py @@ -595,12 +595,13 @@ def test_fully_armed_battery_evaluates_gate_for_gate(self) -> None: # student-loan enum gate; their evaluators have direct tests. The BRMA # enum gate is no longer among them: it moved to the spine battery's # assembled boundary, where its column is first written. - # 15 before this lane, plus uk_uc_capital_coherence from #829, plus + # 15 before this lane, plus uk_uc_capital_coherence from #829 and the + # UC-deduction enum gate from #685, plus # the two frame-only local weight diagnostics that also pass in this # unscoped compatibility probe. The local ladder gate fails because # this national fixture deliberately carries no ladder columns; the # three evidence-backed local arms are named gaps below. - assert len(passed) == 18 + assert len(passed) == 19 qrf = by_id["uk_qrf_tail_concentration"] assert qrf.status is GateStatus.FAILED assert "declared QRF output is absent" in qrf.result.failures[0] diff --git a/packages/microcosm-build/tests/test_uk_frs_spine.py b/packages/microcosm-build/tests/test_uk_frs_spine.py index 9dd7a3b16..dfb1f3b67 100644 --- a/packages/microcosm-build/tests/test_uk_frs_spine.py +++ b/packages/microcosm-build/tests/test_uk_frs_spine.py @@ -1689,6 +1689,10 @@ def test_e8_manifest_seeds_all_reach_the_build_sidecar_harvester() -> None: assert declared["cgt_incidence_clone"] == {"cgt_prior_amount": 0} assert declared["uc_capital_coherence"] == {"frs_benunit_capital": 0} + assert declared["uc_deduction_attributes"] == { + "uc_deduction_random_draw": 0, + "uc_deduction_type_random_draw": 0, + } assert declared["cgt_band_donors"] == {"stack_band_donor_households": 1} assert declared["hmrc_cgt_gains_spine"] == {"within_band_draws": 552} assert declared["salary_sacrifice"] == { @@ -1932,7 +1936,7 @@ def test_in_kind_benefits_map_from_the_raw_person_tapes(tmp_path: Path) -> None: def test_boundary_evidence_asks_only_the_stages_that_have_run() -> None: """The first licensed battery run failed at the assembled boundary because - the evidence provider consulted all 26 implementations, and an un-run + the evidence provider consulted all 27 implementations, and an un-run stage's checkpoint hook (correctly) refuses. Each boundary must offer only its executed prefix — an un-run stage being consulted is the regression. """ diff --git a/packages/microcosm-build/tests/test_uk_graph.py b/packages/microcosm-build/tests/test_uk_graph.py index d5578ef63..890447d37 100644 --- a/packages/microcosm-build/tests/test_uk_graph.py +++ b/packages/microcosm-build/tests/test_uk_graph.py @@ -161,9 +161,10 @@ def test_uk_spine_graph_contains_manifest_stages_and_named_exclusions() -> None: graph = uk_spine_graph(spec) ids = {node.id for node in graph.nodes} - # 27 with the #832 uc_reporter_redraw stage; the two named exclusions are - # the certified-pair alternatives, not steps of this pipeline. - assert len(expected) == 27 + # 28 with the #832 uc_reporter_redraw and #685 uc_deduction_attributes + # stages; the two named exclusions are the certified-pair alternatives, + # not steps of this pipeline. + assert len(expected) == 28 assert UK_SPINE_EXCLUSIONS == { "frs_hmrc_retained_leaves", "hmrc_spi_income", diff --git a/packages/microcosm-build/tests/test_uk_identity_stability_receipts.py b/packages/microcosm-build/tests/test_uk_identity_stability_receipts.py index cc663745b..12c0074da 100644 --- a/packages/microcosm-build/tests/test_uk_identity_stability_receipts.py +++ b/packages/microcosm-build/tests/test_uk_identity_stability_receipts.py @@ -212,9 +212,7 @@ def materialize(self, frame, variables, period): } ), benunit=pd.DataFrame({"benunit_id": [10], "benunit_household_id": [100]}), - household=pd.DataFrame( - {"household_id": [100], "household_weight": [1.0]} - ), + household=pd.DataFrame({"household_id": [100], "household_weight": [1.0]}), time_period="2024", weight_kind=WeightKind.DESIGN, ) @@ -319,9 +317,7 @@ def test_e8_carrier_recompute_is_invariant_to_the_age_tail_rewrite(): clamped = after_age_tail.assign( age=np.minimum( - pd.to_numeric(after_age_tail["age"], errors="coerce").to_numpy( - dtype=float - ), + pd.to_numeric(after_age_tail["age"], errors="coerce").to_numpy(dtype=float), float(TOP), ) ) @@ -329,3 +325,95 @@ def test_e8_carrier_recompute_is_invariant_to_the_age_tail_rewrite(): _oldest_adult_indices(clamped, household_ids={7}).tolist() == stage_choice.tolist() ) + + +def _e9_frame(*, clone_flag: bool = False): + """Three benunits over two households with a region, ready for the E9 stage.""" + + from microcosm.build.uk_runtime.cgt_structure import HOUSEHOLD_IS_CGT_CLONE + + person = pd.DataFrame( + { + "person_id": [1, 2, 3, 4], + "person_benunit_id": [10, 10, 20, 30], + "person_household_id": [100, 100, 200, 200], + } + ) + benunit = pd.DataFrame({"benunit_id": [10, 20, 30]}) + household = pd.DataFrame( + { + "household_id": [100, 200], + "household_weight": [10.0, 20.0], + "region": ["LONDON", "NORTH_EAST"], + } + ) + if clone_flag: + household[HOUSEHOLD_IS_CGT_CLONE] = [False, True] + return uk_national_frame( + person=person, + benunit=benunit, + household=household, + time_period="2024", + weight_kind=WeightKind.DESIGN, + ) + + +class TestE9Receipt: + def test_stage_output_recomputes_identically_and_matches_stored(self) -> None: + from microcosm.build.uk_runtime.uc_deduction_attributes import ( + assign_uc_deduction_attributes, + load_uc_deduction_distributions, + ) + + tool = _load_tool() + staged = assign_uc_deduction_attributes( + _e9_frame(), resource=load_uc_deduction_distributions() + ).frame + receipt = tool.e9_identity_receipt(staged, permutation_seed=7) + + assert receipt["identical_under_permutation"] is True + assert receipt["matches_stored_columns"] is True + assert receipt["benunits_recomputed"] == 3 + assert receipt["benunits_excluded_as_copies"] == 0 + + def test_a_tampered_stored_rate_is_reported(self) -> None: + from microcosm.build.uk_runtime.uc_deduction_attributes import ( + assign_uc_deduction_attributes, + load_uc_deduction_distributions, + ) + + tool = _load_tool() + staged = assign_uc_deduction_attributes( + _e9_frame(), resource=load_uc_deduction_distributions() + ).frame + benunit = staged.table("benunit").copy() + benunit.loc[0, "uc_latent_deduction_rate"] = 0.123 + tampered = uk_national_frame( + person=staged.table("person").copy(), + benunit=benunit, + household=staged.table("household").copy(), + time_period="2024", + weight_kind=WeightKind.DESIGN, + household_weights=staged.weights_for("household").values, + ) + receipt = tool.e9_identity_receipt(tampered, permutation_seed=7) + + assert receipt["identical_under_permutation"] is True + assert receipt["matches_stored_columns"] is False + assert receipt["stored_mismatches"] == {"benunit": ["uc_latent_deduction_rate"]} + + def test_cloned_households_are_excluded_from_the_recompute(self) -> None: + from microcosm.build.uk_runtime.uc_deduction_attributes import ( + assign_uc_deduction_attributes, + load_uc_deduction_distributions, + ) + + tool = _load_tool() + staged = assign_uc_deduction_attributes( + _e9_frame(clone_flag=True), resource=load_uc_deduction_distributions() + ).frame + receipt = tool.e9_identity_receipt(staged, permutation_seed=7) + + assert receipt["benunits_recomputed"] == 1 + assert receipt["benunits_excluded_as_copies"] == 2 + assert receipt["matches_stored_columns"] is True diff --git a/packages/microcosm-build/tests/test_uk_source_runtime.py b/packages/microcosm-build/tests/test_uk_source_runtime.py index d9901c77a..7bae6abf3 100644 --- a/packages/microcosm-build/tests/test_uk_source_runtime.py +++ b/packages/microcosm-build/tests/test_uk_source_runtime.py @@ -115,6 +115,7 @@ def hmrc(frame: Frame) -> Frame: retained_leaves_transform=retained, hmrc_income_transform=hmrc, was_wealth_transform=retained, + uc_deduction_attributes_transform=hmrc, regional_property_uprating_transform=hmrc, lcfs_consumption_transform=retained, etb_vat_transform=hmrc, @@ -128,6 +129,7 @@ def hmrc(frame: Frame) -> Frame: "frs_hmrc_retained_leaves": retained, "hmrc_spi_income": hmrc, "was_wealth": retained, + "uc_deduction_attributes": hmrc, "regional_property_uprating": hmrc, "lcfs_consumption": retained, "etb_vat": hmrc, diff --git a/packages/microcosm-build/tests/test_uk_source_stages.py b/packages/microcosm-build/tests/test_uk_source_stages.py index 67c4bf59b..a5348e27e 100644 --- a/packages/microcosm-build/tests/test_uk_source_stages.py +++ b/packages/microcosm-build/tests/test_uk_source_stages.py @@ -54,6 +54,9 @@ UC_COHERENCE_STAGE_NAMES = [ "uc_capital_coherence", ] +E9_STAGE_NAMES = [ + "uc_deduction_attributes", +] E8_STAGE_NAMES = [ "cgt_incidence_clone", "cgt_band_donors", @@ -77,6 +80,7 @@ *E7_STAGE_NAMES, *UC_REPORTER_REDRAW_STAGE_NAMES, *UC_COHERENCE_STAGE_NAMES, + *E9_STAGE_NAMES, *E8_STAGE_NAMES, *POST_E8_STAGE_NAMES, "frs_hmrc_retained_leaves", @@ -159,14 +163,14 @@ def test_e7_block_sits_between_e6_and_e8(self) -> None: canonical = _load_json(CANONICAL_SOURCE_STAGES) names = [stage["stage"] for stage in canonical["stages"]] - assert ( - names[names.index("etb_services") + 1 : names.index("cgt_incidence_clone")] - == [ - *E7_STAGE_NAMES, - *UC_REPORTER_REDRAW_STAGE_NAMES, - *UC_COHERENCE_STAGE_NAMES, - ] - ) + assert names[ + names.index("etb_services") + 1 : names.index("cgt_incidence_clone") + ] == [ + *E7_STAGE_NAMES, + *UC_REPORTER_REDRAW_STAGE_NAMES, + *UC_COHERENCE_STAGE_NAMES, + *E9_STAGE_NAMES, + ] def test_e8_block_is_contiguous_and_the_certified_pair_stays_last(self) -> None: # Two invariants, and only two: the E8 stages stay contiguous, and the @@ -300,6 +304,7 @@ def test_country_stage_plan_assembles_spine_plan(self) -> None: "hmrc_spi_income_spine": _identity, "uc_reporter_redraw": _identity, "uc_capital_coherence": _identity, + "uc_deduction_attributes": _identity, "cgt_incidence_clone": _identity, "cgt_band_donors": _identity, "hmrc_cgt_gains_spine": _identity, @@ -394,6 +399,10 @@ def test_e3_outputs_are_backed_by_runtime_written_columns(self) -> None: from microcosm.build.uk_runtime.regional_uprating import ( UK_REGIONAL_PROPERTY_REWRITES, ) + from microcosm.build.uk_runtime.uc_deduction_attributes import ( + UC_DEDUCTION_NONNEGATIVE_OUTPUT_COLUMNS, + UC_DEDUCTION_OUTPUT_COLUMNS, + ) from microcosm.build.uk_runtime.was_wealth import ( UK_WAS_WEALTH_NONNEGATIVE_OUTPUT_COLUMNS, UK_WAS_WEALTH_OUTPUT_COLUMNS, @@ -433,6 +442,11 @@ def test_e3_outputs_are_backed_by_runtime_written_columns(self) -> None: stages["was_wealth"].nonnegative_outputs == UK_WAS_WEALTH_NONNEGATIVE_OUTPUT_COLUMNS ) + assert stages["uc_deduction_attributes"].outputs == UC_DEDUCTION_OUTPUT_COLUMNS + assert ( + stages["uc_deduction_attributes"].nonnegative_outputs + == UC_DEDUCTION_NONNEGATIVE_OUTPUT_COLUMNS + ) assert stages["regional_property_uprating"].outputs == () assert ( stages["regional_property_uprating"].rewrites @@ -655,6 +669,12 @@ def test_e3_operation_kinds_are_declared_in_order(self) -> None: "redraw_spi_reporter_capital", "derive", ] + assert [op.kind for op in stages["uc_deduction_attributes"].operations] == [ + "assign_uniform_draw", + "assign_uniform_draw", + "map_uniform_to_banded_rate", + "map_uniform_to_categorical", + ] assert [op.kind for op in stages["cgt_incidence_clone"].operations] == [ "clone_records", "draw_capital_gains_prior_from_banded_quantiles", @@ -839,6 +859,27 @@ def test_every_e4_stochastic_operation_declares_integer_seed(self) -> None: }: assert isinstance(operation.parameters.get("seed"), int) + def test_e5_debt_segment_predictors_lockstep(self) -> None: + from microcosm.build.uk_runtime.was_wealth import ( + UK_WAS_DEBT_SEGMENT_PREDICTORS, + UK_WAS_WEALTH_PREDICTORS, + ) + + spec = load_country_spec("uk") + stages = {stage.stage: stage for stage in spec.sources.stages} + qrf = stages["was_wealth"].operations[2] + + assert qrf.kind == "fit_weighted_qrf_chain" + assert tuple(qrf.parameters["debt_segment_predictors"]) == ( + UK_WAS_DEBT_SEGMENT_PREDICTORS + ) + # The extra predictor belongs to the debt segment only: the shared + # base list, and so E5's first three segments, are unchanged. + assert not set(UK_WAS_DEBT_SEGMENT_PREDICTORS) & set( + qrf.parameters["predictors"] + ) + assert tuple(qrf.parameters["predictors"]) == UK_WAS_WEALTH_PREDICTORS + def test_e5_qrf_operation_declares_integer_seed(self) -> None: spec = load_country_spec("uk") stages = {stage.stage: stage for stage in spec.sources.stages} @@ -892,6 +933,20 @@ def test_e8_declared_seed_lockstep(self) -> None: for operation in stages["student_loans"].operations[1:] ] == [42, 42] + def test_e9_declared_seed_lockstep(self) -> None: + from microcosm.build.uk_runtime.uc_deduction_attributes import ( + UK_UC_DEDUCTION_ATTRIBUTES_DECLARED_SEEDS, + ) + + stage = load_country_spec("uk").sources.stage_map()["uc_deduction_attributes"] + declared = { + operation.parameters["output"]: operation.parameters["seed"] + for operation in stage.operations + if operation.kind == "assign_uniform_draw" + } + + assert declared == UK_UC_DEDUCTION_ATTRIBUTES_DECLARED_SEEDS + def test_full_uk_source_stage_plan_compiles_with_e4_stages(self) -> None: spec = load_country_spec("uk") implementations = {name: _identity for name in UK_SOURCE_STAGE_NAMES} diff --git a/packages/microcosm-build/tests/test_uk_spine_acceptance_receipt.py b/packages/microcosm-build/tests/test_uk_spine_acceptance_receipt.py index 23be9019e..fb559c56b 100644 --- a/packages/microcosm-build/tests/test_uk_spine_acceptance_receipt.py +++ b/packages/microcosm-build/tests/test_uk_spine_acceptance_receipt.py @@ -45,12 +45,20 @@ def test_receipt_roster_is_the_production_plan(): receipt = _receipt() accepted_roster = tuple(receipt["candidate"]["stage_roster"]) production_roster = _production_graph_stage_names() - expected_extra_stages = {"uc_capital_coherence", "uc_reporter_redraw"} + expected_extra_stages = { + "uc_capital_coherence", + "uc_reporter_redraw", + "uc_deduction_attributes", + } assert set(production_roster) - set(accepted_roster) == expected_extra_stages assert tuple( stage for stage in production_roster if stage not in expected_extra_stages ) == accepted_roster + # The E9 stage runs directly after the capital-coherence stage. + assert production_roster.index("uc_deduction_attributes") == ( + production_roster.index("uc_capital_coherence") + 1 + ) assert receipt["candidate"]["stage_count"] == len(accepted_roster) diff --git a/packages/microcosm-build/tests/test_uk_stage_health.py b/packages/microcosm-build/tests/test_uk_stage_health.py index 8ec28f067..af08407df 100644 --- a/packages/microcosm-build/tests/test_uk_stage_health.py +++ b/packages/microcosm-build/tests/test_uk_stage_health.py @@ -389,3 +389,76 @@ class the #787 review named. A non-exempt column now needs both bounds assert not _passed(result) assert any("no clipped_low_rows allowance" in f for f in result.failures) assert any("no clipped_high_rows allowance" in f for f in result.failures) + + +def _latent_receipt() -> dict: + row = {"target": 0.5, "realized": 0.51, "tolerance": 0.05, "rows": 1000} + return { + "stage": "uc_deduction_attributes", + "coherence_violation_count": 0, + "incidence_by_region": {"LONDON": dict(row)}, + "latent_rate_bands": {"AT_25": dict(row)}, + "combination_shares": {"ADVANCE_ONLY": dict(row)}, + } + + +def _latent_gate(evidence: dict): + return uk_stage_health_gate( + evidence=evidence, + stage="uc_deduction_attributes", + check="latent_attribute_realization", + parameters={ + "stage": "uc_deduction_attributes", + "check": "latent_attribute_realization", + }, + ) + + +def test_latent_attribute_realization_passes_a_coherent_in_band_receipt() -> None: + result = _latent_gate(_latent_receipt()) + + assert _passed(result) + assert result.details["cells_checked"] == 3 + assert result.details["coherence_violation_count"] == 0 + + +def test_latent_attribute_realization_fails_on_coherence_violations() -> None: + evidence = _latent_receipt() + evidence["coherence_violation_count"] = 2 + + assert not _latent_gate(evidence).passed + + +def test_latent_attribute_realization_fails_when_the_count_is_missing() -> None: + evidence = _latent_receipt() + del evidence["coherence_violation_count"] + + assert not _latent_gate(evidence).passed + + +def test_latent_attribute_realization_fails_beyond_the_declared_tolerance() -> None: + evidence = _latent_receipt() + evidence["latent_rate_bands"]["AT_25"]["realized"] = 0.56 + + assert not _latent_gate(evidence).passed + + +def test_latent_attribute_realization_caps_a_widened_producer_tolerance() -> None: + # 1,000 rows at a 0.5 share give a four-sigma band of ~0.063; a producer + # that declares 1.0 must not widen the pass rule. + evidence = _latent_receipt() + evidence["incidence_by_region"]["LONDON"].update( + {"tolerance": 1.0, "realized": 0.6} + ) + + assert not _latent_gate(evidence).passed + + +def test_latent_attribute_realization_fails_on_empty_blocks_and_zero_rows() -> None: + empty = _latent_receipt() + empty["combination_shares"] = {} + assert not _latent_gate(empty).passed + + zero_rows = _latent_receipt() + zero_rows["incidence_by_region"]["LONDON"]["rows"] = 0 + assert not _latent_gate(zero_rows).passed diff --git a/packages/microcosm-build/tests/test_uk_uc_capital_coherence.py b/packages/microcosm-build/tests/test_uk_uc_capital_coherence.py index 172d8c682..13fdcb83d 100644 --- a/packages/microcosm-build/tests/test_uk_uc_capital_coherence.py +++ b/packages/microcosm-build/tests/test_uk_uc_capital_coherence.py @@ -104,7 +104,7 @@ def test_stage_orders_after_every_universal_credit_report_writer() -> None: assert reporter_writers assert all(index < coherence_index for index, _ in reporter_writers) - assert stages[coherence_index + 1].stage == "cgt_incidence_clone" + assert stages[coherence_index + 1].stage == "uc_deduction_attributes" def test_or_refresh_truth_table_and_same_capital_source() -> None: diff --git a/packages/microcosm-build/tests/test_uk_uc_deduction_attributes.py b/packages/microcosm-build/tests/test_uk_uc_deduction_attributes.py new file mode 100644 index 000000000..5a5c4ca5b --- /dev/null +++ b/packages/microcosm-build/tests/test_uk_uc_deduction_attributes.py @@ -0,0 +1,383 @@ +from __future__ import annotations + +import copy +import json +from pathlib import Path + +import numpy as np +import pandas as pd +import pytest + +from microcosm.build.source_manifest import SourceStageSpec +from microcosm.build.stochastic_assignment import stable_identity_uniforms +from microcosm.build.uk_runtime.national_frame import uk_national_frame +from microcosm.build.uk_runtime.stage_health import uk_stage_health_gate +from microcosm.build.uk_runtime.uc_deduction_attributes import ( + FLOAT32_UNIFORM_MAX, + UC_DEDUCTION_BANDS, + UC_DEDUCTION_COMBINATIONS, + UC_DEDUCTION_OUTPUT_COLUMNS, + UC_DEDUCTION_REGIONS, + UC_DEDUCTION_RESOURCE, + UKUCDeductionAttributesStageTransform, + _identity_float32_uniforms, + load_uc_deduction_distributions, + map_uniform_to_banded_rate, + map_uniform_to_categorical, + validate_uc_deduction_resource, +) + +ROOT = Path(__file__).resolve().parents[3] +RESOURCE = ( + ROOT / "packages/microcosm-build/src/microcosm/build/uk" / UC_DEDUCTION_RESOURCE +) + + +def _stage_mapping() -> dict[str, object]: + return { + "stage": "uc_deduction_attributes", + "survey": "test", + "source": "test", + "grain": "benunit", + "artifacts": [], + "operations": [ + { + "kind": "assign_uniform_draw", + "output": "uc_deduction_random_draw", + "seed": 0, + }, + { + "kind": "assign_uniform_draw", + "output": "uc_deduction_type_random_draw", + "seed": 0, + }, + { + "kind": "map_uniform_to_banded_rate", + "output": "uc_latent_deduction_rate", + "draw": "uc_deduction_random_draw", + "draw_dtype": "float32", + "resource": UC_DEDUCTION_RESOURCE, + "distribution": "latent_rate_distribution", + "incidence_modifier": { + "column": "region", + "entity": "household", + "table": "region_incidence_factor", + }, + "none_value": 0.0, + }, + { + "kind": "map_uniform_to_categorical", + "output": "uc_deduction_combination", + "draw": "uc_deduction_type_random_draw", + "draw_dtype": "float32", + "resource": UC_DEDUCTION_RESOURCE, + "distribution": "type_combination", + "gate": { + "column": "uc_latent_deduction_rate", + "positive": True, + }, + "none_value": "NONE", + }, + ], + "outputs": list(UC_DEDUCTION_OUTPUT_COLUMNS), + "nonnegative_outputs": list(UC_DEDUCTION_OUTPUT_COLUMNS[:3]), + } + + +def _stage() -> SourceStageSpec: + return SourceStageSpec.from_mapping(_stage_mapping()) + + +def _frame(n: int = 512, *, region_names: tuple[str, ...] | None = None): + ids = np.arange(1, n + 1, dtype=np.int64) + regions = np.resize( + np.asarray(region_names or tuple(sorted(UC_DEDUCTION_REGIONS)), dtype=object), + n, + ) + person = pd.DataFrame( + { + "person_id": ids * 10 + 1, + "person_benunit_id": ids, + "person_household_id": ids, + "age": np.full(n, 40), + "is_benunit_head": np.ones(n, dtype=bool), + } + ) + benunit = pd.DataFrame( + { + "benunit_id": ids, + "would_claim_uc": np.ones(n, dtype=bool), + "universal_credit_pre_benefit_cap": np.full(n, 500.0), + "benefit_cap_reduction": np.zeros(n), + } + ) + household = pd.DataFrame( + { + "household_id": ids, + "region": regions, + "council_tax": np.zeros(n), + "tenure_type": np.full(n, "OWNED_OUTRIGHT", dtype=object), + "rent": np.zeros(n), + } + ) + return uk_national_frame( + person=person, + benunit=benunit, + household=household, + household_weights=np.ones(n), + time_period="2024", + ) + + +def test_resource_is_well_formed_and_carries_pinned_provenance() -> None: + resource = json.loads(RESOURCE.read_text(encoding="utf-8")) + + validate_uc_deduction_resource(resource) + assert resource["source"]["ods_sha256"] == ( + "307ec8fa49a1f1e23db3c59f1282e16609de03444e501824e204b4151e2e5c9b" + ) + assert resource["source"]["chronicle_package_id"] == ( + "dwp-uc-deductions-march-2025-february-2026" + ) + assert resource["source"]["chronicle_candidate"] is True + assert ( + tuple(row["name"] for row in resource["latent_rate_distribution"]["bands"]) + == UC_DEDUCTION_BANDS + ) + assert ( + tuple(row["name"] for row in resource["type_combination"]["shares"]) + == UC_DEDUCTION_COMBINATIONS + ) + assert set(resource["region_incidence_factor"]["factors"]) == (UC_DEDUCTION_REGIONS) + + +@pytest.mark.requires_uk +def test_resource_values_lockstep_with_engine_parameter_tree() -> None: + system_module = pytest.importorskip("policyengine_uk.system") + latent_module = pytest.importorskip( + "policyengine_uk.variables.gov.dwp.universal_credit.deductions." + "uc_latent_deduction_rate" + ) + region_module = pytest.importorskip( + "policyengine_uk.variables.household.demographic.geography" + ) + resource = load_uc_deduction_distributions() + parameters = system_module.system.parameters.gov.simulation.uc_deductions + bands = resource["latent_rate_distribution"]["bands"] + + np.testing.assert_array_equal( + np.asarray([row["lower"] for row in bands]), latent_module.BAND_LOWER + ) + np.testing.assert_array_equal( + np.asarray([row["upper"] for row in bands]), latent_module.BAND_UPPER + ) + assert [ + float(getattr(parameters.latent_rate_distribution, row["name"])("2024")) + for row in bands + ] == [row["share"] for row in bands] + assert ( + float(parameters.calibration_cap("2024")) + == (resource["latent_rate_distribution"]["calibration_cap"]["value"]) + ) + combinations = resource["type_combination"]["shares"] + assert [ + float(getattr(parameters.type_combination, row["name"])("2024")) + for row in combinations + ] == [row["share"] for row in combinations] + factors = resource["region_incidence_factor"]["factors"] + assert { + name: float(getattr(parameters.region_incidence_factor, name)("2024")) + for name in factors + } == factors + assert set(region_module.Region.__members__) == UC_DEDUCTION_REGIONS + + +def test_mapping_matches_engine_formula_on_synthetic_draws() -> None: + resource = load_uc_deduction_distributions() + draws = np.asarray([0.0235, 0.05, 0.1, 0.15, 0.5], dtype=np.float32).astype( + np.float64 + ) + regions = np.asarray(["UNKNOWN"] * len(draws)) + rates = map_uniform_to_banded_rate(draws, regions, resource) + combinations = map_uniform_to_categorical( + np.asarray([0.0, 0.33, 0.5, 0.95, 0.2], dtype=np.float32), + gate=rates > 0.0, + resource=resource, + ) + + np.testing.assert_allclose( + rates, + np.asarray( + [ + draws[0] / 0.047 * 0.05, + 0.05, + 0.05 + (draws[2] - 0.077) / 0.07 * 0.05, + 0.1, + 0.0, + ] + ), + atol=1e-7, + ) + assert combinations.tolist() == [ + "ADVANCE_ONLY", + "THIRD_PARTY_ONLY", + "GOVERNMENT_ONLY", + "ALL_THREE", + "NONE", + ] + + +def test_stage_is_identity_keyed_and_float32_rounded_with_enum_names() -> None: + resource = load_uc_deduction_distributions() + original = _frame(512) + transform = UKUCDeductionAttributesStageTransform(stage=_stage(), resource=resource) + assigned = transform(original) + expected_draw = stable_identity_uniforms( + original.table("benunit")["benunit_id"], + seed=0, + salt="uc_deduction_random_draw", + ).astype(np.float32) + expected_draw = np.minimum(expected_draw, np.float32(FLOAT32_UNIFORM_MAX)) + np.testing.assert_array_equal( + assigned.table("benunit")["uc_deduction_random_draw"], + expected_draw.astype(np.float64), + ) + assert assigned.table("benunit")["uc_deduction_combination"].map(type).eq(str).all() + assert set(assigned.table("benunit")["uc_deduction_combination"]) <= { + "NONE", + *UC_DEDUCTION_COMBINATIONS, + } + + ids = original.table("benunit")["benunit_id"].to_numpy() + permuted_ids = ids[::-1] + original_draws = _identity_float32_uniforms(ids, seed=0, salt="permutation") + permuted_draws = _identity_float32_uniforms( + permuted_ids, seed=0, salt="permutation" + ) + assert dict(zip(ids, original_draws, strict=True)) == dict( + zip(permuted_ids, permuted_draws, strict=True) + ) + + +def test_stage_refuses_operation_drift_and_incoherent_resource() -> None: + stage_mapping = _stage_mapping() + stage_mapping["operations"][2]["unexpected"] = True + transform = UKUCDeductionAttributesStageTransform( + stage=SourceStageSpec.from_mapping(stage_mapping), + resource=load_uc_deduction_distributions(), + ) + with pytest.raises(ValueError, match="declaration drifted"): + transform(_frame(16)) + + resource = copy.deepcopy(load_uc_deduction_distributions()) + resource["type_combination"]["shares"][0]["share"] = -1 + with pytest.raises(ValueError, match="finite and positive"): + validate_uc_deduction_resource(resource) + + +def test_stage_receipt_passes_latent_attribute_health_gate() -> None: + transform = UKUCDeductionAttributesStageTransform( + stage=_stage(), resource=load_uc_deduction_distributions() + ) + transform(_frame(8192, region_names=("UNKNOWN",))) + evidence = transform.checkpoint_metadata()["evidence"] + + result = uk_stage_health_gate( + evidence=evidence, + stage="uc_deduction_attributes", + check="latent_attribute_realization", + parameters={}, + ) + + assert result.passed, result.failures + assert evidence["coherence_violation_count"] == 0 + + +@pytest.mark.requires_uk +def test_engine_golden_mirror_on_held_float32_draws() -> None: + policyengine_uk = pytest.importorskip("policyengine_uk") + resource = load_uc_deduction_distributions() + # Every real region factor is exercised against the engine: 1,024 units + # cycle through the twelve UK regions (~85 each). UNKNOWN is left to the + # hermetic mapping test: the engine cannot uprate rents for a household + # without a region (no private_rental_prices.UNKNOWN parameter), so it + # cannot host a simulation, while its factor of 1.0 is exercised in the + # stage's own mapping. + engine_regions = tuple(sorted(UC_DEDUCTION_REGIONS - {"UNKNOWN"})) + assigned = UKUCDeductionAttributesStageTransform(stage=_stage(), resource=resource)( + _frame(1024, region_names=engine_regions) + ) + assigned_benunit = assigned.table("benunit") + assert set( + assigned.table("household")["region"].map(lambda v: str(getattr(v, "name", v))) + ) == set(engine_regions) + fallback_benunit = assigned_benunit.drop( + columns=["uc_latent_deduction_rate", "uc_deduction_combination"] + ) + fallback = uk_national_frame( + person=assigned.table("person").copy(), + benunit=fallback_benunit, + household=assigned.table("household").copy(), + household_weights=assigned.weights_for("household").values, + time_period="2024", + ) + + from microcosm.frame.adapters.policyengine_uk import PolicyEngineUKEngine + + adapter = PolicyEngineUKEngine() + simulation = policyengine_uk.Microsimulation( + dataset=adapter._build_dataset(fallback, 2024) + ) + engine_has = np.asarray(simulation.calculate("uc_has_deduction", 2024)) + engine_rate = np.asarray(simulation.calculate("uc_latent_deduction_rate", 2024)) + engine_combination = ( + simulation.calculate("uc_deduction_combination", 2024) + .astype(str) + .to_numpy(dtype=object) + ) + + expected_rate = assigned_benunit["uc_latent_deduction_rate"].to_numpy() + np.testing.assert_array_equal(engine_has, expected_rate > 0.0) + np.testing.assert_allclose(engine_rate, expected_rate, atol=1e-6) + np.testing.assert_array_equal( + engine_combination, + assigned_benunit["uc_deduction_combination"].to_numpy(), + ) + + +def _engine_splitmix64_uniform(ids: np.ndarray, salt: int = 0) -> np.ndarray: + """PolicyEngine-UK's fallback draw (``utils/stochastic.py`` at 2.92.1), inlined.""" + + with np.errstate(over="ignore"): + z = ids.astype(np.uint64) + np.uint64(salt) * np.uint64(0x632BE59BD9B4E019) + z = z + np.uint64(0x9E3779B97F4A7C15) + z = (z ^ (z >> np.uint64(30))) * np.uint64(0xBF58476D1CE4E5B9) + z = (z ^ (z >> np.uint64(27))) * np.uint64(0x94D049BB133111EB) + z = z ^ (z >> np.uint64(31)) + draws = (z >> np.uint64(11)).astype(np.float64) / 2.0**53 + return np.minimum(draws, 1.0 - 2.0**-24) + + +def test_persisted_draws_are_not_the_engine_fallback_and_stay_exported() -> None: + """The engine reads the persisted draws; it does not recompute them. + + microcosm's draws are identity-keyed blake2b uniforms, the engine's fallback + is splitmix64 of ``benunit_id``: they agree on essentially no row, which is + fine only because the engine consumes the persisted columns. The design + therefore rests on the two draw columns staying on the export surface, so + that is pinned here alongside the divergence. + """ + + from microcosm.build.uk_runtime.terminal_gates import ( + UK_ALLOWED_EXTRA_EXPORT_COLUMNS, + ) + + ids = np.arange(1, 4097, dtype=np.int64) + ours = _identity_float32_uniforms(ids, seed=0, salt="uc_deduction_random_draw") + engine_fallback = _engine_splitmix64_uniform(ids, salt=0) + agreement = np.mean(np.isclose(ours, engine_fallback, atol=1e-6)) + assert agreement < 1e-3, agreement + + for column in ("uc_deduction_random_draw", "uc_deduction_type_random_draw"): + assert f"benunit.{column}" in UK_ALLOWED_EXTRA_EXPORT_COLUMNS + assert column in _stage().outputs diff --git a/packages/microcosm-build/tests/test_uk_was_wealth.py b/packages/microcosm-build/tests/test_uk_was_wealth.py index d1c29df9c..f15b3b43f 100644 --- a/packages/microcosm-build/tests/test_uk_was_wealth.py +++ b/packages/microcosm-build/tests/test_uk_was_wealth.py @@ -50,7 +50,12 @@ def _stage() -> SourceStageSpec: "grain": "household", "artifacts": [], "operations": [ - {"kind": "fit_weighted_qrf_chain", "seed": 0, "n_estimators": 2} + { + "kind": "fit_weighted_qrf_chain", + "seed": 0, + "n_estimators": 2, + "debt_segment_predictors": ["has_mortgage_tenure"], + } ], "outputs": list(UK_WAS_WEALTH_OUTPUT_COLUMNS), "nonnegative_outputs": [ @@ -86,7 +91,10 @@ def _raw_was() -> pd.DataFrame: "DVPriRntR8": [1, 2], "CTAmtR8": [1000.0, 1200.0], "HFINWNTR8_Sum": [-5.0, 50.0], + "HFINWNTR8_exSLC_Sum": [20.0, 40.0], "HFINWR8_SUM": [30.0, 40.0], + "HMortGR8": [1000.0, 0.0], + "Ten1R8": [2, 4], "DVhvalueR8": [100000.0, 200000.0], "DVHseValR8_sum": [1000.0, 2000.0], "DVBlDValR8_sum": [3000.0, 4000.0], @@ -137,6 +145,7 @@ def _frame() -> object: "council_tax": [1000.0, 1200.0], "household_net_income": [50000.0, 60000.0], "is_renting": [False, True], + "tenure_type": ["OWNED_WITH_MORTGAGE", "OWNED_OUTRIGHT"], } ) return uk_national_frame( @@ -158,6 +167,9 @@ def test_was_donor_cleaning_arithmetic_and_exact_case_insensitive_columns() -> N assert donor["corporate_wealth_excl_isa"].tolist() == [13.0, 16.0] assert donor["corporate_wealth"].tolist() == [18.0, 22.0] assert donor["student_loan_balance"].tolist() == [5000.0, 2000.0] + assert donor["mortgage_debt"].tolist() == [1000.0, 0.0] + assert donor["consumer_debt"].tolist() == [10.0, 0.0] + assert donor["has_mortgage_tenure"].tolist() == [True, False] assert donor["region"].tolist() == ["LONDON", "SCOTLAND"] assert donor["is_renting"].tolist() == [True, False] assert 3 not in REGIONS @@ -203,6 +215,7 @@ def test_recipient_predictors_sum_person_and_benunit_variables_to_household() -> assert predictors["num_children"].tolist() == [0.0, 0.0] assert predictors["household_net_income"].tolist() == [50000.0, 60000.0] assert predictors["is_renting"].tolist() == [False, True] + assert predictors["has_mortgage_tenure"].tolist() == [True, False] def test_recipient_predictors_fail_loud_on_entity_mismatch() -> None: @@ -375,7 +388,12 @@ def test_stage_transform_refuses_sha_mismatched_tab(tmp_path) -> None: } ], "operations": [ - {"kind": "fit_weighted_qrf_chain", "seed": 0, "n_estimators": 2} + { + "kind": "fit_weighted_qrf_chain", + "seed": 0, + "n_estimators": 2, + "debt_segment_predictors": ["has_mortgage_tenure"], + } ], "outputs": list(UK_WAS_WEALTH_OUTPUT_COLUMNS), } @@ -450,6 +468,12 @@ def fit_draw_next( assert "corporate_wealth" in calls[2][0] assert "private_pension_wealth" not in calls[1][0] assert calls[2][1][-1] == "cash_isa" + assert calls[3][1] == ("mortgage_debt", "consumer_debt") + assert calls[3][0] == ( + *calls[0][0], + "has_mortgage_tenure", + *UK_WAS_WEALTH_OUTPUT_COLUMNS[:-2], + ) fitted_targets = [name for _, targets in calls for name in targets] assert [record.fit_name for record in result.fit_weight_records] == [ f"uk_was_2018_20_wealth:{target}" for target in fitted_targets @@ -457,7 +481,7 @@ def fit_draw_next( assert {record.weight_kind for record in result.fit_weight_records} == {"explicit"} # One independent RNG root per segment, derived from the declared seed. assert seeds == list(module.was_wealth_segment_seeds(0)) - assert len(set(seeds)) == 3 + assert len(set(seeds)) == 4 assert result.segment_seeds == tuple(seeds) @@ -498,9 +522,68 @@ def test_segment_seeds_are_distinct_and_deterministic() -> None: # Golden pin: the production roots for the declared stage seed 0. Moving # them is a spec-visible RNG change, never an accident. - assert seeds == (3757552657, 673228719, 3241444873) - assert len(seeds) == 3 - assert len(set(seeds)) == 3 + assert seeds == (3757552657, 673228719, 3241444873, 3685993406) + assert len(seeds) == 4 + assert len(set(seeds)) == 4 + assert seeds[:3] == module.was_wealth_segment_seeds(0, segments=3) assert seeds == module.was_wealth_segment_seeds(0) assert seeds != module.was_wealth_segment_seeds(1) assert all(isinstance(seed, int) for seed in seeds) + + +def test_debt_segment_leaves_the_fourteen_e5_columns_byte_equal() -> None: + """The fourth chain segment cannot move the first three segments' draws. + + Runs the real chain on the hermetic H2 fixture donor twice, stopping after + the third segment and after the fourth, and asserts the fourteen E5 + columns are identical: the debt segment draws from the fourth child seed + and only appends columns. + """ + + from microcosm.build.uk_runtime.was_wealth import ( + UK_WAS_DEBT_OUTPUT_COLUMNS, + UK_WAS_WEALTH_OUTPUT_COLUMNS, + UK_WAS_WEALTH_PREDICTORS, + clean_was_household_table, + impute_was_wealth, + ) + from tools.graph_uk_spine_fixture import _was_donor + + donor = clean_was_household_table(_was_donor()) + rng = np.random.default_rng(685) + n = 48 + regions = np.resize( + np.asarray(["LONDON", "SCOTLAND", "WALES", "NORTH_EAST", "SOUTH_WEST"]), n + ) + recipient = pd.DataFrame( + { + "household_net_income": rng.uniform(5_000, 90_000, n), + "num_adults": rng.integers(1, 4, n), + "num_children": rng.integers(0, 3, n), + "private_pension_income": rng.uniform(0, 20_000, n), + "employment_income": rng.uniform(0, 60_000, n), + "self_employment_income": rng.uniform(0, 10_000, n), + "capital_income": rng.uniform(0, 5_000, n), + "num_bedrooms": rng.integers(1, 6, n), + "council_tax": rng.uniform(800, 3_000, n), + "is_renting": rng.random(n) < 0.35, + "region": regions, + "has_mortgage_tenure": rng.random(n) < 0.3, + } + ) + assert list(recipient.columns[:11]) == list(UK_WAS_WEALTH_PREDICTORS) + + three = impute_was_wealth(donor, recipient, seed=0, n_estimators=2, segments=3) + four = impute_was_wealth(donor, recipient, seed=0, n_estimators=2, segments=4) + + e5_columns = [ + column + for column in UK_WAS_WEALTH_OUTPUT_COLUMNS + if column not in UK_WAS_DEBT_OUTPUT_COLUMNS + ] + assert list(three.draws.columns) == e5_columns + assert list(four.draws.columns) == list(UK_WAS_WEALTH_OUTPUT_COLUMNS) + pd.testing.assert_frame_equal(three.draws, four.draws.loc[:, e5_columns]) + assert three.segment_seeds == four.segment_seeds + assert len(three.fit_weight_records) == len(e5_columns) + assert len(four.fit_weight_records) == len(UK_WAS_WEALTH_OUTPUT_COLUMNS) diff --git a/packages/microcosm-build/tests/test_uk_wealth_resources.py b/packages/microcosm-build/tests/test_uk_wealth_resources.py index 376b423bd..9df7ce6ea 100644 --- a/packages/microcosm-build/tests/test_uk_wealth_resources.py +++ b/packages/microcosm-build/tests/test_uk_wealth_resources.py @@ -123,7 +123,10 @@ def test_support_bounds_tool_rounds_synthetic_donor_outward(tmp_path: Path) -> N "DVPriRntR8": [1, 2], "CTAmtR8": [1000, 1200], "HFINWNTR8_Sum": [-5, 50], + "HFINWNTR8_exSLC_Sum": [20, 40], "HFINWR8_SUM": [30, 40], + "HMortGR8": [1000, 0], + "Ten1R8": [2, 4], "DVhvalueR8": [100000, 200000], "DVHseValR8_sum": [1000, 2000], "DVBlDValR8_sum": [3000, 4000], @@ -141,3 +144,5 @@ def test_support_bounds_tool_rounds_synthetic_donor_outward(tmp_path: Path) -> N assert payload["bounds"]["owned_land"] == [10.0, 100.0] assert payload["bounds"]["net_financial_wealth"] == [-5.0, 50.0] + assert payload["bounds"]["mortgage_debt"] == [0.0, 1000.0] + assert payload["bounds"]["consumer_debt"] == [0.0, 10.0] diff --git a/packages/microcosm-data/src/microcosm/data/contract.py b/packages/microcosm-data/src/microcosm/data/contract.py index 91376f846..c76164545 100644 --- a/packages/microcosm-data/src/microcosm/data/contract.py +++ b/packages/microcosm-data/src/microcosm/data/contract.py @@ -375,13 +375,13 @@ # fingerprint derives from the manifest digest. Editing the spec moves all # three here in the same reviewed change. _UK_GATE_BATTERY_POLICY_SHA256 = ( - "bcbcfd552424313b0843bc68ad64afea04c3b4146b01458cc9fdaf54b85aea82" + "9d4f1d5bdcf803bc718eeb27a3cae58730b3a7933eb8b417306c52234775651d" ) _UK_GATE_BATTERY_GATES_MANIFEST_SHA256 = ( - "76c861d48d48d73fd8f18f3d5cddac2855294c2839105ca2b6ac9a8a9d4b5f2f" + "a59a9308811298813fb5b92ae0c13ba50939c5e206b47a64e94d2ec5897087b3" ) _UK_GATE_BATTERY_SPEC_FINGERPRINT = ( - "dfbad2aa2930ef7e615742bd43bccd6cadaeb77ceb46f9161bb81b715dda20aa" + "306dabe4a310a8bf0a50f95e00c393e79e64fb723d6830328dd92a553044552e" ) #: Spec entry id -> the legacy gate name whose observable detail checks #: apply unchanged (the battery re-keys the report by entry id; the gate @@ -400,6 +400,7 @@ "uk_export_surface": "export_surface", "uk_take_up_signal": "take_up_signal", "uk_brma_enum_domain": "enum_domain", + "uk_uc_deduction_combination_enum_domain": "enum_domain", "uk_student_loan_plan_enum_domain": "enum_domain", "uk_target_surface": "target_surface", "uk_target_fit": "target_fit", @@ -423,6 +424,7 @@ "preflight", ), "uk_stage_was_wealth_support": ("stage_health", "transferred"), + "uk_stage_uc_deduction_attributes": ("stage_health", "transferred"), "uk_stage_lcfs_consumption_support": ("stage_health", "transferred"), "uk_stage_etb_vat_support": ("stage_health", "transferred"), "uk_stage_etb_services_support": ("stage_health", "transferred"), @@ -465,6 +467,7 @@ "uk_export_surface": ("export_surface", "terminal"), "uk_take_up_signal": ("take_up_signal", "terminal"), "uk_brma_enum_domain": ("enum_domain", "assembled"), + "uk_uc_deduction_combination_enum_domain": ("enum_domain", "terminal"), "uk_student_loan_plan_enum_domain": ("enum_domain", "terminal"), "uk_calibration_reference_coverage": ( "calibration_reference_coverage", @@ -506,6 +509,7 @@ "uk_degenerate_release_surface", "uk_input_mass_parity", "uk_stage_was_wealth_support", + "uk_stage_uc_deduction_attributes", "uk_stage_lcfs_consumption_support", "uk_stage_etb_vat_support", "uk_stage_etb_services_support", @@ -603,6 +607,7 @@ "uk_stage_salary_sacrifice_realization", "uk_stage_spi_support_channel_mass", "uk_stage_student_loans_realization", + "uk_stage_uc_deduction_attributes", "uk_stage_was_wealth_support", } ), @@ -628,6 +633,7 @@ "uk_ledger_compile_parity_production_2023", "uk_nonnegative_columns", "uk_uc_capital_coherence", + "uk_uc_deduction_combination_enum_domain", "uk_qrf_tail_concentration", "uk_release_family_build_stages", "uk_release_input_coverage", @@ -643,10 +649,10 @@ _UK_CERTIFICATION_PART_DIGESTS: Mapping[str, Mapping[str, str]] = { "spine": { "gates_manifest_sha256": ( - "1605cf3fe1be4983cfb4ed806a34d69375cdc3e4e0c8883cc49481ac5870399a" + "0f1d1102611c5115d403f0c5a6ac823adba207f20b47adbd40d2b226e8b73477" ), "policy_sha256": ( - "3d14ad24eff7f5afd343164560db24095d27fafb36c619ddf725c32e00b35a69" + "861e665cdd6c6ad8bd599483139df3fe36df5a011c840f601a800a2fc0990355" ), }, "calibration_seam": { @@ -659,10 +665,10 @@ }, "release_cut": { "gates_manifest_sha256": ( - "97f25fa3cf48b8828450831be7c986704740eddb35ed08a27da950e8dc412b64" + "57311d9aedbf7ba9ebcbe9721a386002f5d27ed77b10a589e3466f2265c3f1ef" ), "policy_sha256": ( - "b72d6c6289e71e0e59556aea707676847b602a230e83f2d96bfd1fd4a9e86883" + "6d6638d38e999e8a8aec8fd72083a31a369dc5be1a1af57569c41e8b423ad39c" ), }, } diff --git a/packages/microcosm-data/tests/test_contract.py b/packages/microcosm-data/tests/test_contract.py index 9c536c332..b8f64c54a 100644 --- a/packages/microcosm-data/tests/test_contract.py +++ b/packages/microcosm-data/tests/test_contract.py @@ -137,13 +137,13 @@ def _trusted_terminal_gate_signing_key(monkeypatch) -> None: UK_GATE_BATTERY_PRODUCER = "microcosm.build.gate_battery" UK_GATE_BATTERY_SIGNING_KEY_ENV = "MICROCOSM_UK_TERMINAL_GATE_SIGNING_KEY" UK_GATE_BATTERY_POLICY_SHA256 = ( - "bcbcfd552424313b0843bc68ad64afea04c3b4146b01458cc9fdaf54b85aea82" + "9d4f1d5bdcf803bc718eeb27a3cae58730b3a7933eb8b417306c52234775651d" ) UK_GATE_BATTERY_GATES_MANIFEST_SHA256 = ( - "76c861d48d48d73fd8f18f3d5cddac2855294c2839105ca2b6ac9a8a9d4b5f2f" + "a59a9308811298813fb5b92ae0c13ba50939c5e206b47a64e94d2ec5897087b3" ) UK_GATE_BATTERY_SPEC_FINGERPRINT = ( - "dfbad2aa2930ef7e615742bd43bccd6cadaeb77ceb46f9161bb81b715dda20aa" + "306dabe4a310a8bf0a50f95e00c393e79e64fb723d6830328dd92a553044552e" ) UK_GATE_BATTERY_DEGENERATE_EVIDENCE_SHA256 = ( "d0d024043132fa07c378c393dbe2b24fe99bf19e876bcc39997d2c80cc9bd4f6" @@ -170,6 +170,11 @@ def _trusted_terminal_gate_signing_key(monkeypatch) -> None: None, ), "uk_stage_was_wealth_support": ("stage_health", "transferred", None), + "uk_stage_uc_deduction_attributes": ( + "stage_health", + "transferred", + None, + ), "uk_stage_lcfs_consumption_support": ("stage_health", "transferred", None), "uk_stage_etb_vat_support": ("stage_health", "transferred", None), "uk_stage_etb_services_support": ("stage_health", "transferred", None), @@ -253,6 +258,11 @@ def _trusted_terminal_gate_signing_key(monkeypatch) -> None: "uk_export_surface": ("export_surface", "terminal", "export_surface"), "uk_take_up_signal": ("take_up_signal", "terminal", "take_up_signal"), "uk_brma_enum_domain": ("enum_domain", "assembled", "enum_domain"), + "uk_uc_deduction_combination_enum_domain": ( + "enum_domain", + "terminal", + "enum_domain", + ), "uk_student_loan_plan_enum_domain": ( "enum_domain", "terminal", @@ -1150,6 +1160,8 @@ def _gate_battery_payload( "frs_hmrc_spine_leaves", "spi_support_channel", "hmrc_spi_income_spine", + "uc_capital_coherence", + "uc_deduction_attributes", "cgt_incidence_clone", "cgt_band_donors", "hmrc_cgt_gains_spine", @@ -1159,6 +1171,7 @@ def _gate_battery_payload( ] stage_health_stages = { "uk_stage_was_wealth_support": "was_wealth", + "uk_stage_uc_deduction_attributes": "uc_deduction_attributes", "uk_stage_lcfs_consumption_support": "lcfs_consumption", "uk_stage_etb_vat_support": "etb_vat", "uk_stage_etb_services_support": "etb_services", diff --git a/packages/microcosm-graph/tests/fixtures/parity/uk_spine/NORMALIZATION.md b/packages/microcosm-graph/tests/fixtures/parity/uk_spine/NORMALIZATION.md index 51654c78c..fa46b4edf 100644 --- a/packages/microcosm-graph/tests/fixtures/parity/uk_spine/NORMALIZATION.md +++ b/packages/microcosm-graph/tests/fixtures/parity/uk_spine/NORMALIZATION.md @@ -1,6 +1,6 @@ # UK spine parity string normalization -The unchanged legacy transforms retain these 21 textual table columns as +The unchanged legacy transforms retain these 22 textual table columns as pandas `object`. The frozen graph dtype token `string` is specified by interface-freeze amendment 10 as pandas `StringDtype(storage="python")`. Before computing the legacy oracle's `uk_frame_content_identity` (live, @@ -9,5 +9,5 @@ dtype. No values, row order, column order, weights, strata, mass records, or metadata change. - `person`: `gender`, `marital_status`, `employment_status`, `employment_sector`, `aa_category`, `dla_sc_category`, `dla_m_category`, `pip_m_category`, `pip_dl_category`, `current_education`, `highest_education`, `person_support_channel`, `student_loan_plan` -- `benunit`: `benunit_support_channel` +- `benunit`: `benunit_support_channel`, `uc_deduction_combination` - `household`: `region`, `tenure_type`, `accommodation_type`, `council_tax_band`, `brma`, `household_support_channel`, `source_household_key` diff --git a/packages/microcosm-graph/tests/fixtures/parity/uk_spine/PRODUCED_BY.txt b/packages/microcosm-graph/tests/fixtures/parity/uk_spine/PRODUCED_BY.txt index 51b1f58b9..b036ca7d1 100644 --- a/packages/microcosm-graph/tests/fixtures/parity/uk_spine/PRODUCED_BY.txt +++ b/packages/microcosm-graph/tests/fixtures/parity/uk_spine/PRODUCED_BY.txt @@ -1 +1 @@ -tools/graph_uk_spine_fixture.py; current 26-transform legacy StagePlan oracle with parsed private-source seams. The acceptance test runs both sides from frs_raw in-process (root weights differ by one ulp between machines); the captured root tables serve the unbound registry's data-only CREATE path. +tools/graph_uk_spine_fixture.py; current 28-transform legacy StagePlan oracle with parsed private-source seams. The acceptance test runs both sides from frs_raw in-process (root weights differ by one ulp between machines); the captured root tables serve the unbound registry's data-only CREATE path. diff --git a/packages/microcosm-graph/tests/fixtures/parity/uk_spine/sources/fixture.json b/packages/microcosm-graph/tests/fixtures/parity/uk_spine/sources/fixture.json index 6688f27b8..a3ead883b 100644 --- a/packages/microcosm-graph/tests/fixtures/parity/uk_spine/sources/fixture.json +++ b/packages/microcosm-graph/tests/fixtures/parity/uk_spine/sources/fixture.json @@ -13,7 +13,7 @@ "spi_sample_fraction": 0.0135, "student_loans_calibration_year": 2025 }, - "description": "Data-only inputs for reconstructing the same 26 current UK stage transform classes used by the legacy StagePlan oracle.", + "description": "Data-only inputs for reconstructing the same 28 current UK stage transform classes used by the legacy StagePlan oracle.", "inputs": { "cgt_distribution": "cgt_distribution.json", "etb": "etb.csv", @@ -2686,6 +2686,72 @@ "stage": "uc_capital_coherence", "survey": "Family Resources Survey 2024-25" }, + "uc_deduction_attributes": { + "artifacts": [ + { + "format": "json", + "kind": "public_aggregate_reference", + "resource": "dwp_uc_deduction_distributions.json", + "role": "deduction_distribution" + } + ], + "grain": "benunit", + "nonnegative_outputs": [ + "uc_deduction_random_draw", + "uc_deduction_type_random_draw", + "uc_latent_deduction_rate" + ], + "notes": "Assigns region-conditioned latent deduction demand to every benefit unit from identity-keyed seed-0 draws, rounded and clamped to the PolicyEngine-UK float32 input contract before applying the engine's exact band and normalized-combination mappings. The attributes are award-independent; law and claimant gating remain in the engine. Clone and donor-stack stages copy the completed attributes byte-for-byte. The persisted draws are microcosm's identity-keyed blake2b uniforms, not PolicyEngine-UK's splitmix64 fallback hash; the engine reads the persisted columns and derives the same flag, rate and combination from them, so the two draw columns must stay on the export surface (allow-listed and test-pinned).", + "operations": [ + { + "kind": "assign_uniform_draw", + "output": "uc_deduction_random_draw", + "seed": 0 + }, + { + "kind": "assign_uniform_draw", + "output": "uc_deduction_type_random_draw", + "seed": 0 + }, + { + "distribution": "latent_rate_distribution", + "draw": "uc_deduction_random_draw", + "draw_dtype": "float32", + "incidence_modifier": { + "column": "region", + "entity": "household", + "table": "region_incidence_factor" + }, + "kind": "map_uniform_to_banded_rate", + "none_value": 0, + "output": "uc_latent_deduction_rate", + "resource": "dwp_uc_deduction_distributions.json" + }, + { + "distribution": "type_combination", + "draw": "uc_deduction_type_random_draw", + "draw_dtype": "float32", + "gate": { + "column": "uc_latent_deduction_rate", + "positive": true + }, + "kind": "map_uniform_to_categorical", + "none_value": "NONE", + "output": "uc_deduction_combination", + "resource": "dwp_uc_deduction_distributions.json" + } + ], + "outputs": [ + "uc_deduction_random_draw", + "uc_deduction_type_random_draw", + "uc_latent_deduction_rate", + "uc_deduction_combination" + ], + "rewrites": [], + "source": "Department for Work and Pensions supplementary Tables 2, 5 and 6, sha-pinned by Chronicle package dwp-uc-deductions-march-2025-february-2026.", + "stage": "uc_deduction_attributes", + "survey": "DWP Universal Credit deductions statistics, March 2025 to February 2026" + }, "uc_reporter_redraw": { "artifacts": [], "grain": "benunit", @@ -2785,14 +2851,18 @@ "num_vehicles", "cash_isa", "stocks_and_shares_isa", - "student_loan_balance" + "student_loan_balance", + "mortgage_debt", + "consumer_debt" ], - "notes": "Ports incumbent WAS round-8 wealth imputation with signed E5 differences: exact lower-case column matching replaces the fuzzy r/w fallback; cash ISA uses DVCISAVR8_aggr and stocks-and-shares ISA uses DVIISAVR8_aggr; private pension wealth (WAS total private pension wealth less the current defined-benefit component; DB-type components at the SCAPE rate, money-purchase components at reported fund values) is drawn as its own output private_pension_wealth and no longer folded into corporate_wealth, which keeps only the share-like holdings - pension rights are disregarded capital in every means test (UC Regs 2013 Sch 10 para 10 and the HB/JSA/ESA/IS/SPC parallels; uk-data#452) - and the downstream chain segment conditions on both components; each chain segment starts its own fit and draw RNG streams from a child seed derived deterministically from the declared seed (start_chain respawns from the model seed on every call, so one shared model coupled the k-th target of every segment); corporate_wealth folds stocks-and-shares ISA after drawing corporate_wealth_excl_isa; recipient Northern Ireland regions are mapped to Wales for prediction only; student_loan_balance is allocated by household id rather than the incumbent positional off-by-one. Engine predictors materialize at their native entity and person/benunit values are summed to household, reproducing the incumbent map_to=household semantics; region is one-hot encoded jointly across donor and recipient (the incumbent's dummy encoding), with unmapped donor GOR codes becoming all-zero dummy rows. The WAS and FRS predictor definitions are not fully like-for-like and are ported as-is; raw WAS missing values are blanket-filled with zero; UKDS negative sentinel codes (-9/-8/-7/-6) are recoded to zero for the nonnegative-domain columns the licensed audit found carrying them (vcarnr8: 2 rows; HBedRmR8: 95.8 percent - the bedrooms question is effectively unasked in the WAS household file, predictor-quality revisit registered on microcosm#145) - a signed difference vs the incumbent, which trains on raw sentinels; DVPriRntR8's -9 is structural not-applicable so the is_renting mapping is unchanged; genuinely negative domains are never recoded.", + "notes": "Ports incumbent WAS round-8 wealth imputation with signed E5 differences: exact lower-case column matching replaces the fuzzy r/w fallback; cash ISA uses DVCISAVR8_aggr and stocks-and-shares ISA uses DVIISAVR8_aggr; private pension wealth (WAS total private pension wealth less the current defined-benefit component; DB-type components at the SCAPE rate, money-purchase components at reported fund values) is drawn as its own output private_pension_wealth and no longer folded into corporate_wealth, which keeps only the share-like holdings - pension rights are disregarded capital in every means test (UC Regs 2013 Sch 10 para 10 and the HB/JSA/ESA/IS/SPC parallels; uk-data#452) - and the downstream chain segment conditions on both components; each chain segment starts its own fit and draw RNG streams from a child seed derived deterministically from the declared seed (start_chain respawns from the model seed on every call, so one shared model coupled the k-th target of every segment); corporate_wealth folds stocks-and-shares ISA after drawing corporate_wealth_excl_isa; recipient Northern Ireland regions are mapped to Wales for prediction only; student_loan_balance is allocated by household id rather than the incumbent positional off-by-one. The debt segment (fourth chain segment, fourth child seed) reads WAS round-8 derived variables HMortGR8 (total outstanding mortgage debt), HFINWNTR8_exSLC_Sum (net financial wealth excluding student loans; gross less it is the non-student-loan financial liabilities) and Ten1R8 (tenure; codes 2 and 3 are buying with a mortgage and part own/part rent, confirmed by the I1 audit of the pinned tab); UKDS SN 7215 documentation is the codebook. Engine predictors materialize at their native entity and person/benunit values are summed to household, reproducing the incumbent map_to=household semantics; region is one-hot encoded jointly across donor and recipient (the incumbent's dummy encoding), with unmapped donor GOR codes becoming all-zero dummy rows. The WAS and FRS predictor definitions are not fully like-for-like and are ported as-is; raw WAS missing values are blanket-filled with zero; UKDS negative sentinel codes (-9/-8/-7/-6) are recoded to zero for the nonnegative-domain columns the licensed audit found carrying them (vcarnr8: 2 rows; HBedRmR8: 95.8 percent - the bedrooms question is effectively unasked in the WAS household file, predictor-quality revisit registered on microcosm#145) - a signed difference vs the incumbent, which trains on raw sentinels; DVPriRntR8's -9 is structural not-applicable so the is_renting mapping is unchanged; genuinely negative domains are never recoded.", "operations": [ { "derived": { "cash_isa": "DVCISAVR8_aggr", + "consumer_debt": "max(HFINWR8_SUM - HFINWNTR8_exSLC_Sum, 0)", "corporate_wealth_excl_isa": "DVFESHARESR8_aggr + DVFShUKVR8_aggr + DVFCollVR8_aggr", + "has_mortgage_tenure": "Ten1R8 in {2, 3}", "is_renting": "DVPriRntR8 == 1", "private_pension_wealth": "totalpenr8_aggr - dvvaldbt_scaper8_aggr", "region": "GORR8 via incumbent REGIONS map", @@ -2802,13 +2872,16 @@ "direct_maps": { "gross_financial_wealth": "HFINWR8_SUM", "main_residence_value": "DVhvalueR8", + "mortgage_debt": "HMortGR8", "net_financial_wealth": "HFINWNTR8_Sum", + "net_financial_wealth_exsl": "HFINWNTR8_exSLC_Sum", "non_residential_property_value": "DVBlDValR8_sum", "num_vehicles": "vcarnr8", "other_residential_property_value": "DVHseValR8_sum", "owned_land": "DVLUKValR8_sum", "property_wealth": "DVPropertyR8", "savings": "DVSaValR8_aggr", + "tenure_code": "Ten1R8", "weight": "R8xshhwgt" }, "fillna": 0, @@ -2849,7 +2922,12 @@ "savings", "num_vehicles", "student_loan_balance", - "cash_isa" + "cash_isa", + "mortgage_debt", + "consumer_debt" + ], + "debt_segment_predictors": [ + "has_mortgage_tenure" ], "integer_outputs": [ "num_vehicles" @@ -2919,7 +2997,9 @@ "num_vehicles", "cash_isa", "stocks_and_shares_isa", - "student_loan_balance" + "student_loan_balance", + "mortgage_debt", + "consumer_debt" ], "rewrites": [], "source": "Office for National Statistics Wealth and Assets Survey, UK Data Service SN 7215, DOI 10.5255/UKDA-SN-7215-20; local licensed 2006-22 household tab.", diff --git a/packages/microcosm-graph/tests/fixtures/parity/uk_spine/sources/was.csv b/packages/microcosm-graph/tests/fixtures/parity/uk_spine/sources/was.csv index 0f8addc83..10b0cbdbd 100644 --- a/packages/microcosm-graph/tests/fixtures/parity/uk_spine/sources/was.csv +++ b/packages/microcosm-graph/tests/fixtures/parity/uk_spine/sources/was.csv @@ -1,65 +1,65 @@ -R8xshhwgt,DVLUKValR8_sum,DVPropertyR8,DVFESHARESR8_aggr,DVFShUKVR8_aggr,DVIISAVR8_aggR,DVCISAVR8_aggr,DVFCollVR8_aggr,totalpenr8_aggr,dvvaldbt_scaper8_aggr,NumAdultR8,NumCh18R8,DVGIPPENR8_AGGR,DVGISER8_AGGR,DVGIINVR8_aggr,DVGIEMPR8_AGGR,HBedRmR8,GORR8,DVPriRntR8,CTAmtR8,HFINWNTR8_Sum,HFINWR8_SUM,DVhvalueR8,DVHseValR8_sum,DVBlDValR8_sum,DVTotinc_bhcR8,DVSaValR8_aggr,vcarnr8,Tot_LosR8_aggr,Tot_los_exc_SLCR8_aggr -1,10,20000,1,3,5,7,9,100,40,1,0,11,13,15,17,1,8,1,900,-50,30,100000,1000,3000,20000,500,0,9000,4000 -1.1000000000000001,11,20500,2,4,6,8,10,110,41,2,1,12,14,16,18,2,11,2,910,-46,31,102000,1050,3060,21000,520,1,9010,4005 -1.2,12,21000,3,5,7,9,11,120,42,3,2,13,15,17,19,3,12,1,920,-42,32,104000,1100,3120,22000,540,2,9020,4010 -1.3,13,21500,4,6,8,10,12,130,43,1,0,14,16,18,20,4,1,2,930,-38,33,106000,1150,3180,23000,560,3,9030,4015 -1.3999999999999999,14,22000,5,7,9,11,13,140,44,2,1,15,17,19,21,5,8,1,940,-34,34,108000,1200,3240,24000,580,0,9040,4020 -1.5,15,22500,6,8,10,12,14,150,45,3,2,16,18,20,22,1,11,2,950,-30,35,110000,1250,3300,25000,600,1,9050,4025 -1.6000000000000001,16,23000,7,9,11,13,15,160,46,1,0,17,19,21,23,2,12,1,960,-26,36,112000,1300,3360,26000,620,2,9060,4030 -1,17,23500,8,10,12,14,16,170,47,2,1,18,20,22,24,3,1,2,970,-22,37,114000,1350,3420,27000,640,3,9070,4035 -1.1000000000000001,18,24000,9,11,13,15,17,180,48,3,2,19,21,23,25,4,8,1,980,-18,38,116000,1400,3480,28000,660,0,9080,4040 -1.2,19,24500,10,12,14,16,18,190,49,1,0,20,22,24,26,5,11,2,990,-14,39,118000,1450,3540,29000,680,1,9090,4045 -1.3,20,25000,11,13,15,17,19,200,50,2,1,21,23,25,27,1,12,1,1000,-10,40,120000,1500,3600,30000,700,2,9100,4050 -1.3999999999999999,21,25500,12,14,16,18,20,210,51,3,2,22,24,26,28,2,1,2,1010,-6,41,122000,1550,3660,31000,720,3,9110,4055 -1.5,22,26000,13,15,17,19,21,220,52,1,0,23,25,27,29,3,8,1,1020,-2,42,124000,1600,3720,32000,740,0,9120,4060 -1.6000000000000001,23,26500,14,16,18,20,22,230,53,2,1,24,26,28,30,4,11,2,1030,2,43,126000,1650,3780,33000,760,1,9130,4065 -1,24,27000,15,17,19,21,23,240,54,3,2,25,27,29,31,5,12,1,1040,6,44,128000,1700,3840,34000,780,2,9140,4070 -1.1000000000000001,25,27500,16,18,20,22,24,250,55,1,0,26,28,30,32,1,1,2,1050,10,45,130000,1750,3900,35000,800,3,9150,4075 -1.2,26,28000,17,19,21,23,25,260,56,2,1,27,29,31,33,2,8,1,1060,14,46,132000,1800,3960,36000,820,0,9160,4080 -1.3,27,28500,18,20,22,24,26,270,57,3,2,28,30,32,34,3,11,2,1070,18,47,134000,1850,4020,37000,840,1,9170,4085 -1.3999999999999999,28,29000,19,21,23,25,27,280,58,1,0,29,31,33,35,4,12,1,1080,22,48,136000,1900,4080,38000,860,2,9180,4090 -1.5,29,29500,20,22,24,26,28,290,59,2,1,30,32,34,36,5,1,2,1090,26,49,138000,1950,4140,39000,880,3,9190,4095 -1.6000000000000001,30,30000,21,23,25,27,29,300,60,3,2,31,33,35,37,1,8,1,1100,30,50,140000,2000,4200,40000,900,0,9200,4100 -1,31,30500,22,24,26,28,30,310,61,1,0,32,34,36,38,2,11,2,1110,34,51,142000,2050,4260,41000,920,1,9210,4105 -1.1000000000000001,32,31000,23,25,27,29,31,320,62,2,1,33,35,37,39,3,12,1,1120,38,52,144000,2100,4320,42000,940,2,9220,4110 -1.2,33,31500,24,26,28,30,32,330,63,3,2,34,36,38,40,4,1,2,1130,42,53,146000,2150,4380,43000,960,3,9230,4115 -1.3,34,32000,25,27,29,31,33,340,64,1,0,35,37,39,41,5,8,1,1140,46,54,148000,2200,4440,44000,980,0,9240,4120 -1.3999999999999999,35,32500,26,28,30,32,34,350,65,2,1,36,38,40,42,1,11,2,1150,50,55,150000,2250,4500,45000,1000,1,9250,4125 -1.5,36,33000,27,29,31,33,35,360,66,3,2,37,39,41,43,2,12,1,1160,54,56,152000,2300,4560,46000,1020,2,9260,4130 -1.6000000000000001,37,33500,28,30,32,34,36,370,67,1,0,38,40,42,44,3,1,2,1170,58,57,154000,2350,4620,47000,1040,3,9270,4135 -1,38,34000,29,31,33,35,37,380,68,2,1,39,41,43,45,4,8,1,1180,62,58,156000,2400,4680,48000,1060,0,9280,4140 -1.1000000000000001,39,34500,30,32,34,36,38,390,69,3,2,40,42,44,46,5,11,2,1190,66,59,158000,2450,4740,49000,1080,1,9290,4145 -1.2,40,35000,31,33,35,37,39,400,70,1,0,41,43,45,47,1,12,1,1200,70,60,160000,2500,4800,50000,1100,2,9300,4150 -1.3,41,35500,32,34,36,38,40,410,71,2,1,42,44,46,48,2,1,2,1210,74,61,162000,2550,4860,51000,1120,3,9310,4155 -1.3999999999999999,42,36000,33,35,37,39,41,420,72,3,2,43,45,47,49,3,8,1,1220,78,62,164000,2600,4920,52000,1140,0,9320,4160 -1.5,43,36500,34,36,38,40,42,430,73,1,0,44,46,48,50,4,11,2,1230,82,63,166000,2650,4980,53000,1160,1,9330,4165 -1.6000000000000001,44,37000,35,37,39,41,43,440,74,2,1,45,47,49,51,5,12,1,1240,86,64,168000,2700,5040,54000,1180,2,9340,4170 -1,45,37500,36,38,40,42,44,450,75,3,2,46,48,50,52,1,1,2,1250,90,65,170000,2750,5100,55000,1200,3,9350,4175 -1.1000000000000001,46,38000,37,39,41,43,45,460,76,1,0,47,49,51,53,2,8,1,1260,94,66,172000,2800,5160,56000,1220,0,9360,4180 -1.2,47,38500,38,40,42,44,46,470,77,2,1,48,50,52,54,3,11,2,1270,98,67,174000,2850,5220,57000,1240,1,9370,4185 -1.3,48,39000,39,41,43,45,47,480,78,3,2,49,51,53,55,4,12,1,1280,102,68,176000,2900,5280,58000,1260,2,9380,4190 -1.3999999999999999,49,39500,40,42,44,46,48,490,79,1,0,50,52,54,56,5,1,2,1290,106,69,178000,2950,5340,59000,1280,3,9390,4195 -1.5,50,40000,41,43,45,47,49,500,80,2,1,51,53,55,57,1,8,1,1300,110,70,180000,3000,5400,60000,1300,0,9400,4200 -1.6000000000000001,51,40500,42,44,46,48,50,510,81,3,2,52,54,56,58,2,11,2,1310,114,71,182000,3050,5460,61000,1320,1,9410,4205 -1,52,41000,43,45,47,49,51,520,82,1,0,53,55,57,59,3,12,1,1320,118,72,184000,3100,5520,62000,1340,2,9420,4210 -1.1000000000000001,53,41500,44,46,48,50,52,530,83,2,1,54,56,58,60,4,1,2,1330,122,73,186000,3150,5580,63000,1360,3,9430,4215 -1.2,54,42000,45,47,49,51,53,540,84,3,2,55,57,59,61,5,8,1,1340,126,74,188000,3200,5640,64000,1380,0,9440,4220 -1.3,55,42500,46,48,50,52,54,550,85,1,0,56,58,60,62,1,11,2,1350,130,75,190000,3250,5700,65000,1400,1,9450,4225 -1.3999999999999999,56,43000,47,49,51,53,55,560,86,2,1,57,59,61,63,2,12,1,1360,134,76,192000,3300,5760,66000,1420,2,9460,4230 -1.5,57,43500,48,50,52,54,56,570,87,3,2,58,60,62,64,3,1,2,1370,138,77,194000,3350,5820,67000,1440,3,9470,4235 -1.6000000000000001,58,44000,49,51,53,55,57,580,88,1,0,59,61,63,65,4,8,1,1380,142,78,196000,3400,5880,68000,1460,0,9480,4240 -1,59,44500,50,52,54,56,58,590,89,2,1,60,62,64,66,5,11,2,1390,146,79,198000,3450,5940,69000,1480,1,9490,4245 -1.1000000000000001,60,45000,51,53,55,57,59,600,90,3,2,61,63,65,67,1,12,1,1400,150,80,200000,3500,6000,70000,1500,2,9500,4250 -1.2,61,45500,52,54,56,58,60,610,91,1,0,62,64,66,68,2,1,2,1410,154,81,202000,3550,6060,71000,1520,3,9510,4255 -1.3,62,46000,53,55,57,59,61,620,92,2,1,63,65,67,69,3,8,1,1420,158,82,204000,3600,6120,72000,1540,0,9520,4260 -1.3999999999999999,63,46500,54,56,58,60,62,630,93,3,2,64,66,68,70,4,11,2,1430,162,83,206000,3650,6180,73000,1560,1,9530,4265 -1.5,64,47000,55,57,59,61,63,640,94,1,0,65,67,69,71,5,12,1,1440,166,84,208000,3700,6240,74000,1580,2,9540,4270 -1.6000000000000001,65,47500,56,58,60,62,64,650,95,2,1,66,68,70,72,1,1,2,1450,170,85,210000,3750,6300,75000,1600,3,9550,4275 -1,66,48000,57,59,61,63,65,660,96,3,2,67,69,71,73,2,8,1,1460,174,86,212000,3800,6360,76000,1620,0,9560,4280 -1.1000000000000001,67,48500,58,60,62,64,66,670,97,1,0,68,70,72,74,3,11,2,1470,178,87,214000,3850,6420,77000,1640,1,9570,4285 -1.2,68,49000,59,61,63,65,67,680,98,2,1,69,71,73,75,4,12,1,1480,182,88,216000,3900,6480,78000,1660,2,9580,4290 -1.3,69,49500,60,62,64,66,68,690,99,3,2,70,72,74,76,5,1,2,1490,186,89,218000,3950,6540,79000,1680,3,9590,4295 -1.3999999999999999,70,50000,61,63,65,67,69,700,100,1,0,71,73,75,77,1,8,1,1500,190,90,220000,4000,6600,80000,1700,0,9600,4300 -1.5,71,50500,62,64,66,68,70,710,101,2,1,72,74,76,78,2,11,2,1510,194,91,222000,4050,6660,81000,1720,1,9610,4305 -1.6000000000000001,72,51000,63,65,67,69,71,720,102,3,2,73,75,77,79,3,12,1,1520,198,92,224000,4100,6720,82000,1740,2,9620,4310 -1,73,51500,64,66,68,70,72,730,103,1,0,74,76,78,80,4,1,2,1530,202,93,226000,4150,6780,83000,1760,3,9630,4315 +R8xshhwgt,DVLUKValR8_sum,DVPropertyR8,DVFESHARESR8_aggr,DVFShUKVR8_aggr,DVIISAVR8_aggR,DVCISAVR8_aggr,DVFCollVR8_aggr,totalpenr8_aggr,dvvaldbt_scaper8_aggr,NumAdultR8,NumCh18R8,DVGIPPENR8_AGGR,DVGISER8_AGGR,DVGIINVR8_aggr,DVGIEMPR8_AGGR,HBedRmR8,GORR8,DVPriRntR8,CTAmtR8,HFINWNTR8_Sum,HFINWNTR8_exSLC_Sum,HMortGR8,Ten1R8,HFINWR8_SUM,DVhvalueR8,DVHseValR8_sum,DVBlDValR8_sum,DVTotinc_bhcR8,DVSaValR8_aggr,vcarnr8,Tot_LosR8_aggr,Tot_los_exc_SLCR8_aggr +1,10,20000,1,3,5,7,9,100,40,1,0,11,13,15,17,1,8,1,900,-50,20,0,1,30,100000,1000,3000,20000,500,0,9000,4000 +1.1000000000000001,11,20500,2,4,6,8,10,110,41,2,1,12,14,16,18,2,11,2,910,-46,20,10100,2,31,102000,1050,3060,21000,520,1,9010,4005 +1.2,12,21000,3,5,7,9,11,120,42,3,2,13,15,17,19,3,12,1,920,-42,20,10200,3,32,104000,1100,3120,22000,540,2,9020,4010 +1.3,13,21500,4,6,8,10,12,130,43,1,0,14,16,18,20,4,1,2,930,-38,20,0,4,33,106000,1150,3180,23000,560,3,9030,4015 +1.3999999999999999,14,22000,5,7,9,11,13,140,44,2,1,15,17,19,21,5,8,1,940,-34,20,0,1,34,108000,1200,3240,24000,580,0,9040,4020 +1.5,15,22500,6,8,10,12,14,150,45,3,2,16,18,20,22,1,11,2,950,-30,25,10500,2,35,110000,1250,3300,25000,600,1,9050,4025 +1.6000000000000001,16,23000,7,9,11,13,15,160,46,1,0,17,19,21,23,2,12,1,960,-26,25,10600,3,36,112000,1300,3360,26000,620,2,9060,4030 +1,17,23500,8,10,12,14,16,170,47,2,1,18,20,22,24,3,1,2,970,-22,25,0,4,37,114000,1350,3420,27000,640,3,9070,4035 +1.1000000000000001,18,24000,9,11,13,15,17,180,48,3,2,19,21,23,25,4,8,1,980,-18,25,0,1,38,116000,1400,3480,28000,660,0,9080,4040 +1.2,19,24500,10,12,14,16,18,190,49,1,0,20,22,24,26,5,11,2,990,-14,25,10900,2,39,118000,1450,3540,29000,680,1,9090,4045 +1.3,20,25000,11,13,15,17,19,200,50,2,1,21,23,25,27,1,12,1,1000,-10,30,11000,3,40,120000,1500,3600,30000,700,2,9100,4050 +1.3999999999999999,21,25500,12,14,16,18,20,210,51,3,2,22,24,26,28,2,1,2,1010,-6,30,0,4,41,122000,1550,3660,31000,720,3,9110,4055 +1.5,22,26000,13,15,17,19,21,220,52,1,0,23,25,27,29,3,8,1,1020,-2,30,0,1,42,124000,1600,3720,32000,740,0,9120,4060 +1.6000000000000001,23,26500,14,16,18,20,22,230,53,2,1,24,26,28,30,4,11,2,1030,2,30,11300,2,43,126000,1650,3780,33000,760,1,9130,4065 +1,24,27000,15,17,19,21,23,240,54,3,2,25,27,29,31,5,12,1,1040,6,30,11400,3,44,128000,1700,3840,34000,780,2,9140,4070 +1.1000000000000001,25,27500,16,18,20,22,24,250,55,1,0,26,28,30,32,1,1,2,1050,10,35,0,4,45,130000,1750,3900,35000,800,3,9150,4075 +1.2,26,28000,17,19,21,23,25,260,56,2,1,27,29,31,33,2,8,1,1060,14,35,0,1,46,132000,1800,3960,36000,820,0,9160,4080 +1.3,27,28500,18,20,22,24,26,270,57,3,2,28,30,32,34,3,11,2,1070,18,35,11700,2,47,134000,1850,4020,37000,840,1,9170,4085 +1.3999999999999999,28,29000,19,21,23,25,27,280,58,1,0,29,31,33,35,4,12,1,1080,22,35,11800,3,48,136000,1900,4080,38000,860,2,9180,4090 +1.5,29,29500,20,22,24,26,28,290,59,2,1,30,32,34,36,5,1,2,1090,26,35,0,4,49,138000,1950,4140,39000,880,3,9190,4095 +1.6000000000000001,30,30000,21,23,25,27,29,300,60,3,2,31,33,35,37,1,8,1,1100,30,40,0,1,50,140000,2000,4200,40000,900,0,9200,4100 +1,31,30500,22,24,26,28,30,310,61,1,0,32,34,36,38,2,11,2,1110,34,40,12100,2,51,142000,2050,4260,41000,920,1,9210,4105 +1.1000000000000001,32,31000,23,25,27,29,31,320,62,2,1,33,35,37,39,3,12,1,1120,38,40,12200,3,52,144000,2100,4320,42000,940,2,9220,4110 +1.2,33,31500,24,26,28,30,32,330,63,3,2,34,36,38,40,4,1,2,1130,42,40,0,4,53,146000,2150,4380,43000,960,3,9230,4115 +1.3,34,32000,25,27,29,31,33,340,64,1,0,35,37,39,41,5,8,1,1140,46,40,0,1,54,148000,2200,4440,44000,980,0,9240,4120 +1.3999999999999999,35,32500,26,28,30,32,34,350,65,2,1,36,38,40,42,1,11,2,1150,50,45,12500,2,55,150000,2250,4500,45000,1000,1,9250,4125 +1.5,36,33000,27,29,31,33,35,360,66,3,2,37,39,41,43,2,12,1,1160,54,45,12600,3,56,152000,2300,4560,46000,1020,2,9260,4130 +1.6000000000000001,37,33500,28,30,32,34,36,370,67,1,0,38,40,42,44,3,1,2,1170,58,45,0,4,57,154000,2350,4620,47000,1040,3,9270,4135 +1,38,34000,29,31,33,35,37,380,68,2,1,39,41,43,45,4,8,1,1180,62,45,0,1,58,156000,2400,4680,48000,1060,0,9280,4140 +1.1000000000000001,39,34500,30,32,34,36,38,390,69,3,2,40,42,44,46,5,11,2,1190,66,45,12900,2,59,158000,2450,4740,49000,1080,1,9290,4145 +1.2,40,35000,31,33,35,37,39,400,70,1,0,41,43,45,47,1,12,1,1200,70,50,13000,3,60,160000,2500,4800,50000,1100,2,9300,4150 +1.3,41,35500,32,34,36,38,40,410,71,2,1,42,44,46,48,2,1,2,1210,74,50,0,4,61,162000,2550,4860,51000,1120,3,9310,4155 +1.3999999999999999,42,36000,33,35,37,39,41,420,72,3,2,43,45,47,49,3,8,1,1220,78,50,0,1,62,164000,2600,4920,52000,1140,0,9320,4160 +1.5,43,36500,34,36,38,40,42,430,73,1,0,44,46,48,50,4,11,2,1230,82,50,13300,2,63,166000,2650,4980,53000,1160,1,9330,4165 +1.6000000000000001,44,37000,35,37,39,41,43,440,74,2,1,45,47,49,51,5,12,1,1240,86,50,13400,3,64,168000,2700,5040,54000,1180,2,9340,4170 +1,45,37500,36,38,40,42,44,450,75,3,2,46,48,50,52,1,1,2,1250,90,55,0,4,65,170000,2750,5100,55000,1200,3,9350,4175 +1.1000000000000001,46,38000,37,39,41,43,45,460,76,1,0,47,49,51,53,2,8,1,1260,94,55,0,1,66,172000,2800,5160,56000,1220,0,9360,4180 +1.2,47,38500,38,40,42,44,46,470,77,2,1,48,50,52,54,3,11,2,1270,98,55,13700,2,67,174000,2850,5220,57000,1240,1,9370,4185 +1.3,48,39000,39,41,43,45,47,480,78,3,2,49,51,53,55,4,12,1,1280,102,55,13800,3,68,176000,2900,5280,58000,1260,2,9380,4190 +1.3999999999999999,49,39500,40,42,44,46,48,490,79,1,0,50,52,54,56,5,1,2,1290,106,55,0,4,69,178000,2950,5340,59000,1280,3,9390,4195 +1.5,50,40000,41,43,45,47,49,500,80,2,1,51,53,55,57,1,8,1,1300,110,60,0,1,70,180000,3000,5400,60000,1300,0,9400,4200 +1.6000000000000001,51,40500,42,44,46,48,50,510,81,3,2,52,54,56,58,2,11,2,1310,114,60,14100,2,71,182000,3050,5460,61000,1320,1,9410,4205 +1,52,41000,43,45,47,49,51,520,82,1,0,53,55,57,59,3,12,1,1320,118,60,14200,3,72,184000,3100,5520,62000,1340,2,9420,4210 +1.1000000000000001,53,41500,44,46,48,50,52,530,83,2,1,54,56,58,60,4,1,2,1330,122,60,0,4,73,186000,3150,5580,63000,1360,3,9430,4215 +1.2,54,42000,45,47,49,51,53,540,84,3,2,55,57,59,61,5,8,1,1340,126,60,0,1,74,188000,3200,5640,64000,1380,0,9440,4220 +1.3,55,42500,46,48,50,52,54,550,85,1,0,56,58,60,62,1,11,2,1350,130,65,14500,2,75,190000,3250,5700,65000,1400,1,9450,4225 +1.3999999999999999,56,43000,47,49,51,53,55,560,86,2,1,57,59,61,63,2,12,1,1360,134,65,14600,3,76,192000,3300,5760,66000,1420,2,9460,4230 +1.5,57,43500,48,50,52,54,56,570,87,3,2,58,60,62,64,3,1,2,1370,138,65,0,4,77,194000,3350,5820,67000,1440,3,9470,4235 +1.6000000000000001,58,44000,49,51,53,55,57,580,88,1,0,59,61,63,65,4,8,1,1380,142,65,0,1,78,196000,3400,5880,68000,1460,0,9480,4240 +1,59,44500,50,52,54,56,58,590,89,2,1,60,62,64,66,5,11,2,1390,146,65,14900,2,79,198000,3450,5940,69000,1480,1,9490,4245 +1.1000000000000001,60,45000,51,53,55,57,59,600,90,3,2,61,63,65,67,1,12,1,1400,150,70,15000,3,80,200000,3500,6000,70000,1500,2,9500,4250 +1.2,61,45500,52,54,56,58,60,610,91,1,0,62,64,66,68,2,1,2,1410,154,70,0,4,81,202000,3550,6060,71000,1520,3,9510,4255 +1.3,62,46000,53,55,57,59,61,620,92,2,1,63,65,67,69,3,8,1,1420,158,70,0,1,82,204000,3600,6120,72000,1540,0,9520,4260 +1.3999999999999999,63,46500,54,56,58,60,62,630,93,3,2,64,66,68,70,4,11,2,1430,162,70,15300,2,83,206000,3650,6180,73000,1560,1,9530,4265 +1.5,64,47000,55,57,59,61,63,640,94,1,0,65,67,69,71,5,12,1,1440,166,70,15400,3,84,208000,3700,6240,74000,1580,2,9540,4270 +1.6000000000000001,65,47500,56,58,60,62,64,650,95,2,1,66,68,70,72,1,1,2,1450,170,75,0,4,85,210000,3750,6300,75000,1600,3,9550,4275 +1,66,48000,57,59,61,63,65,660,96,3,2,67,69,71,73,2,8,1,1460,174,75,0,1,86,212000,3800,6360,76000,1620,0,9560,4280 +1.1000000000000001,67,48500,58,60,62,64,66,670,97,1,0,68,70,72,74,3,11,2,1470,178,75,15700,2,87,214000,3850,6420,77000,1640,1,9570,4285 +1.2,68,49000,59,61,63,65,67,680,98,2,1,69,71,73,75,4,12,1,1480,182,75,15800,3,88,216000,3900,6480,78000,1660,2,9580,4290 +1.3,69,49500,60,62,64,66,68,690,99,3,2,70,72,74,76,5,1,2,1490,186,75,0,4,89,218000,3950,6540,79000,1680,3,9590,4295 +1.3999999999999999,70,50000,61,63,65,67,69,700,100,1,0,71,73,75,77,1,8,1,1500,190,80,0,1,90,220000,4000,6600,80000,1700,0,9600,4300 +1.5,71,50500,62,64,66,68,70,710,101,2,1,72,74,76,78,2,11,2,1510,194,80,16100,2,91,222000,4050,6660,81000,1720,1,9610,4305 +1.6000000000000001,72,51000,63,65,67,69,71,720,102,3,2,73,75,77,79,3,12,1,1520,198,80,16200,3,92,224000,4100,6720,82000,1740,2,9620,4310 +1,73,51500,64,66,68,70,72,730,103,1,0,74,76,78,80,4,1,2,1530,202,80,0,4,93,226000,4150,6780,83000,1760,3,9630,4315 diff --git a/packages/microcosm-graph/tests/fixtures/parity/uk_spine/uk_spine.json b/packages/microcosm-graph/tests/fixtures/parity/uk_spine/uk_spine.json index 3b9bcb921..af6212507 100644 --- a/packages/microcosm-graph/tests/fixtures/parity/uk_spine/uk_spine.json +++ b/packages/microcosm-graph/tests/fixtures/parity/uk_spine/uk_spine.json @@ -1 +1 @@ -{"country":"uk","nodes":[{"base":null,"citation":"","description":"Load the source-bound UK FRS root population.","id":"create_uk_frs","inputs":[],"kernel":"uk.create@1","mass":"conserve","outputs":[{"column":"age","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"gender","dtype":"string","entity":"person","ownership":"produced","rows":"all"},{"column":"marital_status","dtype":"string","entity":"person","ownership":"produced","rows":"all"},{"column":"hours_worked","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"is_household_head","dtype":"bool","entity":"person","ownership":"produced","rows":"all"},{"column":"is_benunit_head","dtype":"bool","entity":"person","ownership":"produced","rows":"all"},{"column":"is_parent","dtype":"bool","entity":"person","ownership":"produced","rows":"all"},{"column":"employment_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"self_employment_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"private_pension_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"tax_free_savings_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"savings_interest_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"dividend_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"property_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"maintenance_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"miscellaneous_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"private_transfer_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"lump_sum_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"student_loan_repayments","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"statutory_sick_pay","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"statutory_maternity_pay","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"student_loans","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"access_fund","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"education_grants","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"healthy_start_vouchers","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"free_school_breakfasts","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"free_school_fruit_veg","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"free_school_meals","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"council_tax_benefit_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"maintenance_expenses","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"childcare_expenses","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"personal_pension_contributions","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"employee_pension_contributions","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"pension_contributions_via_salary_sacrifice","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"salary_sacrifice_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"salary_sacrifice_asked","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"child_benefit_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"income_support_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"housing_benefit_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"attendance_allowance_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"dla_sc_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"dla_m_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"iidb_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"carers_allowance_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"sda_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"afcs_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"ssmg_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"pension_credit_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"child_tax_credit_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"working_tax_credit_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"state_pension_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"winter_fuel_allowance_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"incapacity_benefit_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"universal_credit_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"pip_m_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"pip_dl_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"jsa_contrib_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"jsa_income_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"esa_contrib_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"esa_income_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"bsp_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"frs_benunit_capital","dtype":"float64","entity":"benunit","ownership":"produced","rows":"all"},{"column":"is_married","dtype":"bool","entity":"benunit","ownership":"produced","rows":"all"},{"column":"dependent_children","dtype":"int64","entity":"benunit","ownership":"produced","rows":"all"},{"column":"region","dtype":"string","entity":"household","ownership":"produced","rows":"all"},{"column":"tenure_type","dtype":"string","entity":"household","ownership":"produced","rows":"all"},{"column":"accommodation_type","dtype":"string","entity":"household","ownership":"produced","rows":"all"},{"column":"num_bedrooms","dtype":"int64","entity":"household","ownership":"produced","rows":"all"},{"column":"council_tax_reported","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"council_tax_band","dtype":"string","entity":"household","ownership":"produced","rows":"all"},{"column":"council_tax_rebate","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"council_tax_single_adult_raw","dtype":"int64","entity":"household","ownership":"produced","rows":"all"},{"column":"water_and_sewerage_charges","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"domestic_rates","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"rent","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"subrent","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"mortgage_interest_repayment","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"mortgage_capital_repayment","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"structural_insurance_payments","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"housing_service_charges","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"external_child_payments","dtype":"float64","entity":"household","ownership":"produced","rows":"all"}],"params":{"sample_fraction":1.0,"sample_seed":578,"stage_contract_sha256":"dfd331a9aba8fd095ebb16250f90cbb5de2ea999874c22fa3bea6a7e33fa83bf","time_period":"2024"},"population":null,"sources":["frs"],"structural":"create","weights":null},{"base":"create_uk_frs","citation":"","description":"Ownership boundary for the source-assembling root stage.","id":"frs_spine.boundary","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","employee_pension_contributions","pension_contributions_via_salary_sacrifice","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","universal_credit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported"],"entity":"person","rows":"all"}],"kernel":"uk.identity@1","mass":"conserve","outputs":[],"params":{},"population":null,"sources":[],"structural":"filter","weights":null},{"base":null,"citation":"","description":"Claim the cells assembled by the UK FRS root transform.","id":"frs_spine","inputs":[],"kernel":"uk.claim@1","mass":"conserve","outputs":[{"column":"age","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"gender","dtype":"string","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"marital_status","dtype":"string","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"hours_worked","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"is_household_head","dtype":"bool","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"is_benunit_head","dtype":"bool","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"is_parent","dtype":"bool","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"employment_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"self_employment_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"private_pension_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"tax_free_savings_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"savings_interest_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"dividend_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"property_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"maintenance_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"miscellaneous_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"private_transfer_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"lump_sum_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"student_loan_repayments","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"statutory_sick_pay","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"statutory_maternity_pay","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"student_loans","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"access_fund","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"education_grants","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"healthy_start_vouchers","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"free_school_breakfasts","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"free_school_fruit_veg","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"free_school_meals","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"council_tax_benefit_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"maintenance_expenses","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"childcare_expenses","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"personal_pension_contributions","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"employee_pension_contributions","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"pension_contributions_via_salary_sacrifice","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"salary_sacrifice_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"salary_sacrifice_asked","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"child_benefit_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"income_support_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"housing_benefit_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"attendance_allowance_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"dla_sc_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"dla_m_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"iidb_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"carers_allowance_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"sda_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"afcs_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"ssmg_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"pension_credit_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"child_tax_credit_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"working_tax_credit_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"state_pension_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"winter_fuel_allowance_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"incapacity_benefit_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"universal_credit_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"pip_m_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"pip_dl_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"jsa_contrib_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"jsa_income_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"esa_contrib_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"esa_income_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"bsp_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"frs_benunit_capital","dtype":"float64","entity":"benunit","ownership":"produced","rewrite":true,"rows":"all"},{"column":"is_married","dtype":"bool","entity":"benunit","ownership":"produced","rewrite":true,"rows":"all"},{"column":"dependent_children","dtype":"int64","entity":"benunit","ownership":"produced","rewrite":true,"rows":"all"},{"column":"region","dtype":"string","entity":"household","ownership":"produced","rewrite":true,"rows":"all"},{"column":"tenure_type","dtype":"string","entity":"household","ownership":"produced","rewrite":true,"rows":"all"},{"column":"accommodation_type","dtype":"string","entity":"household","ownership":"produced","rewrite":true,"rows":"all"},{"column":"num_bedrooms","dtype":"int64","entity":"household","ownership":"produced","rewrite":true,"rows":"all"},{"column":"council_tax_reported","dtype":"float64","entity":"household","ownership":"produced","rewrite":true,"rows":"all"},{"column":"council_tax_band","dtype":"string","entity":"household","ownership":"produced","rewrite":true,"rows":"all"},{"column":"council_tax_rebate","dtype":"float64","entity":"household","ownership":"produced","rewrite":true,"rows":"all"},{"column":"council_tax_single_adult_raw","dtype":"int64","entity":"household","ownership":"produced","rewrite":true,"rows":"all"},{"column":"water_and_sewerage_charges","dtype":"float64","entity":"household","ownership":"produced","rewrite":true,"rows":"all"},{"column":"domestic_rates","dtype":"float64","entity":"household","ownership":"produced","rewrite":true,"rows":"all"},{"column":"rent","dtype":"float64","entity":"household","ownership":"produced","rewrite":true,"rows":"all"},{"column":"subrent","dtype":"float64","entity":"household","ownership":"produced","rewrite":true,"rows":"all"},{"column":"mortgage_interest_repayment","dtype":"float64","entity":"household","ownership":"produced","rewrite":true,"rows":"all"},{"column":"mortgage_capital_repayment","dtype":"float64","entity":"household","ownership":"produced","rewrite":true,"rows":"all"},{"column":"structural_insurance_payments","dtype":"float64","entity":"household","ownership":"produced","rewrite":true,"rows":"all"},{"column":"housing_service_charges","dtype":"float64","entity":"household","ownership":"produced","rewrite":true,"rows":"all"},{"column":"external_child_payments","dtype":"float64","entity":"household","ownership":"produced","rewrite":true,"rows":"all"}],"params":{},"population":"frs_spine.boundary","sources":[],"structural":"none","weights":null},{"base":null,"citation":"","description":"Run UK spine stage frs_employment.","id":"frs_employment","inputs":[{"columns":["frs_benunit_capital"],"entity":"benunit","rows":"all"},{"columns":["external_child_payments","region"],"entity":"household","rows":"all"},{"columns":["age"],"entity":"person","rows":"all"}],"kernel":"uk.stage.frs_employment@1","mass":"conserve","outputs":[{"column":"employment_status","dtype":"string","entity":"person","ownership":"produced","rows":"all"},{"column":"employment_sector","dtype":"string","entity":"person","ownership":"produced","rows":"all"},{"column":"sic_industry_division","dtype":"int64","entity":"person","ownership":"produced","rows":"all"}],"params":{"stage":"frs_employment","stage_contract_sha256":"ddbefaf05b44788d794a6e4b0e8926c14318d66e50d2f11f50ca549538bbf60c","time_period":"2024"},"population":"frs_spine.boundary","sources":["frs"],"structural":"none","weights":null},{"base":null,"citation":"","description":"Run UK spine stage frs_council_tax.","id":"frs_council_tax","inputs":[{"columns":["frs_benunit_capital"],"entity":"benunit","rows":"all"},{"columns":["region"],"entity":"household","rows":"all"},{"columns":["age","sic_industry_division"],"entity":"person","rows":"all"}],"kernel":"uk.stage.frs_council_tax@1","mass":"conserve","outputs":[{"column":"council_tax","dtype":"float64","entity":"household","ownership":"produced","rows":"all"}],"params":{"stage":"frs_council_tax","stage_contract_sha256":"3381cd9f7a736514d5073c57480e07f5098890f3ca9ed3865392043594771eb9","time_period":"2024"},"population":"frs_spine.boundary","sources":["frs"],"structural":"none","weights":null},{"base":null,"citation":"","description":"Run UK spine stage frs_disability.","id":"frs_disability","inputs":[{"columns":["frs_benunit_capital"],"entity":"benunit","rows":"all"},{"columns":["council_tax","region"],"entity":"household","rows":"all"},{"columns":["afcs_reported","age","attendance_allowance_reported","dla_m_reported","dla_sc_reported","esa_contrib_reported","esa_income_reported","iidb_reported","incapacity_benefit_reported","pip_dl_reported","pip_m_reported","sda_reported"],"entity":"person","rows":"all"}],"kernel":"uk.stage.frs_disability@1","mass":"conserve","outputs":[{"column":"aa_category","dtype":"string","entity":"person","ownership":"produced","rows":"all"},{"column":"dla_sc_category","dtype":"string","entity":"person","ownership":"produced","rows":"all"},{"column":"dla_m_category","dtype":"string","entity":"person","ownership":"produced","rows":"all"},{"column":"pip_m_category","dtype":"string","entity":"person","ownership":"produced","rows":"all"},{"column":"pip_dl_category","dtype":"string","entity":"person","ownership":"produced","rows":"all"},{"column":"is_disabled_for_benefits","dtype":"bool","entity":"person","ownership":"produced","rows":"all"},{"column":"is_enhanced_disabled_for_benefits","dtype":"bool","entity":"person","ownership":"produced","rows":"all"},{"column":"is_severely_disabled_for_benefits","dtype":"bool","entity":"person","ownership":"produced","rows":"all"}],"params":{"stage":"frs_disability","stage_contract_sha256":"2d5ed820f9f9fbe4a45a333f5c5e1dfbdfe2b53a055584ada4c44eacb4b58954","time_period":"2024"},"population":"frs_spine.boundary","sources":["frs"],"structural":"none","weights":null},{"base":null,"citation":"","description":"Run UK spine stage frs_education.","id":"frs_education","inputs":[{"columns":["frs_benunit_capital"],"entity":"benunit","rows":"all"},{"columns":["region"],"entity":"household","rows":"all"},{"columns":["age","esa_contrib_reported","esa_income_reported","is_severely_disabled_for_benefits","jsa_contrib_reported","jsa_income_reported","universal_credit_reported"],"entity":"person","rows":"all"}],"kernel":"uk.stage.frs_education@1","mass":"conserve","outputs":[{"column":"current_education","dtype":"string","entity":"person","ownership":"produced","rows":"all"},{"column":"highest_education","dtype":"string","entity":"person","ownership":"produced","rows":"all"},{"column":"is_in_non_advanced_education","dtype":"bool","entity":"person","ownership":"produced","rows":"all"},{"column":"is_in_approved_training","dtype":"bool","entity":"person","ownership":"produced","rows":"all"},{"column":"age_started_or_accepted_current_education_or_training","dtype":"int64","entity":"person","ownership":"produced","rows":"all"},{"column":"is_before_universal_credit_qualifying_young_person_terminal_date","dtype":"bool","entity":"person","ownership":"produced","rows":"all"},{"column":"adult_ema","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"child_ema","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"receives_benefits_in_own_right","dtype":"bool","entity":"person","ownership":"produced","rows":"all"}],"params":{"stage":"frs_education","stage_contract_sha256":"836cb0f5a582fee1425190e96c9cb81bdef859bd236c8b6bc27660b6e3d0c2f8","time_period":"2024"},"population":"frs_spine.boundary","sources":["frs"],"structural":"none","weights":null},{"base":null,"citation":"","description":"Run UK spine stage frs_legacy_proxies.","id":"frs_legacy_proxies","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","employee_pension_contributions","pension_contributions_via_salary_sacrifice","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","universal_credit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported","employment_status","employment_sector","sic_industry_division","aa_category","dla_sc_category","dla_m_category","pip_m_category","pip_dl_category","is_disabled_for_benefits","is_enhanced_disabled_for_benefits","is_severely_disabled_for_benefits","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right"],"entity":"person","rows":"all"}],"kernel":"uk.stage.frs_legacy_proxies@1","mass":"conserve","outputs":[{"column":"legacy_jobseeker_proxy","dtype":"bool","entity":"person","ownership":"produced","rows":"all"},{"column":"esa_health_condition_proxy","dtype":"bool","entity":"person","ownership":"produced","rows":"all"},{"column":"esa_support_group_proxy","dtype":"bool","entity":"person","ownership":"produced","rows":"all"}],"params":{"stage":"frs_legacy_proxies","stage_contract_sha256":"1ecf761cf3fa7180da15659e138e67a8654e58272b4fde28344aa27a874ad0aa","time_period":"2024"},"population":"frs_spine.boundary","sources":["frs"],"structural":"none","weights":null},{"base":"frs_spine.boundary","citation":"","description":"Ownership boundary before frs_education_grant_split rewrites.","id":"frs_education_grant_split.boundary","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","employee_pension_contributions","pension_contributions_via_salary_sacrifice","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","universal_credit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported","employment_status","employment_sector","sic_industry_division","aa_category","dla_sc_category","dla_m_category","pip_m_category","pip_dl_category","is_disabled_for_benefits","is_enhanced_disabled_for_benefits","is_severely_disabled_for_benefits","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right","legacy_jobseeker_proxy","esa_health_condition_proxy","esa_support_group_proxy"],"entity":"person","rows":"all"}],"kernel":"uk.identity@1","mass":"conserve","outputs":[],"params":{},"population":null,"sources":[],"structural":"filter","weights":null},{"base":null,"citation":"","description":"Run UK spine stage frs_education_grant_split.","id":"frs_education_grant_split","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","employee_pension_contributions","pension_contributions_via_salary_sacrifice","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","universal_credit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported","employment_status","employment_sector","sic_industry_division","aa_category","dla_sc_category","dla_m_category","pip_m_category","pip_dl_category","is_disabled_for_benefits","is_enhanced_disabled_for_benefits","is_severely_disabled_for_benefits","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right","legacy_jobseeker_proxy","esa_health_condition_proxy","esa_support_group_proxy"],"entity":"person","rows":"all"}],"kernel":"uk.stage.frs_education_grant_split@1","mass":"conserve","outputs":[{"column":"disabled_students_allowance_eligible_expenses","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"education_grants","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"}],"params":{"stage":"frs_education_grant_split","stage_contract_sha256":"6f0fcfa6a1aa614f7ccefe39364b4ebd208c9a4ed081f7618e026b2e87f152fa","time_period":"2024"},"population":"frs_education_grant_split.boundary","sources":["frs"],"structural":"none","weights":null},{"base":null,"citation":"","description":"Run UK spine stage frs_take_up.","id":"frs_take_up","inputs":[{"columns":["frs_benunit_capital"],"entity":"benunit","rows":"all"},{"columns":["region"],"entity":"household","rows":"all"},{"columns":["age","child_benefit_reported","education_grants","pension_credit_reported","universal_credit_reported"],"entity":"person","rows":"all"}],"kernel":"uk.stage.frs_take_up@1","mass":"conserve","outputs":[{"column":"would_claim_child_benefit","dtype":"bool","entity":"benunit","ownership":"produced","rows":"all"},{"column":"child_benefit_opts_out","dtype":"bool","entity":"benunit","ownership":"produced","rows":"all"},{"column":"would_claim_pc","dtype":"bool","entity":"benunit","ownership":"produced","rows":"all"},{"column":"would_claim_uc","dtype":"bool","entity":"benunit","ownership":"produced","rows":"all"},{"column":"would_claim_tfc","dtype":"bool","entity":"benunit","ownership":"produced","rows":"all"},{"column":"would_claim_extended_childcare","dtype":"bool","entity":"benunit","ownership":"produced","rows":"all"},{"column":"would_claim_universal_childcare","dtype":"bool","entity":"benunit","ownership":"produced","rows":"all"},{"column":"would_claim_targeted_childcare","dtype":"bool","entity":"benunit","ownership":"produced","rows":"all"},{"column":"maximum_extended_childcare_hours_usage","dtype":"float64","entity":"benunit","ownership":"produced","rows":"all"}],"params":{"stage":"frs_take_up","stage_contract_sha256":"d10ca01bfa5d719640ded8e62196b0692236d3be0a2417899deeefd241088abf","time_period":"2024"},"population":"frs_education_grant_split.boundary","sources":["frs"],"structural":"none","weights":null},{"base":null,"citation":"","description":"Run UK spine stage frs_person_draws.","id":"frs_person_draws","inputs":[{"columns":["frs_benunit_capital","maximum_extended_childcare_hours_usage"],"entity":"benunit","rows":"all"},{"columns":["region"],"entity":"household","rows":"all"},{"columns":["age"],"entity":"person","rows":"all"}],"kernel":"uk.stage.frs_person_draws@1","mass":"conserve","outputs":[{"column":"would_claim_marriage_allowance","dtype":"bool","entity":"person","ownership":"produced","rows":"all"},{"column":"would_claim_scp","dtype":"bool","entity":"person","ownership":"produced","rows":"all"},{"column":"attends_private_school_random_draw","dtype":"float64","entity":"person","ownership":"produced","rows":"all"}],"params":{"stage":"frs_person_draws","stage_contract_sha256":"1543fd39f943d628d224adf8a31d5379d4f498d3f3861e67ec26981a24e76d20","time_period":"2024"},"population":"frs_education_grant_split.boundary","sources":["frs"],"structural":"none","weights":null},{"base":null,"citation":"","description":"Run UK spine stage frs_household_draws.","id":"frs_household_draws","inputs":[{"columns":["frs_benunit_capital"],"entity":"benunit","rows":"all"},{"columns":["region"],"entity":"household","rows":"all"},{"columns":["age","attends_private_school_random_draw"],"entity":"person","rows":"all"}],"kernel":"uk.stage.frs_household_draws@1","mass":"conserve","outputs":[{"column":"household_owns_tv","dtype":"bool","entity":"household","ownership":"produced","rows":"all"},{"column":"would_evade_tv_licence_fee","dtype":"bool","entity":"household","ownership":"produced","rows":"all"},{"column":"main_residential_property_purchased_is_first_home","dtype":"bool","entity":"household","ownership":"produced","rows":"all"},{"column":"property_purchased","dtype":"bool","entity":"household","ownership":"produced","rows":"all"}],"params":{"stage":"frs_household_draws","stage_contract_sha256":"c74c63b09264319c4bf0049dabba00ecd0ce660beb7b53d6dae71518434bc949","time_period":"2024"},"population":"frs_education_grant_split.boundary","sources":["frs"],"structural":"none","weights":null},{"base":null,"citation":"","description":"Run UK spine stage frs_brma.","id":"frs_brma","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children","would_claim_child_benefit","child_benefit_opts_out","would_claim_pc","would_claim_uc","would_claim_tfc","would_claim_extended_childcare","would_claim_universal_childcare","would_claim_targeted_childcare","maximum_extended_childcare_hours_usage"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax","household_owns_tv","would_evade_tv_licence_fee","main_residential_property_purchased_is_first_home","property_purchased"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","employee_pension_contributions","pension_contributions_via_salary_sacrifice","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","universal_credit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported","employment_status","employment_sector","sic_industry_division","aa_category","dla_sc_category","dla_m_category","pip_m_category","pip_dl_category","is_disabled_for_benefits","is_enhanced_disabled_for_benefits","is_severely_disabled_for_benefits","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right","legacy_jobseeker_proxy","esa_health_condition_proxy","esa_support_group_proxy","disabled_students_allowance_eligible_expenses","would_claim_marriage_allowance","would_claim_scp","attends_private_school_random_draw"],"entity":"person","rows":"all"}],"kernel":"uk.stage.frs_brma@1","mass":"conserve","outputs":[{"column":"brma","dtype":"string","entity":"household","ownership":"produced","rows":"all"}],"params":{"stage":"frs_brma","stage_contract_sha256":"5b8f0b361310c7cd3efbaae3762b347649b7d4afb9e9943131d9bef9266fc3c8","time_period":"2024"},"population":"frs_education_grant_split.boundary","sources":["frs"],"structural":"none","weights":null},{"base":"frs_education_grant_split.boundary","citation":"","description":"Freeze the assembled-spine gate population.","id":"frs_brma.checkpoint","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children","would_claim_child_benefit","child_benefit_opts_out","would_claim_pc","would_claim_uc","would_claim_tfc","would_claim_extended_childcare","would_claim_universal_childcare","would_claim_targeted_childcare","maximum_extended_childcare_hours_usage"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax","household_owns_tv","would_evade_tv_licence_fee","main_residential_property_purchased_is_first_home","property_purchased","brma"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","employee_pension_contributions","pension_contributions_via_salary_sacrifice","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","universal_credit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported","employment_status","employment_sector","sic_industry_division","aa_category","dla_sc_category","dla_m_category","pip_m_category","pip_dl_category","is_disabled_for_benefits","is_enhanced_disabled_for_benefits","is_severely_disabled_for_benefits","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right","legacy_jobseeker_proxy","esa_health_condition_proxy","esa_support_group_proxy","disabled_students_allowance_eligible_expenses","would_claim_marriage_allowance","would_claim_scp","attends_private_school_random_draw"],"entity":"person","rows":"all"}],"kernel":"uk.identity@1","mass":"conserve","outputs":[],"params":{},"population":null,"sources":[],"structural":"filter","weights":null},{"base":null,"citation":"","description":"Run UK spine stage was_wealth.","id":"was_wealth","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children","would_claim_child_benefit","child_benefit_opts_out","would_claim_pc","would_claim_uc","would_claim_tfc","would_claim_extended_childcare","would_claim_universal_childcare","would_claim_targeted_childcare","maximum_extended_childcare_hours_usage"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax","household_owns_tv","would_evade_tv_licence_fee","main_residential_property_purchased_is_first_home","property_purchased","brma"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","employee_pension_contributions","pension_contributions_via_salary_sacrifice","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","universal_credit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported","employment_status","employment_sector","sic_industry_division","aa_category","dla_sc_category","dla_m_category","pip_m_category","pip_dl_category","is_disabled_for_benefits","is_enhanced_disabled_for_benefits","is_severely_disabled_for_benefits","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right","legacy_jobseeker_proxy","esa_health_condition_proxy","esa_support_group_proxy","disabled_students_allowance_eligible_expenses","would_claim_marriage_allowance","would_claim_scp","attends_private_school_random_draw"],"entity":"person","rows":"all"}],"kernel":"uk.stage.was_wealth@1","mass":"conserve","outputs":[{"column":"owned_land","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"property_wealth","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"corporate_wealth","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"private_pension_wealth","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"gross_financial_wealth","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"net_financial_wealth","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"main_residence_value","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"other_residential_property_value","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"non_residential_property_value","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"savings","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"num_vehicles","dtype":"int64","entity":"household","ownership":"produced","rows":"all"},{"column":"cash_isa","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"stocks_and_shares_isa","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"student_loan_balance","dtype":"float64","entity":"person","ownership":"produced","rows":"all"}],"params":{"stage":"was_wealth","stage_contract_sha256":"a158ee27d13ce9cbf45812a0266ffbe16bef459ed2489084ae67e36f9e02008b","time_period":"2024"},"population":"frs_brma.checkpoint","sources":["frs"],"structural":"none","weights":null},{"base":"frs_brma.checkpoint","citation":"","description":"Ownership boundary before regional_property_uprating rewrites.","id":"regional_property_uprating.boundary","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children","would_claim_child_benefit","child_benefit_opts_out","would_claim_pc","would_claim_uc","would_claim_tfc","would_claim_extended_childcare","would_claim_universal_childcare","would_claim_targeted_childcare","maximum_extended_childcare_hours_usage"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax","household_owns_tv","would_evade_tv_licence_fee","main_residential_property_purchased_is_first_home","property_purchased","brma","owned_land","property_wealth","corporate_wealth","private_pension_wealth","gross_financial_wealth","net_financial_wealth","main_residence_value","other_residential_property_value","non_residential_property_value","savings","num_vehicles","cash_isa","stocks_and_shares_isa"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","employee_pension_contributions","pension_contributions_via_salary_sacrifice","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","universal_credit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported","employment_status","employment_sector","sic_industry_division","aa_category","dla_sc_category","dla_m_category","pip_m_category","pip_dl_category","is_disabled_for_benefits","is_enhanced_disabled_for_benefits","is_severely_disabled_for_benefits","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right","legacy_jobseeker_proxy","esa_health_condition_proxy","esa_support_group_proxy","disabled_students_allowance_eligible_expenses","would_claim_marriage_allowance","would_claim_scp","attends_private_school_random_draw","student_loan_balance"],"entity":"person","rows":"all"}],"kernel":"uk.identity@1","mass":"conserve","outputs":[],"params":{},"population":null,"sources":[],"structural":"filter","weights":null},{"base":null,"citation":"","description":"Run UK spine stage regional_property_uprating.","id":"regional_property_uprating","inputs":[{"columns":["frs_benunit_capital"],"entity":"benunit","rows":"all"},{"columns":["region"],"entity":"household","rows":"all"},{"columns":["age","student_loan_balance"],"entity":"person","rows":"all"}],"kernel":"uk.stage.regional_property_uprating@1","mass":"conserve","outputs":[{"column":"main_residence_value","dtype":"float64","entity":"household","ownership":"produced","rewrite":true,"rows":"all"},{"column":"property_wealth","dtype":"float64","entity":"household","ownership":"produced","rewrite":true,"rows":"all"}],"params":{"stage":"regional_property_uprating","stage_contract_sha256":"4304356c6c5ba91a04883148cbf9078761ee926c8387aadb6c15b6b70ebb31f6","time_period":"2024"},"population":"regional_property_uprating.boundary","sources":["frs"],"structural":"none","weights":null},{"base":null,"citation":"","description":"Run UK spine stage lcfs_consumption.","id":"lcfs_consumption","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children","would_claim_child_benefit","child_benefit_opts_out","would_claim_pc","would_claim_uc","would_claim_tfc","would_claim_extended_childcare","would_claim_universal_childcare","would_claim_targeted_childcare","maximum_extended_childcare_hours_usage"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax","household_owns_tv","would_evade_tv_licence_fee","main_residential_property_purchased_is_first_home","property_purchased","brma","owned_land","property_wealth","corporate_wealth","private_pension_wealth","gross_financial_wealth","net_financial_wealth","main_residence_value","other_residential_property_value","non_residential_property_value","savings","num_vehicles","cash_isa","stocks_and_shares_isa"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","employee_pension_contributions","pension_contributions_via_salary_sacrifice","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","universal_credit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported","employment_status","employment_sector","sic_industry_division","aa_category","dla_sc_category","dla_m_category","pip_m_category","pip_dl_category","is_disabled_for_benefits","is_enhanced_disabled_for_benefits","is_severely_disabled_for_benefits","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right","legacy_jobseeker_proxy","esa_health_condition_proxy","esa_support_group_proxy","disabled_students_allowance_eligible_expenses","would_claim_marriage_allowance","would_claim_scp","attends_private_school_random_draw","student_loan_balance"],"entity":"person","rows":"all"}],"kernel":"uk.stage.lcfs_consumption@1","mass":"conserve","outputs":[{"column":"food_and_non_alcoholic_beverages_consumption","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"alcohol_and_tobacco_consumption","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"clothing_and_footwear_consumption","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"housing_water_and_electricity_consumption","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"household_furnishings_consumption","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"health_consumption","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"transport_consumption","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"communication_consumption","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"recreation_consumption","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"education_consumption","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"restaurants_and_hotels_consumption","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"miscellaneous_consumption","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"petrol_spending","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"diesel_spending","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"bus_fare_spending","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"domestic_energy_consumption","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"electricity_consumption","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"gas_consumption","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"has_fuel_consumption","dtype":"bool","entity":"household","ownership":"produced","rows":"all"}],"params":{"stage":"lcfs_consumption","stage_contract_sha256":"eb0a529c357c84290a001209a18c5a43b0a0310553c77963d6842fd0111b73ad","time_period":"2024"},"population":"regional_property_uprating.boundary","sources":["frs"],"structural":"none","weights":null},{"base":null,"citation":"","description":"Run UK spine stage etb_vat.","id":"etb_vat","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children","would_claim_child_benefit","child_benefit_opts_out","would_claim_pc","would_claim_uc","would_claim_tfc","would_claim_extended_childcare","would_claim_universal_childcare","would_claim_targeted_childcare","maximum_extended_childcare_hours_usage"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax","household_owns_tv","would_evade_tv_licence_fee","main_residential_property_purchased_is_first_home","property_purchased","brma","owned_land","property_wealth","corporate_wealth","private_pension_wealth","gross_financial_wealth","net_financial_wealth","main_residence_value","other_residential_property_value","non_residential_property_value","savings","num_vehicles","cash_isa","stocks_and_shares_isa","food_and_non_alcoholic_beverages_consumption","alcohol_and_tobacco_consumption","clothing_and_footwear_consumption","housing_water_and_electricity_consumption","household_furnishings_consumption","health_consumption","transport_consumption","communication_consumption","recreation_consumption","education_consumption","restaurants_and_hotels_consumption","miscellaneous_consumption","petrol_spending","diesel_spending","bus_fare_spending","domestic_energy_consumption","electricity_consumption","gas_consumption","has_fuel_consumption"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","employee_pension_contributions","pension_contributions_via_salary_sacrifice","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","universal_credit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported","employment_status","employment_sector","sic_industry_division","aa_category","dla_sc_category","dla_m_category","pip_m_category","pip_dl_category","is_disabled_for_benefits","is_enhanced_disabled_for_benefits","is_severely_disabled_for_benefits","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right","legacy_jobseeker_proxy","esa_health_condition_proxy","esa_support_group_proxy","disabled_students_allowance_eligible_expenses","would_claim_marriage_allowance","would_claim_scp","attends_private_school_random_draw","student_loan_balance"],"entity":"person","rows":"all"}],"kernel":"uk.stage.etb_vat@1","mass":"conserve","outputs":[{"column":"full_rate_vat_expenditure_rate","dtype":"float64","entity":"household","ownership":"produced","rows":"all"}],"params":{"stage":"etb_vat","stage_contract_sha256":"99a6756256adfe8255672fafda89710c08ae56bd2b8f011b20c3b8381eabfa1e","time_period":"2024"},"population":"regional_property_uprating.boundary","sources":["frs"],"structural":"none","weights":null},{"base":null,"citation":"","description":"Run UK spine stage etb_services.","id":"etb_services","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children","would_claim_child_benefit","child_benefit_opts_out","would_claim_pc","would_claim_uc","would_claim_tfc","would_claim_extended_childcare","would_claim_universal_childcare","would_claim_targeted_childcare","maximum_extended_childcare_hours_usage"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax","household_owns_tv","would_evade_tv_licence_fee","main_residential_property_purchased_is_first_home","property_purchased","brma","owned_land","property_wealth","corporate_wealth","private_pension_wealth","gross_financial_wealth","net_financial_wealth","main_residence_value","other_residential_property_value","non_residential_property_value","savings","num_vehicles","cash_isa","stocks_and_shares_isa","food_and_non_alcoholic_beverages_consumption","alcohol_and_tobacco_consumption","clothing_and_footwear_consumption","housing_water_and_electricity_consumption","household_furnishings_consumption","health_consumption","transport_consumption","communication_consumption","recreation_consumption","education_consumption","restaurants_and_hotels_consumption","miscellaneous_consumption","petrol_spending","diesel_spending","bus_fare_spending","domestic_energy_consumption","electricity_consumption","gas_consumption","has_fuel_consumption","full_rate_vat_expenditure_rate"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","employee_pension_contributions","pension_contributions_via_salary_sacrifice","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","universal_credit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported","employment_status","employment_sector","sic_industry_division","aa_category","dla_sc_category","dla_m_category","pip_m_category","pip_dl_category","is_disabled_for_benefits","is_enhanced_disabled_for_benefits","is_severely_disabled_for_benefits","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right","legacy_jobseeker_proxy","esa_health_condition_proxy","esa_support_group_proxy","disabled_students_allowance_eligible_expenses","would_claim_marriage_allowance","would_claim_scp","attends_private_school_random_draw","student_loan_balance"],"entity":"person","rows":"all"}],"kernel":"uk.stage.etb_services@1","mass":"conserve","outputs":[{"column":"dfe_education_spending","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"rail_subsidy_spending","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"bus_subsidy_spending","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"rail_usage","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"a_and_e_visits","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"admitted_patient_visits","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"outpatient_visits","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"nhs_a_and_e_spending","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"nhs_admitted_patient_spending","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"nhs_outpatient_spending","dtype":"float64","entity":"person","ownership":"produced","rows":"all"}],"params":{"stage":"etb_services","stage_contract_sha256":"5ca1399954560400665579d29791831eac81202aada216939bf91789136cdd03","time_period":"2024"},"population":"regional_property_uprating.boundary","sources":["frs"],"structural":"none","weights":null},{"base":null,"citation":"","description":"Run UK spine stage frs_hmrc_spine_leaves.","id":"frs_hmrc_spine_leaves","inputs":[{"columns":["frs_benunit_capital"],"entity":"benunit","rows":"all"},{"columns":["region"],"entity":"household","rows":"all"},{"columns":["age","employee_pension_contributions","nhs_outpatient_spending"],"entity":"person","rows":"all"}],"kernel":"uk.stage.frs_hmrc_spine_leaves@1","mass":"conserve","outputs":[{"column":"hmrc_spi_pay","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"hmrc_spi_unemployment_benefit_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"hmrc_spi_incapacity_benefit_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"ossben_identifiable_subset","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"srp_regular_code5","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"employer_pension_contributions","dtype":"float64","entity":"person","ownership":"produced","rows":"all"}],"params":{"stage":"frs_hmrc_spine_leaves","stage_contract_sha256":"2bb3d068003489b47cc8676ac26c203ce3c39a9b6e92f3ecd0a3c06acd6addc4","time_period":"2024"},"population":"regional_property_uprating.boundary","sources":["frs"],"structural":"none","weights":null},{"base":"regional_property_uprating.boundary","citation":"","description":"Run structural UK stage spi_support_channel.","id":"spi_support_channel","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children","would_claim_child_benefit","child_benefit_opts_out","would_claim_pc","would_claim_uc","would_claim_tfc","would_claim_extended_childcare","would_claim_universal_childcare","would_claim_targeted_childcare","maximum_extended_childcare_hours_usage"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax","household_owns_tv","would_evade_tv_licence_fee","main_residential_property_purchased_is_first_home","property_purchased","brma","owned_land","property_wealth","corporate_wealth","private_pension_wealth","gross_financial_wealth","net_financial_wealth","main_residence_value","other_residential_property_value","non_residential_property_value","savings","num_vehicles","cash_isa","stocks_and_shares_isa","food_and_non_alcoholic_beverages_consumption","alcohol_and_tobacco_consumption","clothing_and_footwear_consumption","housing_water_and_electricity_consumption","household_furnishings_consumption","health_consumption","transport_consumption","communication_consumption","recreation_consumption","education_consumption","restaurants_and_hotels_consumption","miscellaneous_consumption","petrol_spending","diesel_spending","bus_fare_spending","domestic_energy_consumption","electricity_consumption","gas_consumption","has_fuel_consumption","full_rate_vat_expenditure_rate","dfe_education_spending","rail_subsidy_spending","bus_subsidy_spending","rail_usage"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","employee_pension_contributions","pension_contributions_via_salary_sacrifice","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","universal_credit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported","employment_status","employment_sector","sic_industry_division","aa_category","dla_sc_category","dla_m_category","pip_m_category","pip_dl_category","is_disabled_for_benefits","is_enhanced_disabled_for_benefits","is_severely_disabled_for_benefits","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right","legacy_jobseeker_proxy","esa_health_condition_proxy","esa_support_group_proxy","disabled_students_allowance_eligible_expenses","would_claim_marriage_allowance","would_claim_scp","attends_private_school_random_draw","student_loan_balance","a_and_e_visits","admitted_patient_visits","outpatient_visits","nhs_a_and_e_spending","nhs_admitted_patient_spending","nhs_outpatient_spending","hmrc_spi_pay","hmrc_spi_unemployment_benefit_income","hmrc_spi_incapacity_benefit_income","ossben_identifiable_subset","srp_regular_code5","employer_pension_contributions"],"entity":"person","rows":"all"}],"kernel":"uk.stage.expand.spi_support_channel@1","mass":"declared","outputs":[],"params":{"expand_cells":[["person","person_source_id","int64"],["person","person_support_channel","string"],["person","person_support_clone_index","int64"],["benunit","benunit_source_id","int64"],["benunit","benunit_support_channel","string"],["benunit","benunit_support_clone_index","int64"],["household","source_household_id","int64"],["household","source_year","int64"],["household","source_household_key","string"],["household","household_source_id","int64"],["household","household_support_channel","string"],["household","household_support_clone_index","int64"],["household","household_is_spi_synthetic","bool"]],"expand_weight_entity":"household","expand_weight_kind":"importance","stage":"spi_support_channel","stage_contract_sha256":"4b10f1a4a215cdf2c406c9974de2d1e580eb3cd3a25ed34973c1b63abf4a54bb","time_period":"2024"},"population":null,"sources":["frs"],"structural":"expand","weights":null},{"base":null,"citation":"","description":"Own the cells materialized by spi_support_channel.","id":"spi_support_channel.owned","inputs":[],"kernel":"uk.claim@1","mass":"conserve","outputs":[{"column":"person_source_id","dtype":"int64","entity":"person","ownership":"produced","rows":"all"},{"column":"person_support_channel","dtype":"string","entity":"person","ownership":"produced","rows":"all"},{"column":"person_support_clone_index","dtype":"int64","entity":"person","ownership":"produced","rows":"all"},{"column":"benunit_source_id","dtype":"int64","entity":"benunit","ownership":"produced","rows":"all"},{"column":"benunit_support_channel","dtype":"string","entity":"benunit","ownership":"produced","rows":"all"},{"column":"benunit_support_clone_index","dtype":"int64","entity":"benunit","ownership":"produced","rows":"all"},{"column":"source_household_id","dtype":"int64","entity":"household","ownership":"produced","rows":"all"},{"column":"source_year","dtype":"int64","entity":"household","ownership":"produced","rows":"all"},{"column":"source_household_key","dtype":"string","entity":"household","ownership":"produced","rows":"all"},{"column":"household_source_id","dtype":"int64","entity":"household","ownership":"produced","rows":"all"},{"column":"household_support_channel","dtype":"string","entity":"household","ownership":"produced","rows":"all"},{"column":"household_support_clone_index","dtype":"int64","entity":"household","ownership":"produced","rows":"all"},{"column":"household_is_spi_synthetic","dtype":"bool","entity":"household","ownership":"produced","rows":"all"}],"params":{"materialized_expand_outputs":["person.person_source_id","person.person_support_channel","person.person_support_clone_index","benunit.benunit_source_id","benunit.benunit_support_channel","benunit.benunit_support_clone_index","household.source_household_id","household.source_year","household.source_household_key","household.household_source_id","household.household_support_channel","household.household_support_clone_index","household.household_is_spi_synthetic"]},"population":"spi_support_channel","sources":[],"structural":"none","weights":null},{"base":null,"citation":"","description":"Run UK spine stage hmrc_spi_income_spine.","id":"hmrc_spi_income_spine","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children","would_claim_child_benefit","child_benefit_opts_out","would_claim_pc","would_claim_uc","would_claim_tfc","would_claim_extended_childcare","would_claim_universal_childcare","would_claim_targeted_childcare","maximum_extended_childcare_hours_usage","benunit_source_id","benunit_support_channel","benunit_support_clone_index"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax","household_owns_tv","would_evade_tv_licence_fee","main_residential_property_purchased_is_first_home","property_purchased","brma","owned_land","property_wealth","corporate_wealth","private_pension_wealth","gross_financial_wealth","net_financial_wealth","main_residence_value","other_residential_property_value","non_residential_property_value","savings","num_vehicles","cash_isa","stocks_and_shares_isa","food_and_non_alcoholic_beverages_consumption","alcohol_and_tobacco_consumption","clothing_and_footwear_consumption","housing_water_and_electricity_consumption","household_furnishings_consumption","health_consumption","transport_consumption","communication_consumption","recreation_consumption","education_consumption","restaurants_and_hotels_consumption","miscellaneous_consumption","petrol_spending","diesel_spending","bus_fare_spending","domestic_energy_consumption","electricity_consumption","gas_consumption","has_fuel_consumption","full_rate_vat_expenditure_rate","dfe_education_spending","rail_subsidy_spending","bus_subsidy_spending","rail_usage","source_household_id","source_year","source_household_key","household_source_id","household_support_channel","household_support_clone_index","household_is_spi_synthetic"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","maintenance_expenses","childcare_expenses","salary_sacrifice_reported","salary_sacrifice_asked","ssmg_reported","incapacity_benefit_reported","employment_status","employment_sector","sic_industry_division","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right","legacy_jobseeker_proxy","esa_health_condition_proxy","esa_support_group_proxy","disabled_students_allowance_eligible_expenses","would_claim_marriage_allowance","would_claim_scp","attends_private_school_random_draw","student_loan_balance","a_and_e_visits","admitted_patient_visits","outpatient_visits","nhs_a_and_e_spending","nhs_admitted_patient_spending","nhs_outpatient_spending","ossben_identifiable_subset","srp_regular_code5","person_source_id","person_support_channel","person_support_clone_index"],"entity":"person","rows":"all"}],"kernel":"uk.stage.hmrc_spi_income_spine@1","mass":"conserve","outputs":[{"column":"charitable_investment_gifts","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"gift_aid","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"other_investment_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"hmrc_spi_employment_benefits","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"hmrc_spi_employment_expenses","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"hmrc_spi_other_social_security_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"hmrc_spi_taxable_termination_pay","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"hmrc_spi_miscellaneous_employment_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"hmrc_spi_other_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"hmrc_spi_state_pension_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"hmrc_spi_employed_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"hmrc_spi_total_earned_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"hmrc_spi_total_investment_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"hmrc_spi_assessable_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"employment_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"self_employment_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"savings_interest_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"dividend_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"private_pension_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"property_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"employee_pension_contributions","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"employer_pension_contributions","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"personal_pension_contributions","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"pension_contributions_via_salary_sacrifice","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"tax_free_savings_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"universal_credit_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"pension_credit_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"child_benefit_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"housing_benefit_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"income_support_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"working_tax_credit_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"child_tax_credit_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"attendance_allowance_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"state_pension_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"dla_sc_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"dla_m_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"pip_m_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"pip_dl_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"sda_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"carers_allowance_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"iidb_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"afcs_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"bsp_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"winter_fuel_allowance_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"council_tax_benefit_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"jsa_contrib_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"jsa_income_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"esa_contrib_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"esa_income_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"hmrc_spi_pay","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"hmrc_spi_unemployment_benefit_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"hmrc_spi_incapacity_benefit_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"aa_category","dtype":"string","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"dla_sc_category","dtype":"string","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"dla_m_category","dtype":"string","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"pip_m_category","dtype":"string","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"pip_dl_category","dtype":"string","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"is_disabled_for_benefits","dtype":"bool","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"is_enhanced_disabled_for_benefits","dtype":"bool","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"is_severely_disabled_for_benefits","dtype":"bool","entity":"person","ownership":"produced","rewrite":true,"rows":"all"}],"params":{"stage":"hmrc_spi_income_spine","stage_contract_sha256":"3db4e7361ea871ca57b52aafce48209409d7b59d836ed973eee946b5657a771a","time_period":"2024"},"population":"spi_support_channel","sources":["frs"],"structural":"none","weights":null},{"base":"spi_support_channel","citation":"","description":"Ownership boundary before uc_reporter_redraw rewrites.","id":"uc_reporter_redraw.boundary","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children","would_claim_child_benefit","child_benefit_opts_out","would_claim_pc","would_claim_uc","would_claim_tfc","would_claim_extended_childcare","would_claim_universal_childcare","would_claim_targeted_childcare","maximum_extended_childcare_hours_usage","benunit_source_id","benunit_support_channel","benunit_support_clone_index"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax","household_owns_tv","would_evade_tv_licence_fee","main_residential_property_purchased_is_first_home","property_purchased","brma","owned_land","property_wealth","corporate_wealth","private_pension_wealth","gross_financial_wealth","net_financial_wealth","main_residence_value","other_residential_property_value","non_residential_property_value","savings","num_vehicles","cash_isa","stocks_and_shares_isa","food_and_non_alcoholic_beverages_consumption","alcohol_and_tobacco_consumption","clothing_and_footwear_consumption","housing_water_and_electricity_consumption","household_furnishings_consumption","health_consumption","transport_consumption","communication_consumption","recreation_consumption","education_consumption","restaurants_and_hotels_consumption","miscellaneous_consumption","petrol_spending","diesel_spending","bus_fare_spending","domestic_energy_consumption","electricity_consumption","gas_consumption","has_fuel_consumption","full_rate_vat_expenditure_rate","dfe_education_spending","rail_subsidy_spending","bus_subsidy_spending","rail_usage","source_household_id","source_year","source_household_key","household_source_id","household_support_channel","household_support_clone_index","household_is_spi_synthetic"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","employee_pension_contributions","pension_contributions_via_salary_sacrifice","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","universal_credit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported","employment_status","employment_sector","sic_industry_division","aa_category","dla_sc_category","dla_m_category","pip_m_category","pip_dl_category","is_disabled_for_benefits","is_enhanced_disabled_for_benefits","is_severely_disabled_for_benefits","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right","legacy_jobseeker_proxy","esa_health_condition_proxy","esa_support_group_proxy","disabled_students_allowance_eligible_expenses","would_claim_marriage_allowance","would_claim_scp","attends_private_school_random_draw","student_loan_balance","a_and_e_visits","admitted_patient_visits","outpatient_visits","nhs_a_and_e_spending","nhs_admitted_patient_spending","nhs_outpatient_spending","hmrc_spi_pay","hmrc_spi_unemployment_benefit_income","hmrc_spi_incapacity_benefit_income","ossben_identifiable_subset","srp_regular_code5","employer_pension_contributions","person_source_id","person_support_channel","person_support_clone_index","charitable_investment_gifts","gift_aid","other_investment_income","hmrc_spi_employment_benefits","hmrc_spi_employment_expenses","hmrc_spi_other_social_security_income","hmrc_spi_taxable_termination_pay","hmrc_spi_miscellaneous_employment_income","hmrc_spi_other_income","hmrc_spi_state_pension_income","hmrc_spi_employed_income","hmrc_spi_total_earned_income","hmrc_spi_total_investment_income","hmrc_spi_assessable_income"],"entity":"person","rows":"all"}],"kernel":"uk.identity@1","mass":"conserve","outputs":[],"params":{},"population":null,"sources":[],"structural":"filter","weights":null},{"base":null,"citation":"","description":"Run UK spine stage uc_reporter_redraw.","id":"uc_reporter_redraw","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children","would_claim_child_benefit","child_benefit_opts_out","would_claim_pc","would_claim_uc","would_claim_tfc","would_claim_extended_childcare","would_claim_universal_childcare","would_claim_targeted_childcare","maximum_extended_childcare_hours_usage","benunit_source_id","benunit_support_channel","benunit_support_clone_index"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax","household_owns_tv","would_evade_tv_licence_fee","main_residential_property_purchased_is_first_home","property_purchased","brma","owned_land","property_wealth","corporate_wealth","private_pension_wealth","gross_financial_wealth","net_financial_wealth","main_residence_value","other_residential_property_value","non_residential_property_value","savings","num_vehicles","cash_isa","stocks_and_shares_isa","food_and_non_alcoholic_beverages_consumption","alcohol_and_tobacco_consumption","clothing_and_footwear_consumption","housing_water_and_electricity_consumption","household_furnishings_consumption","health_consumption","transport_consumption","communication_consumption","recreation_consumption","education_consumption","restaurants_and_hotels_consumption","miscellaneous_consumption","petrol_spending","diesel_spending","bus_fare_spending","domestic_energy_consumption","electricity_consumption","gas_consumption","has_fuel_consumption","full_rate_vat_expenditure_rate","dfe_education_spending","rail_subsidy_spending","bus_subsidy_spending","rail_usage","source_household_id","source_year","source_household_key","household_source_id","household_support_channel","household_support_clone_index","household_is_spi_synthetic"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","employee_pension_contributions","pension_contributions_via_salary_sacrifice","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported","employment_status","employment_sector","sic_industry_division","aa_category","dla_sc_category","dla_m_category","pip_m_category","pip_dl_category","is_disabled_for_benefits","is_enhanced_disabled_for_benefits","is_severely_disabled_for_benefits","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right","legacy_jobseeker_proxy","esa_health_condition_proxy","esa_support_group_proxy","disabled_students_allowance_eligible_expenses","would_claim_marriage_allowance","would_claim_scp","attends_private_school_random_draw","student_loan_balance","a_and_e_visits","admitted_patient_visits","outpatient_visits","nhs_a_and_e_spending","nhs_admitted_patient_spending","nhs_outpatient_spending","hmrc_spi_pay","hmrc_spi_unemployment_benefit_income","hmrc_spi_incapacity_benefit_income","ossben_identifiable_subset","srp_regular_code5","employer_pension_contributions","person_source_id","person_support_channel","person_support_clone_index","charitable_investment_gifts","gift_aid","other_investment_income","hmrc_spi_employment_benefits","hmrc_spi_employment_expenses","hmrc_spi_other_social_security_income","hmrc_spi_taxable_termination_pay","hmrc_spi_miscellaneous_employment_income","hmrc_spi_other_income","hmrc_spi_state_pension_income","hmrc_spi_employed_income","hmrc_spi_total_earned_income","hmrc_spi_total_investment_income","hmrc_spi_assessable_income"],"entity":"person","rows":"all"}],"kernel":"uk.stage.uc_reporter_redraw@1","mass":"conserve","outputs":[{"column":"universal_credit_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"}],"params":{"stage":"uc_reporter_redraw","stage_contract_sha256":"23a23df85489708d15c7c0be3d1339ff5e1fe0932abaea0f95fcbf9c6d368205","time_period":"2024"},"population":"uc_reporter_redraw.boundary","sources":["frs"],"structural":"none","weights":null},{"base":"uc_reporter_redraw.boundary","citation":"","description":"Ownership boundary before uc_capital_coherence rewrites.","id":"uc_capital_coherence.boundary","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children","would_claim_child_benefit","child_benefit_opts_out","would_claim_pc","would_claim_uc","would_claim_tfc","would_claim_extended_childcare","would_claim_universal_childcare","would_claim_targeted_childcare","maximum_extended_childcare_hours_usage","benunit_source_id","benunit_support_channel","benunit_support_clone_index"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax","household_owns_tv","would_evade_tv_licence_fee","main_residential_property_purchased_is_first_home","property_purchased","brma","owned_land","property_wealth","corporate_wealth","private_pension_wealth","gross_financial_wealth","net_financial_wealth","main_residence_value","other_residential_property_value","non_residential_property_value","savings","num_vehicles","cash_isa","stocks_and_shares_isa","food_and_non_alcoholic_beverages_consumption","alcohol_and_tobacco_consumption","clothing_and_footwear_consumption","housing_water_and_electricity_consumption","household_furnishings_consumption","health_consumption","transport_consumption","communication_consumption","recreation_consumption","education_consumption","restaurants_and_hotels_consumption","miscellaneous_consumption","petrol_spending","diesel_spending","bus_fare_spending","domestic_energy_consumption","electricity_consumption","gas_consumption","has_fuel_consumption","full_rate_vat_expenditure_rate","dfe_education_spending","rail_subsidy_spending","bus_subsidy_spending","rail_usage","source_household_id","source_year","source_household_key","household_source_id","household_support_channel","household_support_clone_index","household_is_spi_synthetic"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","employee_pension_contributions","pension_contributions_via_salary_sacrifice","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","universal_credit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported","employment_status","employment_sector","sic_industry_division","aa_category","dla_sc_category","dla_m_category","pip_m_category","pip_dl_category","is_disabled_for_benefits","is_enhanced_disabled_for_benefits","is_severely_disabled_for_benefits","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right","legacy_jobseeker_proxy","esa_health_condition_proxy","esa_support_group_proxy","disabled_students_allowance_eligible_expenses","would_claim_marriage_allowance","would_claim_scp","attends_private_school_random_draw","student_loan_balance","a_and_e_visits","admitted_patient_visits","outpatient_visits","nhs_a_and_e_spending","nhs_admitted_patient_spending","nhs_outpatient_spending","hmrc_spi_pay","hmrc_spi_unemployment_benefit_income","hmrc_spi_incapacity_benefit_income","ossben_identifiable_subset","srp_regular_code5","employer_pension_contributions","person_source_id","person_support_channel","person_support_clone_index","charitable_investment_gifts","gift_aid","other_investment_income","hmrc_spi_employment_benefits","hmrc_spi_employment_expenses","hmrc_spi_other_social_security_income","hmrc_spi_taxable_termination_pay","hmrc_spi_miscellaneous_employment_income","hmrc_spi_other_income","hmrc_spi_state_pension_income","hmrc_spi_employed_income","hmrc_spi_total_earned_income","hmrc_spi_total_investment_income","hmrc_spi_assessable_income"],"entity":"person","rows":"all"}],"kernel":"uk.identity@1","mass":"conserve","outputs":[],"params":{},"population":null,"sources":[],"structural":"filter","weights":null},{"base":null,"citation":"","description":"Run UK spine stage uc_capital_coherence.","id":"uc_capital_coherence","inputs":[{"columns":["benunit_support_channel","dependent_children","is_married"],"entity":"benunit","rows":"all"},{"columns":["region"],"entity":"household","rows":"all"},{"columns":["age","person_support_channel","universal_credit_reported"],"entity":"person","rows":"all"}],"kernel":"uk.stage.uc_capital_coherence@1","mass":"conserve","outputs":[{"column":"uc_reported_capital","dtype":"float64","entity":"benunit","ownership":"produced","rows":"all"},{"column":"frs_benunit_capital","dtype":"float64","entity":"benunit","ownership":"produced","rewrite":true,"rows":"all"},{"column":"would_claim_uc","dtype":"bool","entity":"benunit","ownership":"produced","rewrite":true,"rows":"all"}],"params":{"stage":"uc_capital_coherence","stage_contract_sha256":"64024e4ca8659cad2452abe5578071cf8226e693f3cb975cf4778078bf8baf17","time_period":"2024"},"population":"uc_capital_coherence.boundary","sources":["frs"],"structural":"none","weights":null},{"base":"uc_capital_coherence.boundary","citation":"","description":"Run structural UK stage cgt_incidence_clone.","id":"cgt_incidence_clone","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children","would_claim_child_benefit","child_benefit_opts_out","would_claim_pc","would_claim_uc","would_claim_tfc","would_claim_extended_childcare","would_claim_universal_childcare","would_claim_targeted_childcare","maximum_extended_childcare_hours_usage","benunit_source_id","benunit_support_channel","benunit_support_clone_index","uc_reported_capital"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax","household_owns_tv","would_evade_tv_licence_fee","main_residential_property_purchased_is_first_home","property_purchased","brma","owned_land","property_wealth","corporate_wealth","private_pension_wealth","gross_financial_wealth","net_financial_wealth","main_residence_value","other_residential_property_value","non_residential_property_value","savings","num_vehicles","cash_isa","stocks_and_shares_isa","food_and_non_alcoholic_beverages_consumption","alcohol_and_tobacco_consumption","clothing_and_footwear_consumption","housing_water_and_electricity_consumption","household_furnishings_consumption","health_consumption","transport_consumption","communication_consumption","recreation_consumption","education_consumption","restaurants_and_hotels_consumption","miscellaneous_consumption","petrol_spending","diesel_spending","bus_fare_spending","domestic_energy_consumption","electricity_consumption","gas_consumption","has_fuel_consumption","full_rate_vat_expenditure_rate","dfe_education_spending","rail_subsidy_spending","bus_subsidy_spending","rail_usage","source_household_id","source_year","source_household_key","household_source_id","household_support_channel","household_support_clone_index","household_is_spi_synthetic"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","employee_pension_contributions","pension_contributions_via_salary_sacrifice","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","universal_credit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported","employment_status","employment_sector","sic_industry_division","aa_category","dla_sc_category","dla_m_category","pip_m_category","pip_dl_category","is_disabled_for_benefits","is_enhanced_disabled_for_benefits","is_severely_disabled_for_benefits","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right","legacy_jobseeker_proxy","esa_health_condition_proxy","esa_support_group_proxy","disabled_students_allowance_eligible_expenses","would_claim_marriage_allowance","would_claim_scp","attends_private_school_random_draw","student_loan_balance","a_and_e_visits","admitted_patient_visits","outpatient_visits","nhs_a_and_e_spending","nhs_admitted_patient_spending","nhs_outpatient_spending","hmrc_spi_pay","hmrc_spi_unemployment_benefit_income","hmrc_spi_incapacity_benefit_income","ossben_identifiable_subset","srp_regular_code5","employer_pension_contributions","person_source_id","person_support_channel","person_support_clone_index","charitable_investment_gifts","gift_aid","other_investment_income","hmrc_spi_employment_benefits","hmrc_spi_employment_expenses","hmrc_spi_other_social_security_income","hmrc_spi_taxable_termination_pay","hmrc_spi_miscellaneous_employment_income","hmrc_spi_other_income","hmrc_spi_state_pension_income","hmrc_spi_employed_income","hmrc_spi_total_earned_income","hmrc_spi_total_investment_income","hmrc_spi_assessable_income"],"entity":"person","rows":"all"}],"kernel":"uk.stage.expand.cgt_incidence_clone@1","mass":"conserve","outputs":[],"params":{"expand_cells":[["household","household_is_capital_gains_clone","bool"],["person","capital_gains","float64"]],"expand_weight_entity":"household","expand_weight_kind":"importance","stage":"cgt_incidence_clone","stage_contract_sha256":"ee30278543cc0297a5366d855b7753aa04af5518971594c193c52be36bf8a7b4","time_period":"2024"},"population":null,"sources":["frs"],"structural":"expand","weights":null},{"base":null,"citation":"","description":"Own the cells materialized by cgt_incidence_clone.","id":"cgt_incidence_clone.owned","inputs":[],"kernel":"uk.claim@1","mass":"conserve","outputs":[{"column":"household_is_capital_gains_clone","dtype":"bool","entity":"household","ownership":"produced","rows":"all"},{"column":"capital_gains","dtype":"float64","entity":"person","ownership":"produced","rows":"all"}],"params":{"materialized_expand_outputs":["household.household_is_capital_gains_clone","person.capital_gains"]},"population":"cgt_incidence_clone","sources":[],"structural":"none","weights":null},{"base":"cgt_incidence_clone","citation":"","description":"Run structural UK stage cgt_band_donors.","id":"cgt_band_donors","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children","would_claim_child_benefit","child_benefit_opts_out","would_claim_pc","would_claim_uc","would_claim_tfc","would_claim_extended_childcare","would_claim_universal_childcare","would_claim_targeted_childcare","maximum_extended_childcare_hours_usage","benunit_source_id","benunit_support_channel","benunit_support_clone_index","uc_reported_capital"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax","household_owns_tv","would_evade_tv_licence_fee","main_residential_property_purchased_is_first_home","property_purchased","brma","owned_land","property_wealth","corporate_wealth","private_pension_wealth","gross_financial_wealth","net_financial_wealth","main_residence_value","other_residential_property_value","non_residential_property_value","savings","num_vehicles","cash_isa","stocks_and_shares_isa","food_and_non_alcoholic_beverages_consumption","alcohol_and_tobacco_consumption","clothing_and_footwear_consumption","housing_water_and_electricity_consumption","household_furnishings_consumption","health_consumption","transport_consumption","communication_consumption","recreation_consumption","education_consumption","restaurants_and_hotels_consumption","miscellaneous_consumption","petrol_spending","diesel_spending","bus_fare_spending","domestic_energy_consumption","electricity_consumption","gas_consumption","has_fuel_consumption","full_rate_vat_expenditure_rate","dfe_education_spending","rail_subsidy_spending","bus_subsidy_spending","rail_usage","source_household_id","source_year","source_household_key","household_source_id","household_support_channel","household_support_clone_index","household_is_spi_synthetic","household_is_capital_gains_clone"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","employee_pension_contributions","pension_contributions_via_salary_sacrifice","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","universal_credit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported","employment_status","employment_sector","sic_industry_division","aa_category","dla_sc_category","dla_m_category","pip_m_category","pip_dl_category","is_disabled_for_benefits","is_enhanced_disabled_for_benefits","is_severely_disabled_for_benefits","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right","legacy_jobseeker_proxy","esa_health_condition_proxy","esa_support_group_proxy","disabled_students_allowance_eligible_expenses","would_claim_marriage_allowance","would_claim_scp","attends_private_school_random_draw","student_loan_balance","a_and_e_visits","admitted_patient_visits","outpatient_visits","nhs_a_and_e_spending","nhs_admitted_patient_spending","nhs_outpatient_spending","hmrc_spi_pay","hmrc_spi_unemployment_benefit_income","hmrc_spi_incapacity_benefit_income","ossben_identifiable_subset","srp_regular_code5","employer_pension_contributions","person_source_id","person_support_channel","person_support_clone_index","charitable_investment_gifts","gift_aid","other_investment_income","hmrc_spi_employment_benefits","hmrc_spi_employment_expenses","hmrc_spi_other_social_security_income","hmrc_spi_taxable_termination_pay","hmrc_spi_miscellaneous_employment_income","hmrc_spi_other_income","hmrc_spi_state_pension_income","hmrc_spi_employed_income","hmrc_spi_total_earned_income","hmrc_spi_total_investment_income","hmrc_spi_assessable_income","capital_gains"],"entity":"person","rows":"all"}],"kernel":"uk.stage.expand.cgt_band_donors@1","mass":"free","outputs":[],"params":{"expand_cells":[["household","household_is_cgt_band_donor","bool"],["person","capital_gains","float64"]],"expand_weight_entity":"household","expand_weight_kind":"importance","stage":"cgt_band_donors","stage_contract_sha256":"e10e10c49c0ca2a6a65048c91b683f818e8b6e207a6c6f57a20d76ea299160b0","time_period":"2024"},"population":null,"sources":["frs"],"structural":"expand","weights":null},{"base":null,"citation":"","description":"Own the cells materialized by cgt_band_donors.","id":"cgt_band_donors.owned","inputs":[],"kernel":"uk.claim@1","mass":"conserve","outputs":[{"column":"household_is_cgt_band_donor","dtype":"bool","entity":"household","ownership":"produced","rows":"all"},{"column":"capital_gains","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"}],"params":{"materialized_expand_outputs":["household.household_is_cgt_band_donor"]},"population":"cgt_band_donors","sources":[],"structural":"none","weights":null},{"base":"cgt_band_donors","citation":"","description":"Ownership boundary before hmrc_cgt_gains_spine rewrites.","id":"hmrc_cgt_gains_spine.boundary","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children","would_claim_child_benefit","child_benefit_opts_out","would_claim_pc","would_claim_uc","would_claim_tfc","would_claim_extended_childcare","would_claim_universal_childcare","would_claim_targeted_childcare","maximum_extended_childcare_hours_usage","benunit_source_id","benunit_support_channel","benunit_support_clone_index","uc_reported_capital"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax","household_owns_tv","would_evade_tv_licence_fee","main_residential_property_purchased_is_first_home","property_purchased","brma","owned_land","property_wealth","corporate_wealth","private_pension_wealth","gross_financial_wealth","net_financial_wealth","main_residence_value","other_residential_property_value","non_residential_property_value","savings","num_vehicles","cash_isa","stocks_and_shares_isa","food_and_non_alcoholic_beverages_consumption","alcohol_and_tobacco_consumption","clothing_and_footwear_consumption","housing_water_and_electricity_consumption","household_furnishings_consumption","health_consumption","transport_consumption","communication_consumption","recreation_consumption","education_consumption","restaurants_and_hotels_consumption","miscellaneous_consumption","petrol_spending","diesel_spending","bus_fare_spending","domestic_energy_consumption","electricity_consumption","gas_consumption","has_fuel_consumption","full_rate_vat_expenditure_rate","dfe_education_spending","rail_subsidy_spending","bus_subsidy_spending","rail_usage","source_household_id","source_year","source_household_key","household_source_id","household_support_channel","household_support_clone_index","household_is_spi_synthetic","household_is_capital_gains_clone","household_is_cgt_band_donor"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","employee_pension_contributions","pension_contributions_via_salary_sacrifice","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","universal_credit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported","employment_status","employment_sector","sic_industry_division","aa_category","dla_sc_category","dla_m_category","pip_m_category","pip_dl_category","is_disabled_for_benefits","is_enhanced_disabled_for_benefits","is_severely_disabled_for_benefits","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right","legacy_jobseeker_proxy","esa_health_condition_proxy","esa_support_group_proxy","disabled_students_allowance_eligible_expenses","would_claim_marriage_allowance","would_claim_scp","attends_private_school_random_draw","student_loan_balance","a_and_e_visits","admitted_patient_visits","outpatient_visits","nhs_a_and_e_spending","nhs_admitted_patient_spending","nhs_outpatient_spending","hmrc_spi_pay","hmrc_spi_unemployment_benefit_income","hmrc_spi_incapacity_benefit_income","ossben_identifiable_subset","srp_regular_code5","employer_pension_contributions","person_source_id","person_support_channel","person_support_clone_index","charitable_investment_gifts","gift_aid","other_investment_income","hmrc_spi_employment_benefits","hmrc_spi_employment_expenses","hmrc_spi_other_social_security_income","hmrc_spi_taxable_termination_pay","hmrc_spi_miscellaneous_employment_income","hmrc_spi_other_income","hmrc_spi_state_pension_income","hmrc_spi_employed_income","hmrc_spi_total_earned_income","hmrc_spi_total_investment_income","hmrc_spi_assessable_income","capital_gains"],"entity":"person","rows":"all"}],"kernel":"uk.identity@1","mass":"conserve","outputs":[],"params":{},"population":null,"sources":[],"structural":"filter","weights":null},{"base":null,"citation":"","description":"Run UK spine stage hmrc_cgt_gains_spine.","id":"hmrc_cgt_gains_spine","inputs":[{"columns":["frs_benunit_capital"],"entity":"benunit","rows":"all"},{"columns":["region"],"entity":"household","rows":"all"},{"columns":["age","dividend_income","employment_income","miscellaneous_income","private_pension_income","property_income","savings_interest_income","self_employment_income","state_pension_reported","tax_free_savings_income"],"entity":"person","rows":"all"}],"kernel":"uk.stage.hmrc_cgt_gains_spine@1","mass":"conserve","outputs":[{"column":"capital_gains","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"}],"params":{"stage":"hmrc_cgt_gains_spine","stage_contract_sha256":"20ada8d8ee94400bd160223e865a910db7aff9c40a7f0c8ee8ca8c40ed901b72","time_period":"2024"},"population":"hmrc_cgt_gains_spine.boundary","sources":["frs"],"structural":"none","weights":null},{"base":null,"citation":"","description":"Run UK spine stage salary_sacrifice.","id":"salary_sacrifice","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children","would_claim_child_benefit","child_benefit_opts_out","would_claim_pc","would_claim_uc","would_claim_tfc","would_claim_extended_childcare","would_claim_universal_childcare","would_claim_targeted_childcare","maximum_extended_childcare_hours_usage","benunit_source_id","benunit_support_channel","benunit_support_clone_index","uc_reported_capital"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax","household_owns_tv","would_evade_tv_licence_fee","main_residential_property_purchased_is_first_home","property_purchased","brma","owned_land","property_wealth","corporate_wealth","private_pension_wealth","gross_financial_wealth","net_financial_wealth","main_residence_value","other_residential_property_value","non_residential_property_value","savings","num_vehicles","cash_isa","stocks_and_shares_isa","food_and_non_alcoholic_beverages_consumption","alcohol_and_tobacco_consumption","clothing_and_footwear_consumption","housing_water_and_electricity_consumption","household_furnishings_consumption","health_consumption","transport_consumption","communication_consumption","recreation_consumption","education_consumption","restaurants_and_hotels_consumption","miscellaneous_consumption","petrol_spending","diesel_spending","bus_fare_spending","domestic_energy_consumption","electricity_consumption","gas_consumption","has_fuel_consumption","full_rate_vat_expenditure_rate","dfe_education_spending","rail_subsidy_spending","bus_subsidy_spending","rail_usage","source_household_id","source_year","source_household_key","household_source_id","household_support_channel","household_support_clone_index","household_is_spi_synthetic","household_is_capital_gains_clone","household_is_cgt_band_donor"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","universal_credit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported","employment_status","employment_sector","sic_industry_division","aa_category","dla_sc_category","dla_m_category","pip_m_category","pip_dl_category","is_disabled_for_benefits","is_enhanced_disabled_for_benefits","is_severely_disabled_for_benefits","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right","legacy_jobseeker_proxy","esa_health_condition_proxy","esa_support_group_proxy","disabled_students_allowance_eligible_expenses","would_claim_marriage_allowance","would_claim_scp","attends_private_school_random_draw","student_loan_balance","a_and_e_visits","admitted_patient_visits","outpatient_visits","nhs_a_and_e_spending","nhs_admitted_patient_spending","nhs_outpatient_spending","hmrc_spi_pay","hmrc_spi_unemployment_benefit_income","hmrc_spi_incapacity_benefit_income","ossben_identifiable_subset","srp_regular_code5","employer_pension_contributions","person_source_id","person_support_channel","person_support_clone_index","charitable_investment_gifts","gift_aid","other_investment_income","hmrc_spi_employment_benefits","hmrc_spi_employment_expenses","hmrc_spi_other_social_security_income","hmrc_spi_taxable_termination_pay","hmrc_spi_miscellaneous_employment_income","hmrc_spi_other_income","hmrc_spi_state_pension_income","hmrc_spi_employed_income","hmrc_spi_total_earned_income","hmrc_spi_total_investment_income","hmrc_spi_assessable_income","capital_gains"],"entity":"person","rows":"all"}],"kernel":"uk.stage.salary_sacrifice@1","mass":"conserve","outputs":[{"column":"pension_contributions_via_salary_sacrifice","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"employee_pension_contributions","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"}],"params":{"stage":"salary_sacrifice","stage_contract_sha256":"a4af925af21cc6766eb7c8855b6743a3aa4eeeb38df12cfc83063d7ad6890539","time_period":"2024"},"population":"hmrc_cgt_gains_spine.boundary","sources":["frs"],"structural":"none","weights":null},{"base":null,"citation":"","description":"Run UK spine stage student_loans.","id":"student_loans","inputs":[{"columns":["frs_benunit_capital"],"entity":"benunit","rows":"all"},{"columns":["region"],"entity":"household","rows":"all"},{"columns":["age","current_education","employee_pension_contributions","highest_education","student_loan_repayments","student_loans"],"entity":"person","rows":"all"}],"kernel":"uk.stage.student_loans@1","mass":"conserve","outputs":[{"column":"student_loan_plan","dtype":"string","entity":"person","ownership":"produced","rows":"all"}],"params":{"stage":"student_loans","stage_contract_sha256":"7c4c76398c2e80b41c7f941a5f06246b57d89b5eb0c36d42e3116ac35e82e203","time_period":"2024"},"population":"hmrc_cgt_gains_spine.boundary","sources":["frs"],"structural":"none","weights":null},{"base":"hmrc_cgt_gains_spine.boundary","citation":"","description":"Ownership boundary before age_tail rewrites.","id":"age_tail.boundary","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children","would_claim_child_benefit","child_benefit_opts_out","would_claim_pc","would_claim_uc","would_claim_tfc","would_claim_extended_childcare","would_claim_universal_childcare","would_claim_targeted_childcare","maximum_extended_childcare_hours_usage","benunit_source_id","benunit_support_channel","benunit_support_clone_index","uc_reported_capital"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax","household_owns_tv","would_evade_tv_licence_fee","main_residential_property_purchased_is_first_home","property_purchased","brma","owned_land","property_wealth","corporate_wealth","private_pension_wealth","gross_financial_wealth","net_financial_wealth","main_residence_value","other_residential_property_value","non_residential_property_value","savings","num_vehicles","cash_isa","stocks_and_shares_isa","food_and_non_alcoholic_beverages_consumption","alcohol_and_tobacco_consumption","clothing_and_footwear_consumption","housing_water_and_electricity_consumption","household_furnishings_consumption","health_consumption","transport_consumption","communication_consumption","recreation_consumption","education_consumption","restaurants_and_hotels_consumption","miscellaneous_consumption","petrol_spending","diesel_spending","bus_fare_spending","domestic_energy_consumption","electricity_consumption","gas_consumption","has_fuel_consumption","full_rate_vat_expenditure_rate","dfe_education_spending","rail_subsidy_spending","bus_subsidy_spending","rail_usage","source_household_id","source_year","source_household_key","household_source_id","household_support_channel","household_support_clone_index","household_is_spi_synthetic","household_is_capital_gains_clone","household_is_cgt_band_donor"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","employee_pension_contributions","pension_contributions_via_salary_sacrifice","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","universal_credit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported","employment_status","employment_sector","sic_industry_division","aa_category","dla_sc_category","dla_m_category","pip_m_category","pip_dl_category","is_disabled_for_benefits","is_enhanced_disabled_for_benefits","is_severely_disabled_for_benefits","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right","legacy_jobseeker_proxy","esa_health_condition_proxy","esa_support_group_proxy","disabled_students_allowance_eligible_expenses","would_claim_marriage_allowance","would_claim_scp","attends_private_school_random_draw","student_loan_balance","a_and_e_visits","admitted_patient_visits","outpatient_visits","nhs_a_and_e_spending","nhs_admitted_patient_spending","nhs_outpatient_spending","hmrc_spi_pay","hmrc_spi_unemployment_benefit_income","hmrc_spi_incapacity_benefit_income","ossben_identifiable_subset","srp_regular_code5","employer_pension_contributions","person_source_id","person_support_channel","person_support_clone_index","charitable_investment_gifts","gift_aid","other_investment_income","hmrc_spi_employment_benefits","hmrc_spi_employment_expenses","hmrc_spi_other_social_security_income","hmrc_spi_taxable_termination_pay","hmrc_spi_miscellaneous_employment_income","hmrc_spi_other_income","hmrc_spi_state_pension_income","hmrc_spi_employed_income","hmrc_spi_total_earned_income","hmrc_spi_total_investment_income","hmrc_spi_assessable_income","capital_gains","student_loan_plan"],"entity":"person","rows":"all"}],"kernel":"uk.identity@1","mass":"conserve","outputs":[],"params":{},"population":null,"sources":[],"structural":"filter","weights":null},{"base":null,"citation":"","description":"Run UK spine stage age_tail.","id":"age_tail","inputs":[{"columns":["frs_benunit_capital"],"entity":"benunit","rows":"all"},{"columns":["region"],"entity":"household","rows":"all"},{"columns":["gender","person_source_id","student_loan_plan"],"entity":"person","rows":"all"}],"kernel":"uk.stage.age_tail@1","mass":"conserve","outputs":[{"column":"age","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"}],"params":{"stage":"age_tail","stage_contract_sha256":"5f4fa9cd1c55cae6e61a4c046a249d91a4222fa792a02aa652b5c71dedbdcfee","time_period":"2024"},"population":"age_tail.boundary","sources":["frs"],"structural":"none","weights":null}],"sources":[{"codec":"csv-tables","description":"Content-bound UK FRS and donor fixture/source bundle.","name":"frs"}]} +{"country":"uk","nodes":[{"base":null,"citation":"","description":"Load the source-bound UK FRS root population.","id":"create_uk_frs","inputs":[],"kernel":"uk.create@1","mass":"conserve","outputs":[{"column":"age","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"gender","dtype":"string","entity":"person","ownership":"produced","rows":"all"},{"column":"marital_status","dtype":"string","entity":"person","ownership":"produced","rows":"all"},{"column":"hours_worked","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"is_household_head","dtype":"bool","entity":"person","ownership":"produced","rows":"all"},{"column":"is_benunit_head","dtype":"bool","entity":"person","ownership":"produced","rows":"all"},{"column":"is_parent","dtype":"bool","entity":"person","ownership":"produced","rows":"all"},{"column":"employment_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"self_employment_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"private_pension_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"tax_free_savings_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"savings_interest_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"dividend_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"property_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"maintenance_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"miscellaneous_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"private_transfer_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"lump_sum_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"student_loan_repayments","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"statutory_sick_pay","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"statutory_maternity_pay","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"student_loans","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"access_fund","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"education_grants","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"healthy_start_vouchers","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"free_school_breakfasts","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"free_school_fruit_veg","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"free_school_meals","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"council_tax_benefit_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"maintenance_expenses","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"childcare_expenses","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"personal_pension_contributions","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"employee_pension_contributions","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"pension_contributions_via_salary_sacrifice","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"salary_sacrifice_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"salary_sacrifice_asked","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"child_benefit_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"income_support_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"housing_benefit_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"attendance_allowance_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"dla_sc_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"dla_m_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"iidb_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"carers_allowance_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"sda_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"afcs_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"ssmg_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"pension_credit_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"child_tax_credit_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"working_tax_credit_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"state_pension_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"winter_fuel_allowance_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"incapacity_benefit_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"universal_credit_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"pip_m_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"pip_dl_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"jsa_contrib_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"jsa_income_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"esa_contrib_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"esa_income_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"bsp_reported","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"frs_benunit_capital","dtype":"float64","entity":"benunit","ownership":"produced","rows":"all"},{"column":"is_married","dtype":"bool","entity":"benunit","ownership":"produced","rows":"all"},{"column":"dependent_children","dtype":"int64","entity":"benunit","ownership":"produced","rows":"all"},{"column":"region","dtype":"string","entity":"household","ownership":"produced","rows":"all"},{"column":"tenure_type","dtype":"string","entity":"household","ownership":"produced","rows":"all"},{"column":"accommodation_type","dtype":"string","entity":"household","ownership":"produced","rows":"all"},{"column":"num_bedrooms","dtype":"int64","entity":"household","ownership":"produced","rows":"all"},{"column":"council_tax_reported","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"council_tax_band","dtype":"string","entity":"household","ownership":"produced","rows":"all"},{"column":"council_tax_rebate","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"council_tax_single_adult_raw","dtype":"int64","entity":"household","ownership":"produced","rows":"all"},{"column":"water_and_sewerage_charges","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"domestic_rates","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"rent","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"subrent","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"mortgage_interest_repayment","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"mortgage_capital_repayment","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"structural_insurance_payments","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"housing_service_charges","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"external_child_payments","dtype":"float64","entity":"household","ownership":"produced","rows":"all"}],"params":{"sample_fraction":1.0,"sample_seed":578,"stage_contract_sha256":"dfd331a9aba8fd095ebb16250f90cbb5de2ea999874c22fa3bea6a7e33fa83bf","time_period":"2024"},"population":null,"sources":["frs"],"structural":"create","weights":null},{"base":"create_uk_frs","citation":"","description":"Ownership boundary for the source-assembling root stage.","id":"frs_spine.boundary","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","employee_pension_contributions","pension_contributions_via_salary_sacrifice","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","universal_credit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported"],"entity":"person","rows":"all"}],"kernel":"uk.identity@1","mass":"conserve","outputs":[],"params":{},"population":null,"sources":[],"structural":"filter","weights":null},{"base":null,"citation":"","description":"Claim the cells assembled by the UK FRS root transform.","id":"frs_spine","inputs":[],"kernel":"uk.claim@1","mass":"conserve","outputs":[{"column":"age","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"gender","dtype":"string","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"marital_status","dtype":"string","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"hours_worked","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"is_household_head","dtype":"bool","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"is_benunit_head","dtype":"bool","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"is_parent","dtype":"bool","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"employment_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"self_employment_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"private_pension_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"tax_free_savings_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"savings_interest_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"dividend_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"property_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"maintenance_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"miscellaneous_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"private_transfer_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"lump_sum_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"student_loan_repayments","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"statutory_sick_pay","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"statutory_maternity_pay","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"student_loans","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"access_fund","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"education_grants","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"healthy_start_vouchers","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"free_school_breakfasts","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"free_school_fruit_veg","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"free_school_meals","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"council_tax_benefit_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"maintenance_expenses","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"childcare_expenses","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"personal_pension_contributions","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"employee_pension_contributions","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"pension_contributions_via_salary_sacrifice","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"salary_sacrifice_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"salary_sacrifice_asked","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"child_benefit_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"income_support_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"housing_benefit_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"attendance_allowance_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"dla_sc_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"dla_m_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"iidb_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"carers_allowance_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"sda_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"afcs_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"ssmg_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"pension_credit_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"child_tax_credit_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"working_tax_credit_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"state_pension_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"winter_fuel_allowance_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"incapacity_benefit_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"universal_credit_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"pip_m_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"pip_dl_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"jsa_contrib_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"jsa_income_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"esa_contrib_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"esa_income_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"bsp_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"frs_benunit_capital","dtype":"float64","entity":"benunit","ownership":"produced","rewrite":true,"rows":"all"},{"column":"is_married","dtype":"bool","entity":"benunit","ownership":"produced","rewrite":true,"rows":"all"},{"column":"dependent_children","dtype":"int64","entity":"benunit","ownership":"produced","rewrite":true,"rows":"all"},{"column":"region","dtype":"string","entity":"household","ownership":"produced","rewrite":true,"rows":"all"},{"column":"tenure_type","dtype":"string","entity":"household","ownership":"produced","rewrite":true,"rows":"all"},{"column":"accommodation_type","dtype":"string","entity":"household","ownership":"produced","rewrite":true,"rows":"all"},{"column":"num_bedrooms","dtype":"int64","entity":"household","ownership":"produced","rewrite":true,"rows":"all"},{"column":"council_tax_reported","dtype":"float64","entity":"household","ownership":"produced","rewrite":true,"rows":"all"},{"column":"council_tax_band","dtype":"string","entity":"household","ownership":"produced","rewrite":true,"rows":"all"},{"column":"council_tax_rebate","dtype":"float64","entity":"household","ownership":"produced","rewrite":true,"rows":"all"},{"column":"council_tax_single_adult_raw","dtype":"int64","entity":"household","ownership":"produced","rewrite":true,"rows":"all"},{"column":"water_and_sewerage_charges","dtype":"float64","entity":"household","ownership":"produced","rewrite":true,"rows":"all"},{"column":"domestic_rates","dtype":"float64","entity":"household","ownership":"produced","rewrite":true,"rows":"all"},{"column":"rent","dtype":"float64","entity":"household","ownership":"produced","rewrite":true,"rows":"all"},{"column":"subrent","dtype":"float64","entity":"household","ownership":"produced","rewrite":true,"rows":"all"},{"column":"mortgage_interest_repayment","dtype":"float64","entity":"household","ownership":"produced","rewrite":true,"rows":"all"},{"column":"mortgage_capital_repayment","dtype":"float64","entity":"household","ownership":"produced","rewrite":true,"rows":"all"},{"column":"structural_insurance_payments","dtype":"float64","entity":"household","ownership":"produced","rewrite":true,"rows":"all"},{"column":"housing_service_charges","dtype":"float64","entity":"household","ownership":"produced","rewrite":true,"rows":"all"},{"column":"external_child_payments","dtype":"float64","entity":"household","ownership":"produced","rewrite":true,"rows":"all"}],"params":{},"population":"frs_spine.boundary","sources":[],"structural":"none","weights":null},{"base":null,"citation":"","description":"Run UK spine stage frs_employment.","id":"frs_employment","inputs":[{"columns":["frs_benunit_capital"],"entity":"benunit","rows":"all"},{"columns":["external_child_payments","region"],"entity":"household","rows":"all"},{"columns":["age"],"entity":"person","rows":"all"}],"kernel":"uk.stage.frs_employment@1","mass":"conserve","outputs":[{"column":"employment_status","dtype":"string","entity":"person","ownership":"produced","rows":"all"},{"column":"employment_sector","dtype":"string","entity":"person","ownership":"produced","rows":"all"},{"column":"sic_industry_division","dtype":"int64","entity":"person","ownership":"produced","rows":"all"}],"params":{"stage":"frs_employment","stage_contract_sha256":"ddbefaf05b44788d794a6e4b0e8926c14318d66e50d2f11f50ca549538bbf60c","time_period":"2024"},"population":"frs_spine.boundary","sources":["frs"],"structural":"none","weights":null},{"base":null,"citation":"","description":"Run UK spine stage frs_council_tax.","id":"frs_council_tax","inputs":[{"columns":["frs_benunit_capital"],"entity":"benunit","rows":"all"},{"columns":["region"],"entity":"household","rows":"all"},{"columns":["age","sic_industry_division"],"entity":"person","rows":"all"}],"kernel":"uk.stage.frs_council_tax@1","mass":"conserve","outputs":[{"column":"council_tax","dtype":"float64","entity":"household","ownership":"produced","rows":"all"}],"params":{"stage":"frs_council_tax","stage_contract_sha256":"3381cd9f7a736514d5073c57480e07f5098890f3ca9ed3865392043594771eb9","time_period":"2024"},"population":"frs_spine.boundary","sources":["frs"],"structural":"none","weights":null},{"base":null,"citation":"","description":"Run UK spine stage frs_disability.","id":"frs_disability","inputs":[{"columns":["frs_benunit_capital"],"entity":"benunit","rows":"all"},{"columns":["council_tax","region"],"entity":"household","rows":"all"},{"columns":["afcs_reported","age","attendance_allowance_reported","dla_m_reported","dla_sc_reported","esa_contrib_reported","esa_income_reported","iidb_reported","incapacity_benefit_reported","pip_dl_reported","pip_m_reported","sda_reported"],"entity":"person","rows":"all"}],"kernel":"uk.stage.frs_disability@1","mass":"conserve","outputs":[{"column":"aa_category","dtype":"string","entity":"person","ownership":"produced","rows":"all"},{"column":"dla_sc_category","dtype":"string","entity":"person","ownership":"produced","rows":"all"},{"column":"dla_m_category","dtype":"string","entity":"person","ownership":"produced","rows":"all"},{"column":"pip_m_category","dtype":"string","entity":"person","ownership":"produced","rows":"all"},{"column":"pip_dl_category","dtype":"string","entity":"person","ownership":"produced","rows":"all"},{"column":"is_disabled_for_benefits","dtype":"bool","entity":"person","ownership":"produced","rows":"all"},{"column":"is_enhanced_disabled_for_benefits","dtype":"bool","entity":"person","ownership":"produced","rows":"all"},{"column":"is_severely_disabled_for_benefits","dtype":"bool","entity":"person","ownership":"produced","rows":"all"}],"params":{"stage":"frs_disability","stage_contract_sha256":"2d5ed820f9f9fbe4a45a333f5c5e1dfbdfe2b53a055584ada4c44eacb4b58954","time_period":"2024"},"population":"frs_spine.boundary","sources":["frs"],"structural":"none","weights":null},{"base":null,"citation":"","description":"Run UK spine stage frs_education.","id":"frs_education","inputs":[{"columns":["frs_benunit_capital"],"entity":"benunit","rows":"all"},{"columns":["region"],"entity":"household","rows":"all"},{"columns":["age","esa_contrib_reported","esa_income_reported","is_severely_disabled_for_benefits","jsa_contrib_reported","jsa_income_reported","universal_credit_reported"],"entity":"person","rows":"all"}],"kernel":"uk.stage.frs_education@1","mass":"conserve","outputs":[{"column":"current_education","dtype":"string","entity":"person","ownership":"produced","rows":"all"},{"column":"highest_education","dtype":"string","entity":"person","ownership":"produced","rows":"all"},{"column":"is_in_non_advanced_education","dtype":"bool","entity":"person","ownership":"produced","rows":"all"},{"column":"is_in_approved_training","dtype":"bool","entity":"person","ownership":"produced","rows":"all"},{"column":"age_started_or_accepted_current_education_or_training","dtype":"int64","entity":"person","ownership":"produced","rows":"all"},{"column":"is_before_universal_credit_qualifying_young_person_terminal_date","dtype":"bool","entity":"person","ownership":"produced","rows":"all"},{"column":"adult_ema","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"child_ema","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"receives_benefits_in_own_right","dtype":"bool","entity":"person","ownership":"produced","rows":"all"}],"params":{"stage":"frs_education","stage_contract_sha256":"836cb0f5a582fee1425190e96c9cb81bdef859bd236c8b6bc27660b6e3d0c2f8","time_period":"2024"},"population":"frs_spine.boundary","sources":["frs"],"structural":"none","weights":null},{"base":null,"citation":"","description":"Run UK spine stage frs_legacy_proxies.","id":"frs_legacy_proxies","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","employee_pension_contributions","pension_contributions_via_salary_sacrifice","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","universal_credit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported","employment_status","employment_sector","sic_industry_division","aa_category","dla_sc_category","dla_m_category","pip_m_category","pip_dl_category","is_disabled_for_benefits","is_enhanced_disabled_for_benefits","is_severely_disabled_for_benefits","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right"],"entity":"person","rows":"all"}],"kernel":"uk.stage.frs_legacy_proxies@1","mass":"conserve","outputs":[{"column":"legacy_jobseeker_proxy","dtype":"bool","entity":"person","ownership":"produced","rows":"all"},{"column":"esa_health_condition_proxy","dtype":"bool","entity":"person","ownership":"produced","rows":"all"},{"column":"esa_support_group_proxy","dtype":"bool","entity":"person","ownership":"produced","rows":"all"}],"params":{"stage":"frs_legacy_proxies","stage_contract_sha256":"1ecf761cf3fa7180da15659e138e67a8654e58272b4fde28344aa27a874ad0aa","time_period":"2024"},"population":"frs_spine.boundary","sources":["frs"],"structural":"none","weights":null},{"base":"frs_spine.boundary","citation":"","description":"Ownership boundary before frs_education_grant_split rewrites.","id":"frs_education_grant_split.boundary","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","employee_pension_contributions","pension_contributions_via_salary_sacrifice","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","universal_credit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported","employment_status","employment_sector","sic_industry_division","aa_category","dla_sc_category","dla_m_category","pip_m_category","pip_dl_category","is_disabled_for_benefits","is_enhanced_disabled_for_benefits","is_severely_disabled_for_benefits","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right","legacy_jobseeker_proxy","esa_health_condition_proxy","esa_support_group_proxy"],"entity":"person","rows":"all"}],"kernel":"uk.identity@1","mass":"conserve","outputs":[],"params":{},"population":null,"sources":[],"structural":"filter","weights":null},{"base":null,"citation":"","description":"Run UK spine stage frs_education_grant_split.","id":"frs_education_grant_split","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","employee_pension_contributions","pension_contributions_via_salary_sacrifice","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","universal_credit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported","employment_status","employment_sector","sic_industry_division","aa_category","dla_sc_category","dla_m_category","pip_m_category","pip_dl_category","is_disabled_for_benefits","is_enhanced_disabled_for_benefits","is_severely_disabled_for_benefits","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right","legacy_jobseeker_proxy","esa_health_condition_proxy","esa_support_group_proxy"],"entity":"person","rows":"all"}],"kernel":"uk.stage.frs_education_grant_split@1","mass":"conserve","outputs":[{"column":"disabled_students_allowance_eligible_expenses","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"education_grants","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"}],"params":{"stage":"frs_education_grant_split","stage_contract_sha256":"6f0fcfa6a1aa614f7ccefe39364b4ebd208c9a4ed081f7618e026b2e87f152fa","time_period":"2024"},"population":"frs_education_grant_split.boundary","sources":["frs"],"structural":"none","weights":null},{"base":null,"citation":"","description":"Run UK spine stage frs_take_up.","id":"frs_take_up","inputs":[{"columns":["frs_benunit_capital"],"entity":"benunit","rows":"all"},{"columns":["region"],"entity":"household","rows":"all"},{"columns":["age","child_benefit_reported","education_grants","pension_credit_reported","universal_credit_reported"],"entity":"person","rows":"all"}],"kernel":"uk.stage.frs_take_up@1","mass":"conserve","outputs":[{"column":"would_claim_child_benefit","dtype":"bool","entity":"benunit","ownership":"produced","rows":"all"},{"column":"child_benefit_opts_out","dtype":"bool","entity":"benunit","ownership":"produced","rows":"all"},{"column":"would_claim_pc","dtype":"bool","entity":"benunit","ownership":"produced","rows":"all"},{"column":"would_claim_uc","dtype":"bool","entity":"benunit","ownership":"produced","rows":"all"},{"column":"would_claim_tfc","dtype":"bool","entity":"benunit","ownership":"produced","rows":"all"},{"column":"would_claim_extended_childcare","dtype":"bool","entity":"benunit","ownership":"produced","rows":"all"},{"column":"would_claim_universal_childcare","dtype":"bool","entity":"benunit","ownership":"produced","rows":"all"},{"column":"would_claim_targeted_childcare","dtype":"bool","entity":"benunit","ownership":"produced","rows":"all"},{"column":"maximum_extended_childcare_hours_usage","dtype":"float64","entity":"benunit","ownership":"produced","rows":"all"}],"params":{"stage":"frs_take_up","stage_contract_sha256":"d10ca01bfa5d719640ded8e62196b0692236d3be0a2417899deeefd241088abf","time_period":"2024"},"population":"frs_education_grant_split.boundary","sources":["frs"],"structural":"none","weights":null},{"base":null,"citation":"","description":"Run UK spine stage frs_person_draws.","id":"frs_person_draws","inputs":[{"columns":["frs_benunit_capital","maximum_extended_childcare_hours_usage"],"entity":"benunit","rows":"all"},{"columns":["region"],"entity":"household","rows":"all"},{"columns":["age"],"entity":"person","rows":"all"}],"kernel":"uk.stage.frs_person_draws@1","mass":"conserve","outputs":[{"column":"would_claim_marriage_allowance","dtype":"bool","entity":"person","ownership":"produced","rows":"all"},{"column":"would_claim_scp","dtype":"bool","entity":"person","ownership":"produced","rows":"all"},{"column":"attends_private_school_random_draw","dtype":"float64","entity":"person","ownership":"produced","rows":"all"}],"params":{"stage":"frs_person_draws","stage_contract_sha256":"1543fd39f943d628d224adf8a31d5379d4f498d3f3861e67ec26981a24e76d20","time_period":"2024"},"population":"frs_education_grant_split.boundary","sources":["frs"],"structural":"none","weights":null},{"base":null,"citation":"","description":"Run UK spine stage frs_household_draws.","id":"frs_household_draws","inputs":[{"columns":["frs_benunit_capital"],"entity":"benunit","rows":"all"},{"columns":["region"],"entity":"household","rows":"all"},{"columns":["age","attends_private_school_random_draw"],"entity":"person","rows":"all"}],"kernel":"uk.stage.frs_household_draws@1","mass":"conserve","outputs":[{"column":"household_owns_tv","dtype":"bool","entity":"household","ownership":"produced","rows":"all"},{"column":"would_evade_tv_licence_fee","dtype":"bool","entity":"household","ownership":"produced","rows":"all"},{"column":"main_residential_property_purchased_is_first_home","dtype":"bool","entity":"household","ownership":"produced","rows":"all"},{"column":"property_purchased","dtype":"bool","entity":"household","ownership":"produced","rows":"all"}],"params":{"stage":"frs_household_draws","stage_contract_sha256":"c74c63b09264319c4bf0049dabba00ecd0ce660beb7b53d6dae71518434bc949","time_period":"2024"},"population":"frs_education_grant_split.boundary","sources":["frs"],"structural":"none","weights":null},{"base":null,"citation":"","description":"Run UK spine stage frs_brma.","id":"frs_brma","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children","would_claim_child_benefit","child_benefit_opts_out","would_claim_pc","would_claim_uc","would_claim_tfc","would_claim_extended_childcare","would_claim_universal_childcare","would_claim_targeted_childcare","maximum_extended_childcare_hours_usage"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax","household_owns_tv","would_evade_tv_licence_fee","main_residential_property_purchased_is_first_home","property_purchased"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","employee_pension_contributions","pension_contributions_via_salary_sacrifice","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","universal_credit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported","employment_status","employment_sector","sic_industry_division","aa_category","dla_sc_category","dla_m_category","pip_m_category","pip_dl_category","is_disabled_for_benefits","is_enhanced_disabled_for_benefits","is_severely_disabled_for_benefits","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right","legacy_jobseeker_proxy","esa_health_condition_proxy","esa_support_group_proxy","disabled_students_allowance_eligible_expenses","would_claim_marriage_allowance","would_claim_scp","attends_private_school_random_draw"],"entity":"person","rows":"all"}],"kernel":"uk.stage.frs_brma@1","mass":"conserve","outputs":[{"column":"brma","dtype":"string","entity":"household","ownership":"produced","rows":"all"}],"params":{"stage":"frs_brma","stage_contract_sha256":"5b8f0b361310c7cd3efbaae3762b347649b7d4afb9e9943131d9bef9266fc3c8","time_period":"2024"},"population":"frs_education_grant_split.boundary","sources":["frs"],"structural":"none","weights":null},{"base":"frs_education_grant_split.boundary","citation":"","description":"Freeze the assembled-spine gate population.","id":"frs_brma.checkpoint","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children","would_claim_child_benefit","child_benefit_opts_out","would_claim_pc","would_claim_uc","would_claim_tfc","would_claim_extended_childcare","would_claim_universal_childcare","would_claim_targeted_childcare","maximum_extended_childcare_hours_usage"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax","household_owns_tv","would_evade_tv_licence_fee","main_residential_property_purchased_is_first_home","property_purchased","brma"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","employee_pension_contributions","pension_contributions_via_salary_sacrifice","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","universal_credit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported","employment_status","employment_sector","sic_industry_division","aa_category","dla_sc_category","dla_m_category","pip_m_category","pip_dl_category","is_disabled_for_benefits","is_enhanced_disabled_for_benefits","is_severely_disabled_for_benefits","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right","legacy_jobseeker_proxy","esa_health_condition_proxy","esa_support_group_proxy","disabled_students_allowance_eligible_expenses","would_claim_marriage_allowance","would_claim_scp","attends_private_school_random_draw"],"entity":"person","rows":"all"}],"kernel":"uk.identity@1","mass":"conserve","outputs":[],"params":{},"population":null,"sources":[],"structural":"filter","weights":null},{"base":null,"citation":"","description":"Run UK spine stage was_wealth.","id":"was_wealth","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children","would_claim_child_benefit","child_benefit_opts_out","would_claim_pc","would_claim_uc","would_claim_tfc","would_claim_extended_childcare","would_claim_universal_childcare","would_claim_targeted_childcare","maximum_extended_childcare_hours_usage"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax","household_owns_tv","would_evade_tv_licence_fee","main_residential_property_purchased_is_first_home","property_purchased","brma"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","employee_pension_contributions","pension_contributions_via_salary_sacrifice","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","universal_credit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported","employment_status","employment_sector","sic_industry_division","aa_category","dla_sc_category","dla_m_category","pip_m_category","pip_dl_category","is_disabled_for_benefits","is_enhanced_disabled_for_benefits","is_severely_disabled_for_benefits","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right","legacy_jobseeker_proxy","esa_health_condition_proxy","esa_support_group_proxy","disabled_students_allowance_eligible_expenses","would_claim_marriage_allowance","would_claim_scp","attends_private_school_random_draw"],"entity":"person","rows":"all"}],"kernel":"uk.stage.was_wealth@1","mass":"conserve","outputs":[{"column":"owned_land","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"property_wealth","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"corporate_wealth","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"private_pension_wealth","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"gross_financial_wealth","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"net_financial_wealth","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"main_residence_value","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"other_residential_property_value","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"non_residential_property_value","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"savings","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"num_vehicles","dtype":"int64","entity":"household","ownership":"produced","rows":"all"},{"column":"cash_isa","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"stocks_and_shares_isa","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"mortgage_debt","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"consumer_debt","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"student_loan_balance","dtype":"float64","entity":"person","ownership":"produced","rows":"all"}],"params":{"stage":"was_wealth","stage_contract_sha256":"c499dcb7c49a0176229dbc363b400415cb5c1da44af266ba2155f9a62a0bd2ed","time_period":"2024"},"population":"frs_brma.checkpoint","sources":["frs"],"structural":"none","weights":null},{"base":"frs_brma.checkpoint","citation":"","description":"Ownership boundary before regional_property_uprating rewrites.","id":"regional_property_uprating.boundary","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children","would_claim_child_benefit","child_benefit_opts_out","would_claim_pc","would_claim_uc","would_claim_tfc","would_claim_extended_childcare","would_claim_universal_childcare","would_claim_targeted_childcare","maximum_extended_childcare_hours_usage"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax","household_owns_tv","would_evade_tv_licence_fee","main_residential_property_purchased_is_first_home","property_purchased","brma","owned_land","property_wealth","corporate_wealth","private_pension_wealth","gross_financial_wealth","net_financial_wealth","main_residence_value","other_residential_property_value","non_residential_property_value","savings","num_vehicles","cash_isa","stocks_and_shares_isa","mortgage_debt","consumer_debt"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","employee_pension_contributions","pension_contributions_via_salary_sacrifice","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","universal_credit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported","employment_status","employment_sector","sic_industry_division","aa_category","dla_sc_category","dla_m_category","pip_m_category","pip_dl_category","is_disabled_for_benefits","is_enhanced_disabled_for_benefits","is_severely_disabled_for_benefits","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right","legacy_jobseeker_proxy","esa_health_condition_proxy","esa_support_group_proxy","disabled_students_allowance_eligible_expenses","would_claim_marriage_allowance","would_claim_scp","attends_private_school_random_draw","student_loan_balance"],"entity":"person","rows":"all"}],"kernel":"uk.identity@1","mass":"conserve","outputs":[],"params":{},"population":null,"sources":[],"structural":"filter","weights":null},{"base":null,"citation":"","description":"Run UK spine stage regional_property_uprating.","id":"regional_property_uprating","inputs":[{"columns":["frs_benunit_capital"],"entity":"benunit","rows":"all"},{"columns":["region"],"entity":"household","rows":"all"},{"columns":["age","student_loan_balance"],"entity":"person","rows":"all"}],"kernel":"uk.stage.regional_property_uprating@1","mass":"conserve","outputs":[{"column":"main_residence_value","dtype":"float64","entity":"household","ownership":"produced","rewrite":true,"rows":"all"},{"column":"property_wealth","dtype":"float64","entity":"household","ownership":"produced","rewrite":true,"rows":"all"}],"params":{"stage":"regional_property_uprating","stage_contract_sha256":"4304356c6c5ba91a04883148cbf9078761ee926c8387aadb6c15b6b70ebb31f6","time_period":"2024"},"population":"regional_property_uprating.boundary","sources":["frs"],"structural":"none","weights":null},{"base":null,"citation":"","description":"Run UK spine stage lcfs_consumption.","id":"lcfs_consumption","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children","would_claim_child_benefit","child_benefit_opts_out","would_claim_pc","would_claim_uc","would_claim_tfc","would_claim_extended_childcare","would_claim_universal_childcare","would_claim_targeted_childcare","maximum_extended_childcare_hours_usage"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax","household_owns_tv","would_evade_tv_licence_fee","main_residential_property_purchased_is_first_home","property_purchased","brma","owned_land","property_wealth","corporate_wealth","private_pension_wealth","gross_financial_wealth","net_financial_wealth","main_residence_value","other_residential_property_value","non_residential_property_value","savings","num_vehicles","cash_isa","stocks_and_shares_isa","mortgage_debt","consumer_debt"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","employee_pension_contributions","pension_contributions_via_salary_sacrifice","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","universal_credit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported","employment_status","employment_sector","sic_industry_division","aa_category","dla_sc_category","dla_m_category","pip_m_category","pip_dl_category","is_disabled_for_benefits","is_enhanced_disabled_for_benefits","is_severely_disabled_for_benefits","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right","legacy_jobseeker_proxy","esa_health_condition_proxy","esa_support_group_proxy","disabled_students_allowance_eligible_expenses","would_claim_marriage_allowance","would_claim_scp","attends_private_school_random_draw","student_loan_balance"],"entity":"person","rows":"all"}],"kernel":"uk.stage.lcfs_consumption@1","mass":"conserve","outputs":[{"column":"food_and_non_alcoholic_beverages_consumption","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"alcohol_and_tobacco_consumption","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"clothing_and_footwear_consumption","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"housing_water_and_electricity_consumption","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"household_furnishings_consumption","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"health_consumption","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"transport_consumption","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"communication_consumption","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"recreation_consumption","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"education_consumption","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"restaurants_and_hotels_consumption","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"miscellaneous_consumption","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"petrol_spending","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"diesel_spending","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"bus_fare_spending","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"domestic_energy_consumption","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"electricity_consumption","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"gas_consumption","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"has_fuel_consumption","dtype":"bool","entity":"household","ownership":"produced","rows":"all"}],"params":{"stage":"lcfs_consumption","stage_contract_sha256":"eb0a529c357c84290a001209a18c5a43b0a0310553c77963d6842fd0111b73ad","time_period":"2024"},"population":"regional_property_uprating.boundary","sources":["frs"],"structural":"none","weights":null},{"base":null,"citation":"","description":"Run UK spine stage etb_vat.","id":"etb_vat","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children","would_claim_child_benefit","child_benefit_opts_out","would_claim_pc","would_claim_uc","would_claim_tfc","would_claim_extended_childcare","would_claim_universal_childcare","would_claim_targeted_childcare","maximum_extended_childcare_hours_usage"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax","household_owns_tv","would_evade_tv_licence_fee","main_residential_property_purchased_is_first_home","property_purchased","brma","owned_land","property_wealth","corporate_wealth","private_pension_wealth","gross_financial_wealth","net_financial_wealth","main_residence_value","other_residential_property_value","non_residential_property_value","savings","num_vehicles","cash_isa","stocks_and_shares_isa","mortgage_debt","consumer_debt","food_and_non_alcoholic_beverages_consumption","alcohol_and_tobacco_consumption","clothing_and_footwear_consumption","housing_water_and_electricity_consumption","household_furnishings_consumption","health_consumption","transport_consumption","communication_consumption","recreation_consumption","education_consumption","restaurants_and_hotels_consumption","miscellaneous_consumption","petrol_spending","diesel_spending","bus_fare_spending","domestic_energy_consumption","electricity_consumption","gas_consumption","has_fuel_consumption"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","employee_pension_contributions","pension_contributions_via_salary_sacrifice","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","universal_credit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported","employment_status","employment_sector","sic_industry_division","aa_category","dla_sc_category","dla_m_category","pip_m_category","pip_dl_category","is_disabled_for_benefits","is_enhanced_disabled_for_benefits","is_severely_disabled_for_benefits","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right","legacy_jobseeker_proxy","esa_health_condition_proxy","esa_support_group_proxy","disabled_students_allowance_eligible_expenses","would_claim_marriage_allowance","would_claim_scp","attends_private_school_random_draw","student_loan_balance"],"entity":"person","rows":"all"}],"kernel":"uk.stage.etb_vat@1","mass":"conserve","outputs":[{"column":"full_rate_vat_expenditure_rate","dtype":"float64","entity":"household","ownership":"produced","rows":"all"}],"params":{"stage":"etb_vat","stage_contract_sha256":"99a6756256adfe8255672fafda89710c08ae56bd2b8f011b20c3b8381eabfa1e","time_period":"2024"},"population":"regional_property_uprating.boundary","sources":["frs"],"structural":"none","weights":null},{"base":null,"citation":"","description":"Run UK spine stage etb_services.","id":"etb_services","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children","would_claim_child_benefit","child_benefit_opts_out","would_claim_pc","would_claim_uc","would_claim_tfc","would_claim_extended_childcare","would_claim_universal_childcare","would_claim_targeted_childcare","maximum_extended_childcare_hours_usage"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax","household_owns_tv","would_evade_tv_licence_fee","main_residential_property_purchased_is_first_home","property_purchased","brma","owned_land","property_wealth","corporate_wealth","private_pension_wealth","gross_financial_wealth","net_financial_wealth","main_residence_value","other_residential_property_value","non_residential_property_value","savings","num_vehicles","cash_isa","stocks_and_shares_isa","mortgage_debt","consumer_debt","food_and_non_alcoholic_beverages_consumption","alcohol_and_tobacco_consumption","clothing_and_footwear_consumption","housing_water_and_electricity_consumption","household_furnishings_consumption","health_consumption","transport_consumption","communication_consumption","recreation_consumption","education_consumption","restaurants_and_hotels_consumption","miscellaneous_consumption","petrol_spending","diesel_spending","bus_fare_spending","domestic_energy_consumption","electricity_consumption","gas_consumption","has_fuel_consumption","full_rate_vat_expenditure_rate"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","employee_pension_contributions","pension_contributions_via_salary_sacrifice","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","universal_credit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported","employment_status","employment_sector","sic_industry_division","aa_category","dla_sc_category","dla_m_category","pip_m_category","pip_dl_category","is_disabled_for_benefits","is_enhanced_disabled_for_benefits","is_severely_disabled_for_benefits","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right","legacy_jobseeker_proxy","esa_health_condition_proxy","esa_support_group_proxy","disabled_students_allowance_eligible_expenses","would_claim_marriage_allowance","would_claim_scp","attends_private_school_random_draw","student_loan_balance"],"entity":"person","rows":"all"}],"kernel":"uk.stage.etb_services@1","mass":"conserve","outputs":[{"column":"dfe_education_spending","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"rail_subsidy_spending","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"bus_subsidy_spending","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"rail_usage","dtype":"float64","entity":"household","ownership":"produced","rows":"all"},{"column":"a_and_e_visits","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"admitted_patient_visits","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"outpatient_visits","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"nhs_a_and_e_spending","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"nhs_admitted_patient_spending","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"nhs_outpatient_spending","dtype":"float64","entity":"person","ownership":"produced","rows":"all"}],"params":{"stage":"etb_services","stage_contract_sha256":"5ca1399954560400665579d29791831eac81202aada216939bf91789136cdd03","time_period":"2024"},"population":"regional_property_uprating.boundary","sources":["frs"],"structural":"none","weights":null},{"base":null,"citation":"","description":"Run UK spine stage frs_hmrc_spine_leaves.","id":"frs_hmrc_spine_leaves","inputs":[{"columns":["frs_benunit_capital"],"entity":"benunit","rows":"all"},{"columns":["region"],"entity":"household","rows":"all"},{"columns":["age","employee_pension_contributions","nhs_outpatient_spending"],"entity":"person","rows":"all"}],"kernel":"uk.stage.frs_hmrc_spine_leaves@1","mass":"conserve","outputs":[{"column":"hmrc_spi_pay","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"hmrc_spi_unemployment_benefit_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"hmrc_spi_incapacity_benefit_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"ossben_identifiable_subset","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"srp_regular_code5","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"employer_pension_contributions","dtype":"float64","entity":"person","ownership":"produced","rows":"all"}],"params":{"stage":"frs_hmrc_spine_leaves","stage_contract_sha256":"2bb3d068003489b47cc8676ac26c203ce3c39a9b6e92f3ecd0a3c06acd6addc4","time_period":"2024"},"population":"regional_property_uprating.boundary","sources":["frs"],"structural":"none","weights":null},{"base":"regional_property_uprating.boundary","citation":"","description":"Run structural UK stage spi_support_channel.","id":"spi_support_channel","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children","would_claim_child_benefit","child_benefit_opts_out","would_claim_pc","would_claim_uc","would_claim_tfc","would_claim_extended_childcare","would_claim_universal_childcare","would_claim_targeted_childcare","maximum_extended_childcare_hours_usage"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax","household_owns_tv","would_evade_tv_licence_fee","main_residential_property_purchased_is_first_home","property_purchased","brma","owned_land","property_wealth","corporate_wealth","private_pension_wealth","gross_financial_wealth","net_financial_wealth","main_residence_value","other_residential_property_value","non_residential_property_value","savings","num_vehicles","cash_isa","stocks_and_shares_isa","mortgage_debt","consumer_debt","food_and_non_alcoholic_beverages_consumption","alcohol_and_tobacco_consumption","clothing_and_footwear_consumption","housing_water_and_electricity_consumption","household_furnishings_consumption","health_consumption","transport_consumption","communication_consumption","recreation_consumption","education_consumption","restaurants_and_hotels_consumption","miscellaneous_consumption","petrol_spending","diesel_spending","bus_fare_spending","domestic_energy_consumption","electricity_consumption","gas_consumption","has_fuel_consumption","full_rate_vat_expenditure_rate","dfe_education_spending","rail_subsidy_spending","bus_subsidy_spending","rail_usage"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","employee_pension_contributions","pension_contributions_via_salary_sacrifice","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","universal_credit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported","employment_status","employment_sector","sic_industry_division","aa_category","dla_sc_category","dla_m_category","pip_m_category","pip_dl_category","is_disabled_for_benefits","is_enhanced_disabled_for_benefits","is_severely_disabled_for_benefits","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right","legacy_jobseeker_proxy","esa_health_condition_proxy","esa_support_group_proxy","disabled_students_allowance_eligible_expenses","would_claim_marriage_allowance","would_claim_scp","attends_private_school_random_draw","student_loan_balance","a_and_e_visits","admitted_patient_visits","outpatient_visits","nhs_a_and_e_spending","nhs_admitted_patient_spending","nhs_outpatient_spending","hmrc_spi_pay","hmrc_spi_unemployment_benefit_income","hmrc_spi_incapacity_benefit_income","ossben_identifiable_subset","srp_regular_code5","employer_pension_contributions"],"entity":"person","rows":"all"}],"kernel":"uk.stage.expand.spi_support_channel@1","mass":"declared","outputs":[],"params":{"expand_cells":[["person","person_source_id","int64"],["person","person_support_channel","string"],["person","person_support_clone_index","int64"],["benunit","benunit_source_id","int64"],["benunit","benunit_support_channel","string"],["benunit","benunit_support_clone_index","int64"],["household","source_household_id","int64"],["household","source_year","int64"],["household","source_household_key","string"],["household","household_source_id","int64"],["household","household_support_channel","string"],["household","household_support_clone_index","int64"],["household","household_is_spi_synthetic","bool"]],"expand_weight_entity":"household","expand_weight_kind":"importance","stage":"spi_support_channel","stage_contract_sha256":"4b10f1a4a215cdf2c406c9974de2d1e580eb3cd3a25ed34973c1b63abf4a54bb","time_period":"2024"},"population":null,"sources":["frs"],"structural":"expand","weights":null},{"base":null,"citation":"","description":"Own the cells materialized by spi_support_channel.","id":"spi_support_channel.owned","inputs":[],"kernel":"uk.claim@1","mass":"conserve","outputs":[{"column":"person_source_id","dtype":"int64","entity":"person","ownership":"produced","rows":"all"},{"column":"person_support_channel","dtype":"string","entity":"person","ownership":"produced","rows":"all"},{"column":"person_support_clone_index","dtype":"int64","entity":"person","ownership":"produced","rows":"all"},{"column":"benunit_source_id","dtype":"int64","entity":"benunit","ownership":"produced","rows":"all"},{"column":"benunit_support_channel","dtype":"string","entity":"benunit","ownership":"produced","rows":"all"},{"column":"benunit_support_clone_index","dtype":"int64","entity":"benunit","ownership":"produced","rows":"all"},{"column":"source_household_id","dtype":"int64","entity":"household","ownership":"produced","rows":"all"},{"column":"source_year","dtype":"int64","entity":"household","ownership":"produced","rows":"all"},{"column":"source_household_key","dtype":"string","entity":"household","ownership":"produced","rows":"all"},{"column":"household_source_id","dtype":"int64","entity":"household","ownership":"produced","rows":"all"},{"column":"household_support_channel","dtype":"string","entity":"household","ownership":"produced","rows":"all"},{"column":"household_support_clone_index","dtype":"int64","entity":"household","ownership":"produced","rows":"all"},{"column":"household_is_spi_synthetic","dtype":"bool","entity":"household","ownership":"produced","rows":"all"}],"params":{"materialized_expand_outputs":["person.person_source_id","person.person_support_channel","person.person_support_clone_index","benunit.benunit_source_id","benunit.benunit_support_channel","benunit.benunit_support_clone_index","household.source_household_id","household.source_year","household.source_household_key","household.household_source_id","household.household_support_channel","household.household_support_clone_index","household.household_is_spi_synthetic"]},"population":"spi_support_channel","sources":[],"structural":"none","weights":null},{"base":null,"citation":"","description":"Run UK spine stage hmrc_spi_income_spine.","id":"hmrc_spi_income_spine","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children","would_claim_child_benefit","child_benefit_opts_out","would_claim_pc","would_claim_uc","would_claim_tfc","would_claim_extended_childcare","would_claim_universal_childcare","would_claim_targeted_childcare","maximum_extended_childcare_hours_usage","benunit_source_id","benunit_support_channel","benunit_support_clone_index"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax","household_owns_tv","would_evade_tv_licence_fee","main_residential_property_purchased_is_first_home","property_purchased","brma","owned_land","property_wealth","corporate_wealth","private_pension_wealth","gross_financial_wealth","net_financial_wealth","main_residence_value","other_residential_property_value","non_residential_property_value","savings","num_vehicles","cash_isa","stocks_and_shares_isa","mortgage_debt","consumer_debt","food_and_non_alcoholic_beverages_consumption","alcohol_and_tobacco_consumption","clothing_and_footwear_consumption","housing_water_and_electricity_consumption","household_furnishings_consumption","health_consumption","transport_consumption","communication_consumption","recreation_consumption","education_consumption","restaurants_and_hotels_consumption","miscellaneous_consumption","petrol_spending","diesel_spending","bus_fare_spending","domestic_energy_consumption","electricity_consumption","gas_consumption","has_fuel_consumption","full_rate_vat_expenditure_rate","dfe_education_spending","rail_subsidy_spending","bus_subsidy_spending","rail_usage","source_household_id","source_year","source_household_key","household_source_id","household_support_channel","household_support_clone_index","household_is_spi_synthetic"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","maintenance_expenses","childcare_expenses","salary_sacrifice_reported","salary_sacrifice_asked","ssmg_reported","incapacity_benefit_reported","employment_status","employment_sector","sic_industry_division","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right","legacy_jobseeker_proxy","esa_health_condition_proxy","esa_support_group_proxy","disabled_students_allowance_eligible_expenses","would_claim_marriage_allowance","would_claim_scp","attends_private_school_random_draw","student_loan_balance","a_and_e_visits","admitted_patient_visits","outpatient_visits","nhs_a_and_e_spending","nhs_admitted_patient_spending","nhs_outpatient_spending","ossben_identifiable_subset","srp_regular_code5","person_source_id","person_support_channel","person_support_clone_index"],"entity":"person","rows":"all"}],"kernel":"uk.stage.hmrc_spi_income_spine@1","mass":"conserve","outputs":[{"column":"charitable_investment_gifts","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"gift_aid","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"other_investment_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"hmrc_spi_employment_benefits","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"hmrc_spi_employment_expenses","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"hmrc_spi_other_social_security_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"hmrc_spi_taxable_termination_pay","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"hmrc_spi_miscellaneous_employment_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"hmrc_spi_other_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"hmrc_spi_state_pension_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"hmrc_spi_employed_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"hmrc_spi_total_earned_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"hmrc_spi_total_investment_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"hmrc_spi_assessable_income","dtype":"float64","entity":"person","ownership":"produced","rows":"all"},{"column":"employment_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"self_employment_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"savings_interest_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"dividend_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"private_pension_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"property_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"employee_pension_contributions","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"employer_pension_contributions","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"personal_pension_contributions","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"pension_contributions_via_salary_sacrifice","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"tax_free_savings_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"universal_credit_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"pension_credit_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"child_benefit_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"housing_benefit_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"income_support_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"working_tax_credit_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"child_tax_credit_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"attendance_allowance_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"state_pension_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"dla_sc_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"dla_m_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"pip_m_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"pip_dl_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"sda_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"carers_allowance_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"iidb_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"afcs_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"bsp_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"winter_fuel_allowance_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"council_tax_benefit_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"jsa_contrib_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"jsa_income_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"esa_contrib_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"esa_income_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"hmrc_spi_pay","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"hmrc_spi_unemployment_benefit_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"hmrc_spi_incapacity_benefit_income","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"aa_category","dtype":"string","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"dla_sc_category","dtype":"string","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"dla_m_category","dtype":"string","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"pip_m_category","dtype":"string","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"pip_dl_category","dtype":"string","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"is_disabled_for_benefits","dtype":"bool","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"is_enhanced_disabled_for_benefits","dtype":"bool","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"is_severely_disabled_for_benefits","dtype":"bool","entity":"person","ownership":"produced","rewrite":true,"rows":"all"}],"params":{"stage":"hmrc_spi_income_spine","stage_contract_sha256":"3db4e7361ea871ca57b52aafce48209409d7b59d836ed973eee946b5657a771a","time_period":"2024"},"population":"spi_support_channel","sources":["frs"],"structural":"none","weights":null},{"base":"spi_support_channel","citation":"","description":"Ownership boundary before uc_reporter_redraw rewrites.","id":"uc_reporter_redraw.boundary","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children","would_claim_child_benefit","child_benefit_opts_out","would_claim_pc","would_claim_uc","would_claim_tfc","would_claim_extended_childcare","would_claim_universal_childcare","would_claim_targeted_childcare","maximum_extended_childcare_hours_usage","benunit_source_id","benunit_support_channel","benunit_support_clone_index"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax","household_owns_tv","would_evade_tv_licence_fee","main_residential_property_purchased_is_first_home","property_purchased","brma","owned_land","property_wealth","corporate_wealth","private_pension_wealth","gross_financial_wealth","net_financial_wealth","main_residence_value","other_residential_property_value","non_residential_property_value","savings","num_vehicles","cash_isa","stocks_and_shares_isa","mortgage_debt","consumer_debt","food_and_non_alcoholic_beverages_consumption","alcohol_and_tobacco_consumption","clothing_and_footwear_consumption","housing_water_and_electricity_consumption","household_furnishings_consumption","health_consumption","transport_consumption","communication_consumption","recreation_consumption","education_consumption","restaurants_and_hotels_consumption","miscellaneous_consumption","petrol_spending","diesel_spending","bus_fare_spending","domestic_energy_consumption","electricity_consumption","gas_consumption","has_fuel_consumption","full_rate_vat_expenditure_rate","dfe_education_spending","rail_subsidy_spending","bus_subsidy_spending","rail_usage","source_household_id","source_year","source_household_key","household_source_id","household_support_channel","household_support_clone_index","household_is_spi_synthetic"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","employee_pension_contributions","pension_contributions_via_salary_sacrifice","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","universal_credit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported","employment_status","employment_sector","sic_industry_division","aa_category","dla_sc_category","dla_m_category","pip_m_category","pip_dl_category","is_disabled_for_benefits","is_enhanced_disabled_for_benefits","is_severely_disabled_for_benefits","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right","legacy_jobseeker_proxy","esa_health_condition_proxy","esa_support_group_proxy","disabled_students_allowance_eligible_expenses","would_claim_marriage_allowance","would_claim_scp","attends_private_school_random_draw","student_loan_balance","a_and_e_visits","admitted_patient_visits","outpatient_visits","nhs_a_and_e_spending","nhs_admitted_patient_spending","nhs_outpatient_spending","hmrc_spi_pay","hmrc_spi_unemployment_benefit_income","hmrc_spi_incapacity_benefit_income","ossben_identifiable_subset","srp_regular_code5","employer_pension_contributions","person_source_id","person_support_channel","person_support_clone_index","charitable_investment_gifts","gift_aid","other_investment_income","hmrc_spi_employment_benefits","hmrc_spi_employment_expenses","hmrc_spi_other_social_security_income","hmrc_spi_taxable_termination_pay","hmrc_spi_miscellaneous_employment_income","hmrc_spi_other_income","hmrc_spi_state_pension_income","hmrc_spi_employed_income","hmrc_spi_total_earned_income","hmrc_spi_total_investment_income","hmrc_spi_assessable_income"],"entity":"person","rows":"all"}],"kernel":"uk.identity@1","mass":"conserve","outputs":[],"params":{},"population":null,"sources":[],"structural":"filter","weights":null},{"base":null,"citation":"","description":"Run UK spine stage uc_reporter_redraw.","id":"uc_reporter_redraw","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children","would_claim_child_benefit","child_benefit_opts_out","would_claim_pc","would_claim_uc","would_claim_tfc","would_claim_extended_childcare","would_claim_universal_childcare","would_claim_targeted_childcare","maximum_extended_childcare_hours_usage","benunit_source_id","benunit_support_channel","benunit_support_clone_index"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax","household_owns_tv","would_evade_tv_licence_fee","main_residential_property_purchased_is_first_home","property_purchased","brma","owned_land","property_wealth","corporate_wealth","private_pension_wealth","gross_financial_wealth","net_financial_wealth","main_residence_value","other_residential_property_value","non_residential_property_value","savings","num_vehicles","cash_isa","stocks_and_shares_isa","mortgage_debt","consumer_debt","food_and_non_alcoholic_beverages_consumption","alcohol_and_tobacco_consumption","clothing_and_footwear_consumption","housing_water_and_electricity_consumption","household_furnishings_consumption","health_consumption","transport_consumption","communication_consumption","recreation_consumption","education_consumption","restaurants_and_hotels_consumption","miscellaneous_consumption","petrol_spending","diesel_spending","bus_fare_spending","domestic_energy_consumption","electricity_consumption","gas_consumption","has_fuel_consumption","full_rate_vat_expenditure_rate","dfe_education_spending","rail_subsidy_spending","bus_subsidy_spending","rail_usage","source_household_id","source_year","source_household_key","household_source_id","household_support_channel","household_support_clone_index","household_is_spi_synthetic"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","employee_pension_contributions","pension_contributions_via_salary_sacrifice","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported","employment_status","employment_sector","sic_industry_division","aa_category","dla_sc_category","dla_m_category","pip_m_category","pip_dl_category","is_disabled_for_benefits","is_enhanced_disabled_for_benefits","is_severely_disabled_for_benefits","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right","legacy_jobseeker_proxy","esa_health_condition_proxy","esa_support_group_proxy","disabled_students_allowance_eligible_expenses","would_claim_marriage_allowance","would_claim_scp","attends_private_school_random_draw","student_loan_balance","a_and_e_visits","admitted_patient_visits","outpatient_visits","nhs_a_and_e_spending","nhs_admitted_patient_spending","nhs_outpatient_spending","hmrc_spi_pay","hmrc_spi_unemployment_benefit_income","hmrc_spi_incapacity_benefit_income","ossben_identifiable_subset","srp_regular_code5","employer_pension_contributions","person_source_id","person_support_channel","person_support_clone_index","charitable_investment_gifts","gift_aid","other_investment_income","hmrc_spi_employment_benefits","hmrc_spi_employment_expenses","hmrc_spi_other_social_security_income","hmrc_spi_taxable_termination_pay","hmrc_spi_miscellaneous_employment_income","hmrc_spi_other_income","hmrc_spi_state_pension_income","hmrc_spi_employed_income","hmrc_spi_total_earned_income","hmrc_spi_total_investment_income","hmrc_spi_assessable_income"],"entity":"person","rows":"all"}],"kernel":"uk.stage.uc_reporter_redraw@1","mass":"conserve","outputs":[{"column":"universal_credit_reported","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"}],"params":{"stage":"uc_reporter_redraw","stage_contract_sha256":"23a23df85489708d15c7c0be3d1339ff5e1fe0932abaea0f95fcbf9c6d368205","time_period":"2024"},"population":"uc_reporter_redraw.boundary","sources":["frs"],"structural":"none","weights":null},{"base":"uc_reporter_redraw.boundary","citation":"","description":"Ownership boundary before uc_capital_coherence rewrites.","id":"uc_capital_coherence.boundary","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children","would_claim_child_benefit","child_benefit_opts_out","would_claim_pc","would_claim_uc","would_claim_tfc","would_claim_extended_childcare","would_claim_universal_childcare","would_claim_targeted_childcare","maximum_extended_childcare_hours_usage","benunit_source_id","benunit_support_channel","benunit_support_clone_index"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax","household_owns_tv","would_evade_tv_licence_fee","main_residential_property_purchased_is_first_home","property_purchased","brma","owned_land","property_wealth","corporate_wealth","private_pension_wealth","gross_financial_wealth","net_financial_wealth","main_residence_value","other_residential_property_value","non_residential_property_value","savings","num_vehicles","cash_isa","stocks_and_shares_isa","mortgage_debt","consumer_debt","food_and_non_alcoholic_beverages_consumption","alcohol_and_tobacco_consumption","clothing_and_footwear_consumption","housing_water_and_electricity_consumption","household_furnishings_consumption","health_consumption","transport_consumption","communication_consumption","recreation_consumption","education_consumption","restaurants_and_hotels_consumption","miscellaneous_consumption","petrol_spending","diesel_spending","bus_fare_spending","domestic_energy_consumption","electricity_consumption","gas_consumption","has_fuel_consumption","full_rate_vat_expenditure_rate","dfe_education_spending","rail_subsidy_spending","bus_subsidy_spending","rail_usage","source_household_id","source_year","source_household_key","household_source_id","household_support_channel","household_support_clone_index","household_is_spi_synthetic"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","employee_pension_contributions","pension_contributions_via_salary_sacrifice","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","universal_credit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported","employment_status","employment_sector","sic_industry_division","aa_category","dla_sc_category","dla_m_category","pip_m_category","pip_dl_category","is_disabled_for_benefits","is_enhanced_disabled_for_benefits","is_severely_disabled_for_benefits","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right","legacy_jobseeker_proxy","esa_health_condition_proxy","esa_support_group_proxy","disabled_students_allowance_eligible_expenses","would_claim_marriage_allowance","would_claim_scp","attends_private_school_random_draw","student_loan_balance","a_and_e_visits","admitted_patient_visits","outpatient_visits","nhs_a_and_e_spending","nhs_admitted_patient_spending","nhs_outpatient_spending","hmrc_spi_pay","hmrc_spi_unemployment_benefit_income","hmrc_spi_incapacity_benefit_income","ossben_identifiable_subset","srp_regular_code5","employer_pension_contributions","person_source_id","person_support_channel","person_support_clone_index","charitable_investment_gifts","gift_aid","other_investment_income","hmrc_spi_employment_benefits","hmrc_spi_employment_expenses","hmrc_spi_other_social_security_income","hmrc_spi_taxable_termination_pay","hmrc_spi_miscellaneous_employment_income","hmrc_spi_other_income","hmrc_spi_state_pension_income","hmrc_spi_employed_income","hmrc_spi_total_earned_income","hmrc_spi_total_investment_income","hmrc_spi_assessable_income"],"entity":"person","rows":"all"}],"kernel":"uk.identity@1","mass":"conserve","outputs":[],"params":{},"population":null,"sources":[],"structural":"filter","weights":null},{"base":null,"citation":"","description":"Run UK spine stage uc_capital_coherence.","id":"uc_capital_coherence","inputs":[{"columns":["benunit_support_channel","dependent_children","is_married"],"entity":"benunit","rows":"all"},{"columns":["region"],"entity":"household","rows":"all"},{"columns":["age","person_support_channel","universal_credit_reported"],"entity":"person","rows":"all"}],"kernel":"uk.stage.uc_capital_coherence@1","mass":"conserve","outputs":[{"column":"uc_reported_capital","dtype":"float64","entity":"benunit","ownership":"produced","rows":"all"},{"column":"frs_benunit_capital","dtype":"float64","entity":"benunit","ownership":"produced","rewrite":true,"rows":"all"},{"column":"would_claim_uc","dtype":"bool","entity":"benunit","ownership":"produced","rewrite":true,"rows":"all"}],"params":{"stage":"uc_capital_coherence","stage_contract_sha256":"64024e4ca8659cad2452abe5578071cf8226e693f3cb975cf4778078bf8baf17","time_period":"2024"},"population":"uc_capital_coherence.boundary","sources":["frs"],"structural":"none","weights":null},{"base":null,"citation":"","description":"Run UK spine stage uc_deduction_attributes.","id":"uc_deduction_attributes","inputs":[{"columns":["frs_benunit_capital","would_claim_uc"],"entity":"benunit","rows":"all"},{"columns":["region"],"entity":"household","rows":"all"},{"columns":["age"],"entity":"person","rows":"all"}],"kernel":"uk.stage.uc_deduction_attributes@1","mass":"conserve","outputs":[{"column":"uc_deduction_random_draw","dtype":"float64","entity":"benunit","ownership":"produced","rows":"all"},{"column":"uc_deduction_type_random_draw","dtype":"float64","entity":"benunit","ownership":"produced","rows":"all"},{"column":"uc_latent_deduction_rate","dtype":"float64","entity":"benunit","ownership":"produced","rows":"all"},{"column":"uc_deduction_combination","dtype":"string","entity":"benunit","ownership":"produced","rows":"all"}],"params":{"stage":"uc_deduction_attributes","stage_contract_sha256":"6bc7e3ec1a5712e23ca1b7d2956e8b17f2526b94431ba3e5a9294b04658e4fe0","time_period":"2024"},"population":"uc_capital_coherence.boundary","sources":["frs"],"structural":"none","weights":null},{"base":"uc_capital_coherence.boundary","citation":"","description":"Run structural UK stage cgt_incidence_clone.","id":"cgt_incidence_clone","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children","would_claim_child_benefit","child_benefit_opts_out","would_claim_pc","would_claim_uc","would_claim_tfc","would_claim_extended_childcare","would_claim_universal_childcare","would_claim_targeted_childcare","maximum_extended_childcare_hours_usage","benunit_source_id","benunit_support_channel","benunit_support_clone_index","uc_reported_capital","uc_deduction_random_draw","uc_deduction_type_random_draw","uc_latent_deduction_rate","uc_deduction_combination"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax","household_owns_tv","would_evade_tv_licence_fee","main_residential_property_purchased_is_first_home","property_purchased","brma","owned_land","property_wealth","corporate_wealth","private_pension_wealth","gross_financial_wealth","net_financial_wealth","main_residence_value","other_residential_property_value","non_residential_property_value","savings","num_vehicles","cash_isa","stocks_and_shares_isa","mortgage_debt","consumer_debt","food_and_non_alcoholic_beverages_consumption","alcohol_and_tobacco_consumption","clothing_and_footwear_consumption","housing_water_and_electricity_consumption","household_furnishings_consumption","health_consumption","transport_consumption","communication_consumption","recreation_consumption","education_consumption","restaurants_and_hotels_consumption","miscellaneous_consumption","petrol_spending","diesel_spending","bus_fare_spending","domestic_energy_consumption","electricity_consumption","gas_consumption","has_fuel_consumption","full_rate_vat_expenditure_rate","dfe_education_spending","rail_subsidy_spending","bus_subsidy_spending","rail_usage","source_household_id","source_year","source_household_key","household_source_id","household_support_channel","household_support_clone_index","household_is_spi_synthetic"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","employee_pension_contributions","pension_contributions_via_salary_sacrifice","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","universal_credit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported","employment_status","employment_sector","sic_industry_division","aa_category","dla_sc_category","dla_m_category","pip_m_category","pip_dl_category","is_disabled_for_benefits","is_enhanced_disabled_for_benefits","is_severely_disabled_for_benefits","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right","legacy_jobseeker_proxy","esa_health_condition_proxy","esa_support_group_proxy","disabled_students_allowance_eligible_expenses","would_claim_marriage_allowance","would_claim_scp","attends_private_school_random_draw","student_loan_balance","a_and_e_visits","admitted_patient_visits","outpatient_visits","nhs_a_and_e_spending","nhs_admitted_patient_spending","nhs_outpatient_spending","hmrc_spi_pay","hmrc_spi_unemployment_benefit_income","hmrc_spi_incapacity_benefit_income","ossben_identifiable_subset","srp_regular_code5","employer_pension_contributions","person_source_id","person_support_channel","person_support_clone_index","charitable_investment_gifts","gift_aid","other_investment_income","hmrc_spi_employment_benefits","hmrc_spi_employment_expenses","hmrc_spi_other_social_security_income","hmrc_spi_taxable_termination_pay","hmrc_spi_miscellaneous_employment_income","hmrc_spi_other_income","hmrc_spi_state_pension_income","hmrc_spi_employed_income","hmrc_spi_total_earned_income","hmrc_spi_total_investment_income","hmrc_spi_assessable_income"],"entity":"person","rows":"all"}],"kernel":"uk.stage.expand.cgt_incidence_clone@1","mass":"conserve","outputs":[],"params":{"expand_cells":[["household","household_is_capital_gains_clone","bool"],["person","capital_gains","float64"]],"expand_weight_entity":"household","expand_weight_kind":"importance","stage":"cgt_incidence_clone","stage_contract_sha256":"ee30278543cc0297a5366d855b7753aa04af5518971594c193c52be36bf8a7b4","time_period":"2024"},"population":null,"sources":["frs"],"structural":"expand","weights":null},{"base":null,"citation":"","description":"Own the cells materialized by cgt_incidence_clone.","id":"cgt_incidence_clone.owned","inputs":[],"kernel":"uk.claim@1","mass":"conserve","outputs":[{"column":"household_is_capital_gains_clone","dtype":"bool","entity":"household","ownership":"produced","rows":"all"},{"column":"capital_gains","dtype":"float64","entity":"person","ownership":"produced","rows":"all"}],"params":{"materialized_expand_outputs":["household.household_is_capital_gains_clone","person.capital_gains"]},"population":"cgt_incidence_clone","sources":[],"structural":"none","weights":null},{"base":"cgt_incidence_clone","citation":"","description":"Run structural UK stage cgt_band_donors.","id":"cgt_band_donors","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children","would_claim_child_benefit","child_benefit_opts_out","would_claim_pc","would_claim_uc","would_claim_tfc","would_claim_extended_childcare","would_claim_universal_childcare","would_claim_targeted_childcare","maximum_extended_childcare_hours_usage","benunit_source_id","benunit_support_channel","benunit_support_clone_index","uc_reported_capital","uc_deduction_random_draw","uc_deduction_type_random_draw","uc_latent_deduction_rate","uc_deduction_combination"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax","household_owns_tv","would_evade_tv_licence_fee","main_residential_property_purchased_is_first_home","property_purchased","brma","owned_land","property_wealth","corporate_wealth","private_pension_wealth","gross_financial_wealth","net_financial_wealth","main_residence_value","other_residential_property_value","non_residential_property_value","savings","num_vehicles","cash_isa","stocks_and_shares_isa","mortgage_debt","consumer_debt","food_and_non_alcoholic_beverages_consumption","alcohol_and_tobacco_consumption","clothing_and_footwear_consumption","housing_water_and_electricity_consumption","household_furnishings_consumption","health_consumption","transport_consumption","communication_consumption","recreation_consumption","education_consumption","restaurants_and_hotels_consumption","miscellaneous_consumption","petrol_spending","diesel_spending","bus_fare_spending","domestic_energy_consumption","electricity_consumption","gas_consumption","has_fuel_consumption","full_rate_vat_expenditure_rate","dfe_education_spending","rail_subsidy_spending","bus_subsidy_spending","rail_usage","source_household_id","source_year","source_household_key","household_source_id","household_support_channel","household_support_clone_index","household_is_spi_synthetic","household_is_capital_gains_clone"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","employee_pension_contributions","pension_contributions_via_salary_sacrifice","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","universal_credit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported","employment_status","employment_sector","sic_industry_division","aa_category","dla_sc_category","dla_m_category","pip_m_category","pip_dl_category","is_disabled_for_benefits","is_enhanced_disabled_for_benefits","is_severely_disabled_for_benefits","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right","legacy_jobseeker_proxy","esa_health_condition_proxy","esa_support_group_proxy","disabled_students_allowance_eligible_expenses","would_claim_marriage_allowance","would_claim_scp","attends_private_school_random_draw","student_loan_balance","a_and_e_visits","admitted_patient_visits","outpatient_visits","nhs_a_and_e_spending","nhs_admitted_patient_spending","nhs_outpatient_spending","hmrc_spi_pay","hmrc_spi_unemployment_benefit_income","hmrc_spi_incapacity_benefit_income","ossben_identifiable_subset","srp_regular_code5","employer_pension_contributions","person_source_id","person_support_channel","person_support_clone_index","charitable_investment_gifts","gift_aid","other_investment_income","hmrc_spi_employment_benefits","hmrc_spi_employment_expenses","hmrc_spi_other_social_security_income","hmrc_spi_taxable_termination_pay","hmrc_spi_miscellaneous_employment_income","hmrc_spi_other_income","hmrc_spi_state_pension_income","hmrc_spi_employed_income","hmrc_spi_total_earned_income","hmrc_spi_total_investment_income","hmrc_spi_assessable_income","capital_gains"],"entity":"person","rows":"all"}],"kernel":"uk.stage.expand.cgt_band_donors@1","mass":"free","outputs":[],"params":{"expand_cells":[["household","household_is_cgt_band_donor","bool"],["person","capital_gains","float64"]],"expand_weight_entity":"household","expand_weight_kind":"importance","stage":"cgt_band_donors","stage_contract_sha256":"e10e10c49c0ca2a6a65048c91b683f818e8b6e207a6c6f57a20d76ea299160b0","time_period":"2024"},"population":null,"sources":["frs"],"structural":"expand","weights":null},{"base":null,"citation":"","description":"Own the cells materialized by cgt_band_donors.","id":"cgt_band_donors.owned","inputs":[],"kernel":"uk.claim@1","mass":"conserve","outputs":[{"column":"household_is_cgt_band_donor","dtype":"bool","entity":"household","ownership":"produced","rows":"all"},{"column":"capital_gains","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"}],"params":{"materialized_expand_outputs":["household.household_is_cgt_band_donor"]},"population":"cgt_band_donors","sources":[],"structural":"none","weights":null},{"base":"cgt_band_donors","citation":"","description":"Ownership boundary before hmrc_cgt_gains_spine rewrites.","id":"hmrc_cgt_gains_spine.boundary","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children","would_claim_child_benefit","child_benefit_opts_out","would_claim_pc","would_claim_uc","would_claim_tfc","would_claim_extended_childcare","would_claim_universal_childcare","would_claim_targeted_childcare","maximum_extended_childcare_hours_usage","benunit_source_id","benunit_support_channel","benunit_support_clone_index","uc_reported_capital","uc_deduction_random_draw","uc_deduction_type_random_draw","uc_latent_deduction_rate","uc_deduction_combination"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax","household_owns_tv","would_evade_tv_licence_fee","main_residential_property_purchased_is_first_home","property_purchased","brma","owned_land","property_wealth","corporate_wealth","private_pension_wealth","gross_financial_wealth","net_financial_wealth","main_residence_value","other_residential_property_value","non_residential_property_value","savings","num_vehicles","cash_isa","stocks_and_shares_isa","mortgage_debt","consumer_debt","food_and_non_alcoholic_beverages_consumption","alcohol_and_tobacco_consumption","clothing_and_footwear_consumption","housing_water_and_electricity_consumption","household_furnishings_consumption","health_consumption","transport_consumption","communication_consumption","recreation_consumption","education_consumption","restaurants_and_hotels_consumption","miscellaneous_consumption","petrol_spending","diesel_spending","bus_fare_spending","domestic_energy_consumption","electricity_consumption","gas_consumption","has_fuel_consumption","full_rate_vat_expenditure_rate","dfe_education_spending","rail_subsidy_spending","bus_subsidy_spending","rail_usage","source_household_id","source_year","source_household_key","household_source_id","household_support_channel","household_support_clone_index","household_is_spi_synthetic","household_is_capital_gains_clone","household_is_cgt_band_donor"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","employee_pension_contributions","pension_contributions_via_salary_sacrifice","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","universal_credit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported","employment_status","employment_sector","sic_industry_division","aa_category","dla_sc_category","dla_m_category","pip_m_category","pip_dl_category","is_disabled_for_benefits","is_enhanced_disabled_for_benefits","is_severely_disabled_for_benefits","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right","legacy_jobseeker_proxy","esa_health_condition_proxy","esa_support_group_proxy","disabled_students_allowance_eligible_expenses","would_claim_marriage_allowance","would_claim_scp","attends_private_school_random_draw","student_loan_balance","a_and_e_visits","admitted_patient_visits","outpatient_visits","nhs_a_and_e_spending","nhs_admitted_patient_spending","nhs_outpatient_spending","hmrc_spi_pay","hmrc_spi_unemployment_benefit_income","hmrc_spi_incapacity_benefit_income","ossben_identifiable_subset","srp_regular_code5","employer_pension_contributions","person_source_id","person_support_channel","person_support_clone_index","charitable_investment_gifts","gift_aid","other_investment_income","hmrc_spi_employment_benefits","hmrc_spi_employment_expenses","hmrc_spi_other_social_security_income","hmrc_spi_taxable_termination_pay","hmrc_spi_miscellaneous_employment_income","hmrc_spi_other_income","hmrc_spi_state_pension_income","hmrc_spi_employed_income","hmrc_spi_total_earned_income","hmrc_spi_total_investment_income","hmrc_spi_assessable_income","capital_gains"],"entity":"person","rows":"all"}],"kernel":"uk.identity@1","mass":"conserve","outputs":[],"params":{},"population":null,"sources":[],"structural":"filter","weights":null},{"base":null,"citation":"","description":"Run UK spine stage hmrc_cgt_gains_spine.","id":"hmrc_cgt_gains_spine","inputs":[{"columns":["frs_benunit_capital"],"entity":"benunit","rows":"all"},{"columns":["region"],"entity":"household","rows":"all"},{"columns":["age","dividend_income","employment_income","miscellaneous_income","private_pension_income","property_income","savings_interest_income","self_employment_income","state_pension_reported","tax_free_savings_income"],"entity":"person","rows":"all"}],"kernel":"uk.stage.hmrc_cgt_gains_spine@1","mass":"conserve","outputs":[{"column":"capital_gains","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"}],"params":{"stage":"hmrc_cgt_gains_spine","stage_contract_sha256":"20ada8d8ee94400bd160223e865a910db7aff9c40a7f0c8ee8ca8c40ed901b72","time_period":"2024"},"population":"hmrc_cgt_gains_spine.boundary","sources":["frs"],"structural":"none","weights":null},{"base":null,"citation":"","description":"Run UK spine stage salary_sacrifice.","id":"salary_sacrifice","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children","would_claim_child_benefit","child_benefit_opts_out","would_claim_pc","would_claim_uc","would_claim_tfc","would_claim_extended_childcare","would_claim_universal_childcare","would_claim_targeted_childcare","maximum_extended_childcare_hours_usage","benunit_source_id","benunit_support_channel","benunit_support_clone_index","uc_reported_capital","uc_deduction_random_draw","uc_deduction_type_random_draw","uc_latent_deduction_rate","uc_deduction_combination"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax","household_owns_tv","would_evade_tv_licence_fee","main_residential_property_purchased_is_first_home","property_purchased","brma","owned_land","property_wealth","corporate_wealth","private_pension_wealth","gross_financial_wealth","net_financial_wealth","main_residence_value","other_residential_property_value","non_residential_property_value","savings","num_vehicles","cash_isa","stocks_and_shares_isa","mortgage_debt","consumer_debt","food_and_non_alcoholic_beverages_consumption","alcohol_and_tobacco_consumption","clothing_and_footwear_consumption","housing_water_and_electricity_consumption","household_furnishings_consumption","health_consumption","transport_consumption","communication_consumption","recreation_consumption","education_consumption","restaurants_and_hotels_consumption","miscellaneous_consumption","petrol_spending","diesel_spending","bus_fare_spending","domestic_energy_consumption","electricity_consumption","gas_consumption","has_fuel_consumption","full_rate_vat_expenditure_rate","dfe_education_spending","rail_subsidy_spending","bus_subsidy_spending","rail_usage","source_household_id","source_year","source_household_key","household_source_id","household_support_channel","household_support_clone_index","household_is_spi_synthetic","household_is_capital_gains_clone","household_is_cgt_band_donor"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","universal_credit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported","employment_status","employment_sector","sic_industry_division","aa_category","dla_sc_category","dla_m_category","pip_m_category","pip_dl_category","is_disabled_for_benefits","is_enhanced_disabled_for_benefits","is_severely_disabled_for_benefits","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right","legacy_jobseeker_proxy","esa_health_condition_proxy","esa_support_group_proxy","disabled_students_allowance_eligible_expenses","would_claim_marriage_allowance","would_claim_scp","attends_private_school_random_draw","student_loan_balance","a_and_e_visits","admitted_patient_visits","outpatient_visits","nhs_a_and_e_spending","nhs_admitted_patient_spending","nhs_outpatient_spending","hmrc_spi_pay","hmrc_spi_unemployment_benefit_income","hmrc_spi_incapacity_benefit_income","ossben_identifiable_subset","srp_regular_code5","employer_pension_contributions","person_source_id","person_support_channel","person_support_clone_index","charitable_investment_gifts","gift_aid","other_investment_income","hmrc_spi_employment_benefits","hmrc_spi_employment_expenses","hmrc_spi_other_social_security_income","hmrc_spi_taxable_termination_pay","hmrc_spi_miscellaneous_employment_income","hmrc_spi_other_income","hmrc_spi_state_pension_income","hmrc_spi_employed_income","hmrc_spi_total_earned_income","hmrc_spi_total_investment_income","hmrc_spi_assessable_income","capital_gains"],"entity":"person","rows":"all"}],"kernel":"uk.stage.salary_sacrifice@1","mass":"conserve","outputs":[{"column":"pension_contributions_via_salary_sacrifice","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"},{"column":"employee_pension_contributions","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"}],"params":{"stage":"salary_sacrifice","stage_contract_sha256":"a4af925af21cc6766eb7c8855b6743a3aa4eeeb38df12cfc83063d7ad6890539","time_period":"2024"},"population":"hmrc_cgt_gains_spine.boundary","sources":["frs"],"structural":"none","weights":null},{"base":null,"citation":"","description":"Run UK spine stage student_loans.","id":"student_loans","inputs":[{"columns":["frs_benunit_capital"],"entity":"benunit","rows":"all"},{"columns":["region"],"entity":"household","rows":"all"},{"columns":["age","current_education","employee_pension_contributions","highest_education","student_loan_repayments","student_loans"],"entity":"person","rows":"all"}],"kernel":"uk.stage.student_loans@1","mass":"conserve","outputs":[{"column":"student_loan_plan","dtype":"string","entity":"person","ownership":"produced","rows":"all"}],"params":{"stage":"student_loans","stage_contract_sha256":"7c4c76398c2e80b41c7f941a5f06246b57d89b5eb0c36d42e3116ac35e82e203","time_period":"2024"},"population":"hmrc_cgt_gains_spine.boundary","sources":["frs"],"structural":"none","weights":null},{"base":"hmrc_cgt_gains_spine.boundary","citation":"","description":"Ownership boundary before age_tail rewrites.","id":"age_tail.boundary","inputs":[{"columns":["frs_benunit_capital","is_married","dependent_children","would_claim_child_benefit","child_benefit_opts_out","would_claim_pc","would_claim_uc","would_claim_tfc","would_claim_extended_childcare","would_claim_universal_childcare","would_claim_targeted_childcare","maximum_extended_childcare_hours_usage","benunit_source_id","benunit_support_channel","benunit_support_clone_index","uc_reported_capital","uc_deduction_random_draw","uc_deduction_type_random_draw","uc_latent_deduction_rate","uc_deduction_combination"],"entity":"benunit","rows":"all"},{"columns":["region","tenure_type","accommodation_type","num_bedrooms","council_tax_reported","council_tax_band","council_tax_rebate","council_tax_single_adult_raw","water_and_sewerage_charges","domestic_rates","rent","subrent","mortgage_interest_repayment","mortgage_capital_repayment","structural_insurance_payments","housing_service_charges","external_child_payments","council_tax","household_owns_tv","would_evade_tv_licence_fee","main_residential_property_purchased_is_first_home","property_purchased","brma","owned_land","property_wealth","corporate_wealth","private_pension_wealth","gross_financial_wealth","net_financial_wealth","main_residence_value","other_residential_property_value","non_residential_property_value","savings","num_vehicles","cash_isa","stocks_and_shares_isa","mortgage_debt","consumer_debt","food_and_non_alcoholic_beverages_consumption","alcohol_and_tobacco_consumption","clothing_and_footwear_consumption","housing_water_and_electricity_consumption","household_furnishings_consumption","health_consumption","transport_consumption","communication_consumption","recreation_consumption","education_consumption","restaurants_and_hotels_consumption","miscellaneous_consumption","petrol_spending","diesel_spending","bus_fare_spending","domestic_energy_consumption","electricity_consumption","gas_consumption","has_fuel_consumption","full_rate_vat_expenditure_rate","dfe_education_spending","rail_subsidy_spending","bus_subsidy_spending","rail_usage","source_household_id","source_year","source_household_key","household_source_id","household_support_channel","household_support_clone_index","household_is_spi_synthetic","household_is_capital_gains_clone","household_is_cgt_band_donor"],"entity":"household","rows":"all"},{"columns":["age","gender","marital_status","hours_worked","is_household_head","is_benunit_head","is_parent","employment_income","self_employment_income","private_pension_income","tax_free_savings_income","savings_interest_income","dividend_income","property_income","maintenance_income","miscellaneous_income","private_transfer_income","lump_sum_income","student_loan_repayments","statutory_sick_pay","statutory_maternity_pay","student_loans","access_fund","education_grants","healthy_start_vouchers","free_school_breakfasts","free_school_fruit_veg","free_school_meals","council_tax_benefit_reported","maintenance_expenses","childcare_expenses","personal_pension_contributions","employee_pension_contributions","pension_contributions_via_salary_sacrifice","salary_sacrifice_reported","salary_sacrifice_asked","child_benefit_reported","income_support_reported","housing_benefit_reported","attendance_allowance_reported","dla_sc_reported","dla_m_reported","iidb_reported","carers_allowance_reported","sda_reported","afcs_reported","ssmg_reported","pension_credit_reported","child_tax_credit_reported","working_tax_credit_reported","state_pension_reported","winter_fuel_allowance_reported","incapacity_benefit_reported","universal_credit_reported","pip_m_reported","pip_dl_reported","jsa_contrib_reported","jsa_income_reported","esa_contrib_reported","esa_income_reported","bsp_reported","employment_status","employment_sector","sic_industry_division","aa_category","dla_sc_category","dla_m_category","pip_m_category","pip_dl_category","is_disabled_for_benefits","is_enhanced_disabled_for_benefits","is_severely_disabled_for_benefits","current_education","highest_education","is_in_non_advanced_education","is_in_approved_training","age_started_or_accepted_current_education_or_training","is_before_universal_credit_qualifying_young_person_terminal_date","adult_ema","child_ema","receives_benefits_in_own_right","legacy_jobseeker_proxy","esa_health_condition_proxy","esa_support_group_proxy","disabled_students_allowance_eligible_expenses","would_claim_marriage_allowance","would_claim_scp","attends_private_school_random_draw","student_loan_balance","a_and_e_visits","admitted_patient_visits","outpatient_visits","nhs_a_and_e_spending","nhs_admitted_patient_spending","nhs_outpatient_spending","hmrc_spi_pay","hmrc_spi_unemployment_benefit_income","hmrc_spi_incapacity_benefit_income","ossben_identifiable_subset","srp_regular_code5","employer_pension_contributions","person_source_id","person_support_channel","person_support_clone_index","charitable_investment_gifts","gift_aid","other_investment_income","hmrc_spi_employment_benefits","hmrc_spi_employment_expenses","hmrc_spi_other_social_security_income","hmrc_spi_taxable_termination_pay","hmrc_spi_miscellaneous_employment_income","hmrc_spi_other_income","hmrc_spi_state_pension_income","hmrc_spi_employed_income","hmrc_spi_total_earned_income","hmrc_spi_total_investment_income","hmrc_spi_assessable_income","capital_gains","student_loan_plan"],"entity":"person","rows":"all"}],"kernel":"uk.identity@1","mass":"conserve","outputs":[],"params":{},"population":null,"sources":[],"structural":"filter","weights":null},{"base":null,"citation":"","description":"Run UK spine stage age_tail.","id":"age_tail","inputs":[{"columns":["frs_benunit_capital"],"entity":"benunit","rows":"all"},{"columns":["region"],"entity":"household","rows":"all"},{"columns":["gender","person_source_id","student_loan_plan"],"entity":"person","rows":"all"}],"kernel":"uk.stage.age_tail@1","mass":"conserve","outputs":[{"column":"age","dtype":"float64","entity":"person","ownership":"produced","rewrite":true,"rows":"all"}],"params":{"stage":"age_tail","stage_contract_sha256":"5f4fa9cd1c55cae6e61a4c046a249d91a4222fa792a02aa652b5c71dedbdcfee","time_period":"2024"},"population":"age_tail.boundary","sources":["frs"],"structural":"none","weights":null}],"sources":[{"codec":"csv-tables","description":"Content-bound UK FRS and donor fixture/source bundle.","name":"frs"}]} diff --git a/packages/microcosm-graph/tests/test_acceptance_h_parity.py b/packages/microcosm-graph/tests/test_acceptance_h_parity.py index 18c2ca112..7045973e0 100644 --- a/packages/microcosm-graph/tests/test_acceptance_h_parity.py +++ b/packages/microcosm-graph/tests/test_acceptance_h_parity.py @@ -40,7 +40,7 @@ #: implementation hash, and the dependency versions the pin was taken under). KERNEL_PARITY = PARITY / "kernels" -#: H2: ``uk_spine.json`` — the 27-stage FRS spine expressed as a graph — plus +#: H2: ``uk_spine.json`` — the 28-stage FRS spine expressed as a graph — plus #: ``sources/``, the data-only bundle both the graph and the legacy oracle #: rebuild their transforms from. The root transform's weights differ at the #: last bit between machines, so both sides recompute the root from the raw @@ -215,12 +215,12 @@ def test_h2_uk_spine_parity(tmp_path: Path) -> None: engine tier (``requires_uk``) and skips in the engine-free fast lane. Expects ``packages/microcosm-graph/tests/fixtures/parity/uk_spine/`` with - ``uk_spine.json`` and ``sources/``; both sides run the 27 transforms from + ``uk_spine.json`` and ``sources/``; both sides run the 28 transforms from those sources in this process, root included, because the root weights differ at the last bit between machines. Stage order comes from declared ``consumes``: the assertion below is that the compiled topological order is derived, so the hand-maintained ``_STAGE_NAMES`` tuple - in ``tools/build_uk_frs_spine.py`` — the 27 names intersected with a + in ``tools/build_uk_frs_spine.py`` — the 28 names intersected with a 28-stage packaged manifest, kept in step by hand — can be deleted. """ _require(UK_SPINE_PARITY, "the UK migration lane (charter H2, María reviews)") @@ -245,7 +245,7 @@ def test_h2_uk_spine_parity(tmp_path: Path) -> None: graph = uk_spine_graph() assert graph_from_json((UK_SPINE_PARITY / "uk_spine.json").read_text()) == graph compiled = compile_graph(graph) - assert len(compiled.order) >= 28, "a CREATE node plus the 27 spine stages" + assert len(compiled.order) >= 29, "a CREATE node plus the 28 spine stages" assert all( set(compiled.predecessors[node_id]) <= set(compiled.order[:index]) for index, node_id in enumerate(compiled.order) diff --git a/tools/build_uk_frs_spine.py b/tools/build_uk_frs_spine.py index f2c9a24dd..3512a79e2 100644 --- a/tools/build_uk_frs_spine.py +++ b/tools/build_uk_frs_spine.py @@ -105,6 +105,9 @@ from microcosm.build.uk_runtime.uc_capital_coherence import ( UKUCCapitalCoherenceStageTransform, ) +from microcosm.build.uk_runtime.uc_deduction_attributes import ( + UKUCDeductionAttributesStageTransform, +) from microcosm.build.uk_runtime.uc_reporter_redraw import ( UKUCReporterRedrawStageTransform, ) @@ -1202,6 +1205,12 @@ def main(argv: list[str] | None = None) -> int: stage=stages_by_name["uc_capital_coherence"] ) ) + if "uc_deduction_attributes" in stage_names: + implementations["uc_deduction_attributes"] = ( + UKUCDeductionAttributesStageTransform( + stage=stages_by_name["uc_deduction_attributes"] + ) + ) if "cgt_incidence_clone" in stage_names: implementations["cgt_incidence_clone"] = UKCGTIncidenceCloneStageTransform( stage=stages_by_name["cgt_incidence_clone"] diff --git a/tools/graph_uk_spine_fixture.py b/tools/graph_uk_spine_fixture.py index 67634ffb6..1f6b2a554 100644 --- a/tools/graph_uk_spine_fixture.py +++ b/tools/graph_uk_spine_fixture.py @@ -1,7 +1,7 @@ #!/usr/bin/env python3 """Regenerate the hermetic charter-H2 UK spine parity fixture. -The oracle is the legacy :class:`microcosm.build.plan.StagePlan`: all 27 +The oracle is the legacy :class:`microcosm.build.plan.StagePlan`: all 28 stages are the current production transform classes. Private source files are replaced only through their supported parsed-input seams. The bundle's ``fixture.json`` is deliberately data-only so the graph's unbound UK registry @@ -99,6 +99,9 @@ from microcosm.build.uk_runtime.uc_capital_coherence import ( UKUCCapitalCoherenceStageTransform, ) +from microcosm.build.uk_runtime.uc_deduction_attributes import ( + UKUCDeductionAttributesStageTransform, +) from microcosm.build.uk_runtime.uc_reporter_redraw import ( UKUCReporterRedrawStageTransform, ) @@ -123,11 +126,11 @@ _SPI_DONOR_SAMPLE_SIZE = 64 #: The packaged FRS spine roster the fixture exercises (manifest minus the #: certified-pair exclusions); moves whenever a spine stage is added. -UK_FIXTURE_STAGE_COUNT = 27 +UK_FIXTURE_STAGE_COUNT = 28 _QRF_ESTIMATORS = 4 # These are the complete object-string surface observed in the unchanged -# legacy 27-stage output. Graph storage uses pandas StringDtype/python. +# legacy 28-stage output. Graph storage uses pandas StringDtype/python. _NORMALIZED_STRING_COLUMNS: Mapping[str, tuple[str, ...]] = { "person": ( "gender", @@ -144,7 +147,7 @@ "person_support_channel", "student_loan_plan", ), - "benunit": ("benunit_support_channel",), + "benunit": ("benunit_support_channel", "uc_deduction_combination"), "household": ( "region", "tenure_type", @@ -563,6 +566,13 @@ def _was_donor() -> pd.DataFrame: "DVPriRntR8": 1 + rows.astype(int) % 2, "CTAmtR8": 900.0 + rows * 10.0, "HFINWNTR8_Sum": -50.0 + rows * 4.0, + "HFINWNTR8_exSLC_Sum": 20.0 + rows - rows % 5, + "HMortGR8": np.where( + np.isin(np.take([1, 2, 3, 4], rows.astype(int) % 4), [2, 3]), + 10_000.0 + rows * 100.0, + 0.0, + ), + "Ten1R8": np.take([1, 2, 3, 4], rows.astype(int) % 4), "HFINWR8_SUM": 30.0 + rows, "DVhvalueR8": 100_000.0 + rows * 2_000.0, "DVHseValR8_sum": 1_000.0 + rows * 50.0, @@ -853,7 +863,7 @@ def _normalization_markdown() -> str: lines = [ "# UK spine parity string normalization", "", - "The unchanged legacy transforms retain these 21 textual table columns as", + "The unchanged legacy transforms retain these 22 textual table columns as", "pandas `object`. The frozen graph dtype token `string` is specified by", 'interface-freeze amendment 10 as pandas `StringDtype(storage="python")`.', "Before computing the legacy oracle's `uk_frame_content_identity` (live,", @@ -909,8 +919,8 @@ def _normalize_legacy_strings(frame: Frame) -> Frame: for entity, columns in _NORMALIZED_STRING_COLUMNS.items() for column in columns ] - if observed != expected_order or len(observed) != 21: - raise RuntimeError("The legacy normalization audit is not exactly 21 cells.") + if observed != expected_order or len(observed) != 22: + raise RuntimeError("The legacy normalization audit is not exactly 22 cells.") return Frame( tables, frame.schema, @@ -1071,6 +1081,9 @@ def capture_root(frame: Frame) -> Frame: "uc_capital_coherence": UKUCCapitalCoherenceStageTransform( stage=stages["uc_capital_coherence"] ), + "uc_deduction_attributes": UKUCDeductionAttributesStageTransform( + stage=stages["uc_deduction_attributes"] + ), "cgt_incidence_clone": UKCGTIncidenceCloneStageTransform( stage=stages["cgt_incidence_clone"] ), @@ -1098,7 +1111,7 @@ def _run_legacy_plan( stages: Iterable[SourceStageSpec], implementations: Mapping[str, object], ) -> Frame: - """Run the legacy 27-stage StagePlan oracle and return its final frame.""" + """Run the legacy 28-stage StagePlan oracle and return its final frame.""" stages = tuple(stages) committed = load_country_spec("uk") @@ -1131,7 +1144,7 @@ def _run_legacy_plan( def legacy_oracle_frame(fixture: Path) -> Frame: """The legacy oracle's final frame on the fixture, computed live. - Rebuilds the same 26 transforms the graph's UK registry reconstructs from + Rebuilds the same 28 transforms the graph's UK registry reconstructs from ``fixture/sources``, runs them through the legacy StagePlan in this process, root included, and applies the one string normalization the fixture documents. The content identity is a byte-exact fingerprint of @@ -1210,7 +1223,7 @@ def generate(output: Path) -> None: descriptor = { "schema_version": "uk-spine-parity-fixture.v1", "description": ( - "Data-only inputs for reconstructing the same 26 current UK stage " + "Data-only inputs for reconstructing the same 28 current UK stage " "transform classes used by the legacy StagePlan oracle." ), "stages": {stage.stage: _stage_payload(stage) for stage in stages}, @@ -1246,7 +1259,7 @@ def generate(output: Path) -> None: _normalization_markdown(), encoding="utf-8" ) (output / "PRODUCED_BY.txt").write_text( - "tools/graph_uk_spine_fixture.py; current 26-transform legacy " + "tools/graph_uk_spine_fixture.py; current 28-transform legacy " "StagePlan oracle with parsed private-source seams. The acceptance " "test runs both sides from frs_raw in-process (root weights differ " "by one ulp between machines); the captured root tables serve the " diff --git a/tools/verify_uk_identity_stability.py b/tools/verify_uk_identity_stability.py index 2b9246aa9..48b546a45 100644 --- a/tools/verify_uk_identity_stability.py +++ b/tools/verify_uk_identity_stability.py @@ -814,9 +814,7 @@ def select_donors(person_t: pd.DataFrame) -> np.ndarray: # Stage-time age basis again: the stage picked each donor household's # carrier before age_tail ran. donor_person["age"] = np.minimum( - pd.to_numeric(donor_person["age"], errors="coerce").to_numpy( - dtype=float - ), + pd.to_numeric(donor_person["age"], errors="coerce").to_numpy(dtype=float), float(UK_AGE_TOP_CODE), ) carrier_rows = _oldest_adult_indices( @@ -910,11 +908,149 @@ def select_donors(person_t: pd.DataFrame) -> np.ndarray: } +def e9_identity_receipt( + frame, + *, + permutation_seed: int, +) -> dict[str, object]: + """Recompute the E9 UC deduction attributes in original and permuted order. + + The four benunit columns are pure functions of ``benunit_id`` and the + household region under the committed resource, so they are re-derived + here in full and compared with the stored artifact. The stage runs + before CGT cloning and band-donor stacking, which copy the completed + columns onto re-keyed rows; those rows are excluded and their count is + reported, since their ids are not the ids the stage drew on. + """ + + from microcosm.build.uk_runtime.cgt_structure import ( + HOUSEHOLD_IS_CGT_BAND_DONOR, + HOUSEHOLD_IS_CGT_CLONE, + ) + from microcosm.build.uk_runtime.uc_deduction_attributes import ( + UC_DEDUCTION_OUTPUT_COLUMNS, + UK_UC_DEDUCTION_ATTRIBUTES_DECLARED_SEEDS, + _banded_rate_mapping, + _identity_float32_uniforms, + load_uc_deduction_distributions, + map_uniform_to_categorical, + validate_uc_deduction_resource, + ) + + resource = load_uc_deduction_distributions() + validate_uc_deduction_resource(resource) + person = frame.table("person") + benunit = frame.table("benunit") + household = frame.table("household") + copied_flags = [ + column + for column in (HOUSEHOLD_IS_CGT_CLONE, HOUSEHOLD_IS_CGT_BAND_DONOR) + if column in household.columns + ] + copied_households = ( + household[copied_flags].astype(bool).any(axis=1) + if copied_flags + else pd.Series(False, index=household.index) + ) + copied_ids = set(household.loc[copied_households, "household_id"]) + benunit_household = person.drop_duplicates("person_benunit_id").set_index( + "person_benunit_id" + )["person_household_id"] + benunit_scope = ~benunit["benunit_id"].map(benunit_household).isin(copied_ids) + scoped = benunit.loc[benunit_scope.to_numpy()].reset_index(drop=True) + region_by_household = household.set_index("household_id")["region"] + + def recompute(benunit_t: pd.DataFrame) -> pd.DataFrame: + ids = benunit_t["benunit_id"].to_numpy() + regions = ( + benunit_t["benunit_id"] + .map(benunit_household) + .map(region_by_household) + .map(_enum_name) + .to_numpy(dtype=object) + ) + u = _identity_float32_uniforms( + ids, + seed=UK_UC_DEDUCTION_ATTRIBUTES_DECLARED_SEEDS["uc_deduction_random_draw"], + salt="uc_deduction_random_draw", + ) + v = _identity_float32_uniforms( + ids, + seed=UK_UC_DEDUCTION_ATTRIBUTES_DECLARED_SEEDS[ + "uc_deduction_type_random_draw" + ], + salt="uc_deduction_type_random_draw", + ) + mapping = _banded_rate_mapping(u, regions, resource) + combination = map_uniform_to_categorical( + v, gate=mapping.rates > 0.0, resource=resource + ) + return pd.DataFrame( + { + "uc_deduction_random_draw": u, + "uc_deduction_type_random_draw": v, + "uc_latent_deduction_rate": mapping.rates, + "uc_deduction_combination": combination.astype(str), + }, + index=ids, + ) + + original = recompute(scoped) + rng = np.random.default_rng(permutation_seed) + permuted = recompute( + scoped.iloc[rng.permutation(len(scoped))].reset_index(drop=True) + ) + stored = benunit.set_index("benunit_id") + permutation_mismatches: list[str] = [] + stored_mismatches: list[str] = [] + stored_columns_missing: list[str] = [] + for column in UC_DEDUCTION_OUTPUT_COLUMNS: + left = original[column] + right = permuted[column].reindex(left.index) + if not np.array_equal(left.to_numpy(), right.to_numpy()): + permutation_mismatches.append(column) + if column not in stored.columns: + stored_columns_missing.append(column) + continue + kept = stored[column].reindex(left.index) + if column == "uc_deduction_combination": + equal = np.array_equal( + left.to_numpy().astype(str), kept.map(_enum_name).to_numpy().astype(str) + ) + else: + equal = np.array_equal(left.to_numpy(), kept.to_numpy(dtype=float)) + if not equal: + stored_mismatches.append(column) + return { + "check": "uk_e9_identity_stability", + "permutation_seed": permutation_seed, + "identical_under_permutation": not permutation_mismatches, + "matches_stored_columns": not stored_mismatches and not stored_columns_missing, + "permutation_mismatches": {"benunit": permutation_mismatches} + if permutation_mismatches + else {}, + "stored_mismatches": {"benunit": stored_mismatches} + if stored_mismatches + else {}, + "stored_columns_missing": {"benunit": stored_columns_missing} + if stored_columns_missing + else {}, + "columns_by_entity": {"benunit": list(UC_DEDUCTION_OUTPUT_COLUMNS)}, + "benunits_recomputed": int(len(scoped)), + "benunits_excluded_as_copies": int(len(benunit) - len(scoped)), + "entity_row_counts": { + entity: int(len(frame.table(entity))) for entity in frame.entities + }, + } + + def main() -> int: parser = argparse.ArgumentParser(description=__doc__) parser.add_argument("--input-h5", type=Path, required=True) parser.add_argument("--output", type=Path, required=True) - parser.add_argument("--check", choices=("e4", "e5", "e6", "e7", "e8"), default="e4") + parser.add_argument( + "--check", choices=("e4", "e5", "e6", "e7", "e8", "e9"), default="e4" + ) parser.add_argument("--permutation-seed", type=int, default=123) args = parser.parse_args() @@ -974,6 +1110,16 @@ def main() -> int: ok = bool( receipt["identical_under_permutation"] and receipt["matches_stored_columns"] ) + elif args.check == "e9": + # E9 drew on every benunit present after the SPI stack and before CGT + # cloning; the block excludes the cloned and band-donor copies itself. + receipt = e9_identity_receipt( + frame, + permutation_seed=args.permutation_seed, + ) + ok = bool( + receipt["identical_under_permutation"] and receipt["matches_stored_columns"] + ) else: receipt = e8_identity_receipt( frame,