Skip to content

UK spine: benefit-unit-grain SPI UC reporter draw with pre-take-up screen (#832); exact capital sentinel (#833) - #835

Merged
juaristi22 merged 7 commits into
mainfrom
uk-uc-reporter-redraw-832
Sep 2, 2026
Merged

UK spine: benefit-unit-grain SPI UC reporter draw with pre-take-up screen (#832); exact capital sentinel (#833)#835
juaristi22 merged 7 commits into
mainfrom
uk-uc-reporter-redraw-832

Conversation

@juaristi22

@juaristi22 juaristi22 commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

Closes #832, with #840 as the next step, plus the #833 sentinel fence fix folded in on request. Base: main (post-#844). Seven commits: the stage with its graph declaration (#832), the sentinel fix (#833), the two licensed-build child-only fixes, the second-review disposition, the H2 parity fixture re-pin, and the third-review close-out (the would_claim_uc lift signature retired on the spine-m actuals, María's ruling).

What this does (#832, fix 1)

New stage uc_reporter_redraw at driver index 19, between hmrc_spi_income_spine and uc_capital_coherence: rewrites universal_credit_reported on SPI-channel rows at benefit-unit grain, replacing the person-grain chain fill that produced ~1.24m weighted dead reporters (uc_income_reduction exhausts the award).

  • Rewrite-after-fill: the stage-2 chain is untouched (universal_credit_reported is chain position 6 of 29 — removing it would redraw 23 downstream columns and move the pinned output surface). Stage-18 outputs stay byte-identical, so the I5 twin compare attributes every diff to this fix.
  • Screen: one temporary engine materialization computes max(0, uc_maximum_amount − uc_income_reduction) > 0 with uc_reported_capital := frs_benunit_capital derived on the temp frame only (nothing engine-known persisted; the coherence stage keeps ownership of the persisted column). Hard restriction: only screened-eligible benefit units can be drawn reporters; the model is trained on the screened base domain so no rate renormalization exists. SPI reporter→positive-award conversion becomes ≈100% by construction.
  • Model: RegimeGatedQRF(seed=44) via its existing DataFrame front door (zero microcosm-fit changes) — the zero/positive regime gate is the income- and composition-conditioned receipt draw; the positive forest draws the amount. Predictors: is_married, UC child band (0/1/2/3+ from the engine's qualifying-young-person flag), BU employment/self-employment/investment income, claimant + partner earnings split, claimant age, region.
  • Landing: full BU amount on the eldest working-age adult (lowest person_id tie-break) — matches the base channel's single-payee pattern (frs_spine.py maps each FRS benefits record to its holder).
  • Downstream uc_capital_coherence picks up the new reporter set automatically (anchor → OR-refresh → capital redraw).

I1 receipts (licensed, on spine-l 5176e6ec…; committed evidence: experiments/832-uc-reporter-receipts.md)

  • Screened-eligible SPI domain 2.416m weighted (14.2%) — demotion-only fallback refuted; only 0.758m of the current 2.001m SPI reporters pass the screen (independently reproduces the issue's 1.249m dead mass).
  • Within-cell earnings gradients 10–100× — the cell hot-deck fallback is dead; QRF validated (held-out 46.0% vs 46.8%, cells within binomial noise, amount quantiles track).
  • SPI application: lone parents +0.131m converters (the largest deferred deficit). Flagged honestly: faithful base rates draw fewer couple-with-children reporters (0.106m) than currently convert (0.152m) — ≈ −0.046m initial pressure on a currently-fitting cell; that composition residual is the Port UK compact source stages and 2024-25 refresh from uk-data #145 draw-quality boundary (plan R3), not stretched here.

#833 (second commit)

Exact sentinel comparison at both domain layers (stage entry + gate binding, including sentinel parity): np.isclose's band admitted ~[−1.00001, −0.99999] as "the sentinel", silently reading near-sentinel corruption as declared absence. Near-sentinel regression tests added at both layers.

Signed differences

Two entries, both I1-derived bounds — I5 re-derives the actuals:

  • uc-reporter-claim-refresh-lift (the UK spine: preserve FRS benefit-unit capital for UC while retaining WAS household wealth #828 lift signature) retired: spine-m measures would_claim_uc at 0.548741, 0.001951 below the incumbent and inside the ±0.02 acceptance band, because the redraw's demotions outweigh the OR-refresh lift — an in-band difference carries no signature, so a future out-of-band move in either direction is flagged as unsigned (Part F of the receipts doc records the ruling).
  • New uc-reporter-benefit-unit-redraw-incidence: person universal_credit_reported nonzero share, incumbent 0.057359, candidate_below, max_abs_delta 0.0184 (structural range [0.038962, 0.067990]; model-expected ≈ 0.050).

Derived surfaces

Schema oneOf branch + op-kind allowlist for redraw_spi_reported_uc; sources.yaml↔source_stages.json lockstep; roster 28→29 stages; UK spec sha re-pin; coverage-manifest sha regeneration; acceptance-receipt binder generalized to the two pending stages (I5 restores strict roster equality when it re-mints the receipt — carried from #829). Not moved, by design: uk/gates.json, contract digests, export allowlist, US surfaces.

Verification

Full UK+machinery funnel green locally (test_uk_* + country_spec + spec_engine + source_runtime + stochastic_assignment + gate_battery + gates + release_input_coverage + data contract), plus ruff and tools/ci_test_groups.py --verify. Two documented pre-existing licensed-cache regeneration failures (engine 2.89.0 committed evidence vs 2.92.1 floor, PR #749's lane) — their inputs are untouched by this diff.

Out of scope / next

Closes #833 and #832. The with-children residual continues as #840 (composition-faithful SPI income draw), which carries the B-lane (num_uc_children) hand-off in its thread; the I5/I6 measurements for this PR are recorded in the comments.

🤖 Generated with Claude Code

juaristi22 added a commit that referenced this pull request Sep 1, 2026
…d blocker)

The first licensed spine-m build failed at uc_reporter_redraw: 'Every
benefit unit must contain an adult claimant candidate.' A 16-19
qualifying young person heading their own benefit unit is its only
member, so the engine's UC child flag leaves no ~uc_child candidate —
the licensed frame carries 238 all-under-18 benefit units, and PR CI can
never reach this path (secrets-free, no licensed builds).

The fallback now takes the unit's eldest member as its de-facto head,
and the fail-closed check moves to the landing, where a POSITIVE draw on
a child claimant refuses — the award screen keeps child-only units out
of the drawn domain, so a positive landing there is exactly the
corruption the old total guard feared. Regression tests for both halves.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@vahid-ahmadi

Copy link
Copy Markdown
Contributor

Automated review pass (Claude Code, high effort, diff plus branch reads — no execution). Draft, so nits skipped.

The #833 sentinel fix verifies clean

I filed that issue so I checked it rather than assuming. All four isclose sites went exact togetherout_of_domain, carrier_out_of_domain and sentinel_mismatch in battery_bindings.py, plus uc_capital_coherence.py:121 — which is the part that mattered: had one site gone exact while another stayed tolerant, a value in the old band would have produced a spurious parity mismatch instead of the old silent acceptance. Boundaries behave (-0.0 / +0.0 still pass via >= 0.0, NaN and ±inf still gated by isfinite), and both new tests use -1.000005, which really is inside the old band (5e-6 against rtol 1e-5) — so there is a genuine regression test rather than one that would have passed before. Thanks for folding it in here rather than leaving it to sit.

Four findings on the #832 stage.

1. uk_runtime/uc_reporter_redraw.py:517 — the zeroing is wider than the declared domain

_land_spi_draws zeroes universal_credit_reported on every SPI person row (person.loc[spi_people, ...] = 0.0), not just the declared rows: spi_channel_screened domain. So screen-failing SPI benefit units silently lose the reported UC that the stage-2 chain filled, even though the spec and _assert_stage_parameters declare the rewrite scoped to screened rows.

Either the domain declaration or the write needs to move. As it stands the stage does strictly more than it says, and the thing it does extra is destructive.

2. uc_reporter_redraw.py:261if target.any(): degrades a refusal into a skip, in the destructive direction

If the pre-take-up screen matches no SPI benefit unit — an engine variable renamed, the screen inverted, uc_maximum_amount all zero — the model is never fitted to target, draws stays all-zero, and (given finding 1) every SPI reported-UC row is wiped to 0 with no error. The receipt simply records screened_spi_benunits: 0.

Only the training side has an emptiness guard. An empty screen is not a valid outcome for this stage, so it should refuse rather than proceed to a total wipe; the two findings compound, which is why I would fix them together.

3. uc_reporter_redraw.py:283 — the retention check cannot fail

raise RuntimeError("Screen-failing SPI benefit units retained reported UC.") asserts a property the preceding lines establish by construction: _land_spi_draws has already zeroed all SPI rows, and draws is zero outside target, so reported_after[spi & ~screen] is identically 0. It asserts the code it just ran rather than the intended invariant.

Worth noting this is the same shape as the closure assertion on #795 and the "bit-identical support" control on #815 — and here it is arguably load-bearing in the wrong direction, because a reader seeing a named refusal for exactly finding 1's scenario would reasonably conclude that scenario is fenced.

4. uk/spine_swap_signed_differences.json — the new entry signs less than its own evidence declares reachable

The uc-reporter-benefit-unit-redraw-incidence entry signs direction: candidate_below, max_abs_delta: 0.0184, while its own magnitude evidence states the structural range is [0.038962, 0.067990] around an incumbent of 0.057359. The upper ~+0.0106 of the range the evidence declares reachable is unsigned, so an above-incumbent outcome inside the stated range would read as an unadjudicated beyond-band defect.

Either the band should cover the range the evidence claims, or the evidence should explain why the upper part is unreachable in practice.

Checked and clear

Worth recording, since grain was the thing most likely to go wrong here: the direction is right at every step — person→benunit sums via groupby(person_benunit_id).sum().reindex(benunit_id), benunit→person only through the single-claimant landing. Claimant selection is keyed by lexsort((person_id, -age)), so it is order-independent and matches the declared eldest_working_age_adult_lowest_person_id. _household_values_by_benunit enforces one household per benunit. The stage sits before uc_capital_coherence and after the SPI income chain in both source_stages.json and sources.yaml, with outputs: [] / rewrites: [universal_credit_reported] matching the code assertion, fixed seed=44, and base-FRS rows untouched.


1 and 2 together are the merge-blocker: an empty screen currently wipes every SPI reporter rather than refusing, and finding 3 means the one check that names that scenario cannot catch it.

@juaristi22

Copy link
Copy Markdown
Collaborator Author

Licensed I5/I6 measurement: spine-m + seam v17 (2026-09-01, measurement session)

Ran this PR's I5 (twin rebuild) and the I6 recalibration on María's direction. Comment-only interaction as before — all runs live on uk-publication-stack-835 (= this PR's head 218c9016 + the deferral register, scorer fixes, and two licensed-build fixes described below). Receipts: data/ukds/acceptance/spine-m-832/, runs/uk-623-first-calibrated/spine-assessment-v17/.

Two licensed-build blockers found and fixed on the measurement stack (cherry-pickable)

PR CI is secrets-free, so neither could surface before a licensed build:

  1. _claimant_rows hard-fails on child-only benefit units (88c058ed). "Every benefit unit must contain an adult claimant candidate" fired on the first spine-m attempt: a 16–19 qualifying young person heading their own unit has no ~uc_child member, and the licensed frame carries 238 all-under-18 benefit units. Fix: the unit's eldest member is its de-facto head, and the fail-closed check moves to the landing (a positive draw on a child claimant refuses). Two regression tests.
  2. The award screen admits child-only units — uc_maximum_amount is mechanical (2c406278). The relocated landing guard then fired on the retry: the engine computes a positive pre-take-up award for a QYP-only unit even though it cannot claim, so screen ∧ draw landed positive amounts on child claimants. Fix: the screen additionally requires a non-child member (_has_adult_member); the landing guard stays as the permanent fence. This is worth folding into I3's spec: "positive engine award" ≠ "claimable unit" on the licensed data.

I5 twin receipt — attribution is exact

Third build green (14/14 gates, engine 2.92.1, frame shape unchanged). Column-level twin compare vs spine-l: the entire diff is four columns, all owned by the two UC stagesperson.universal_credit_reported (nonzero reporter records 7,298 → 5,730; dead mass demoted), benunit.frs_benunit_capital + benunit.uc_reported_capital (moving together on the same 2,942 rows — the coherence redraw following the new reporter set), benunit.would_claim_uc (−804 records). Everything upstream and the whole household surface byte-identical. Transition receipt confirms the FRS channel fully held (0 demotions/promotions) with all rewriting on SPI.

I6 verdict — the honest answer to "does this fix the with-children gap": no. It fixes something else that needed fixing.

v16 (spine-l) → v17 (spine-m), initial = pre-solve support, fix-isolated:

Cell init err v16 → v17 fin v16 → v17
households (aggregate) −24.2% → −25.4% −13.7% → −14.5%
couple_no_children +89.7% → +66.2% +0.1% → −0.1%
single_with_children −55.0% → −54.6% −33.6% → −34.2%
children_1 −39.8% → −39.5% −26.6% → −27.9%
children_2 −38.3% → −40.2% −33.6% → −34.4%
children_5_or_more −38.0% → −38.0% −29.1% → −29.1%
couple_with_children −5.3% → −8.1% −0.0% → −1.4%
  • The big structured movement is the correction of the childless-couple overshoot (+89.7 → +66.2 initial): the redraw removed ~1.24m dead/wrong reporters, which is precisely the semantic honesty this PR promised — reporter→positive-award conversion by construction, dead mass gone.
  • The lone-parent +0.131m converters materialized but were nearly offset by demotions of dead with-children reporters: single_with_children initial +0.4pp net, children_1 +0.2pp. The honestly-flagged couple-with-children pressure landed as predicted (−2.8pp initial; its final slips −0.0% → −1.4%). children_2 −1.9pp initial.
  • Net: aggregate initial support −80k, loss 0.02545 → 0.02577, and the four with-children receipts stand essentially unchanged (−27.9 to −34.4%).

Conclusion: this PR makes the reporter channel truthful — a prerequisite — but the with-children caseload deficit survives it intact, and is now even more precisely located: the screened-eligible SPI domain (2.416m weighted) simply does not contain enough with-children claimants at faithful base rates. The B-lane (num_uc_children + floor bump + numbered-child rebind, plan D5) and #145's draw-shape work are where the four standing receipts get resolved.

Housekeeping the run settled

Measurement session (Claude Fable 5, at María's direction).

juaristi22 added a commit that referenced this pull request Sep 1, 2026
…d blocker)

The first licensed spine-m build failed at uc_reporter_redraw: 'Every
benefit unit must contain an adult claimant candidate.' A 16-19
qualifying young person heading their own benefit unit is its only
member, so the engine's UC child flag leaves no ~uc_child candidate —
the licensed frame carries 238 all-under-18 benefit units, and PR CI can
never reach this path (secrets-free, no licensed builds).

The fallback now takes the unit's eldest member as its de-facto head,
and the fail-closed check moves to the landing, where a POSITIVE draw on
a child claimant refuses — the award screen keeps child-only units out
of the drawn domain, so a positive landing there is exactly the
corruption the old total guard feared. Regression tests for both halves.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@juaristi22

Copy link
Copy Markdown
Collaborator Author

Folded the two licensed-build fixes from the spine-m measurement into this PR as their own commits (cherry-picked from uk-publication-stack-835, authorship preserved): the child-only benefit-unit claimant guard (3de56b6, the 238 all-under-18 units) and the non-child-member screen requirement (198cf21, uc_maximum_amount pays QYP-only units mechanically). PR CI cannot reach either path — the licensed twin build is their evidence (spine-m receipts, findings comment above). Targeted suites + ruff green over the fold; CI rerunning.

@juaristi22

Copy link
Copy Markdown
Collaborator Author

Addendum: the full-surface measurement, and a merge assessment (all 364 bound targets, v16 → v18)

The composition table above under-reports what this PR does. Measured across the whole bound surface, fix-isolated (initial layer, pre-solve):

  • 72 of 364 targets moved — all in the UC domain except two ±0.2–0.4pp passported ripples through would_claim_uc (isc.private_school_students, scotgov.scottish_child_payment_spending), exactly the containment the 4-column twin receipt predicts.
  • The fix's largest measured effect is the payment-band distribution, not the composition counts. The dead-reporter mass had been piling into the childless-couple low-payment bands at +250–460% initial overshoot; the redraw removes 15–155pp from ~14 of those bands (COUPLE_NO_CHILDREN 0–1.2k: +457% → +302%; …2.4–3.6k: +272% → +154%). Mean |initial| across the 84 bound uc_payment_dist cells: 81.7% → 74.2% — the gains concentrate exactly where the dead mass sat; the median band barely moves. The solver was already crushing these finals (mean |final| 0.22% → 0.26%), so the win is honest pre-calibration shape, which is what survives into every weight-independent use of the artifact.
  • Gap ledger: outside-25% went 6 → 5obr.capital_gains_tax came back inside; no cell entered; the four with-children receipts plus the private-pension marginal remain, owned by the B-lane and Port UK compact source stages and 2024-25 refresh from uk-data #145 as above.

What got worse — the complete accounting. 115 of 193 final-layer movers slipped, none by more than ~1.4pp (couple_with_children −0.0% → −1.4%, children_1 −26.6% → −27.9%, single_no_children −0.1% → −1.0%); loss 0.02545 → 0.02577; scored rule-1 loss 0.140 → 0.143 (wins 323 → 325). At the initial layer: aggregate UC support net −80k, children_2 −1.9pp, couple_with_children −2.8pp (the PR's own flagged base-rates pressure), and one with-children payment band −25.9% → −37.6% as with-children reporter mass thinned. These are the price of removing support that was right for the wrong reason: the ~1.24m impossible reporters were flattering the aggregate and the with-children counts. No receipt expanded, no new cell breached, no family regressed beyond re-solve noise, and the v18 gate is green with the staging artifact written.

Merge assessment (measurement session's recommendation to María): merge-worthy, on two conditions.

For: (1) semantic correctness that survives outside calibration — reporter→award conversion ≈100% by construction, so dashboards, reform deltas, and take-up analysis stop reading 1.24m impossible claimants; (2) the payment-distribution cleanup is the largest structured UC improvement since the #755 pension split, and it is pre-calibration shape, not solver make-up; (3) it is the foundation the actual gap-closer builds on — the B-lane must draw with-children reporters into an honest channel; (4) the effect is perfectly contained and receipted.

Conditions: (1) fold in the two licensed-build fixes (88c058ed child-only-benunit claimant fallback + landing guard, 2c406278 screen requires a non-child member because uc_maximum_amount is mechanical) — without them the PR cannot complete a licensed build at all; (2) land with the honest framing — a distribution-honesty fix, with the with-children composition gap explicitly still open under the five standing receipts (window 2026-09-30), so the merge is not read as the UC caseload being resolved.

Measurement session (Claude Fable 5); posted at María's direction.

@vahid-ahmadi

Copy link
Copy Markdown
Contributor

Second pass at 198cf21e (Claude Code, high effort; branch checkout, targeted suites run, plus a small probe against the test fixture's stub engine). Two threads to close out: the four findings from the first pass, and the two folded licensed-build commits.

Status of the first-pass findings

None of the four is touched by 3de56b63 or 198cf21e; both commits are confined to the child-only-benunit path. Taking them in order, with one correction to my own earlier framing.

1. Reframed, still open — this is a spec-truthfulness gap, not a destructive bug. Reading the I5 measurement comment made it clear that zeroing screen-failing SPI reporters is the fix — that is the "dead mass demoted" line in the twin receipt. So the write at uc_reporter_redraw.py:549 (person.loc[spi_people, ...] = 0.0 over every SPI row) is the intended behaviour, and I withdraw "destructive". What remains is that the spec no longer matches the code, in two places. The unambiguous one is new: sources.yaml:1954 declares screen: positive_pre_takeup_uc_award, derived at :1937 as max(0, uc_maximum_amount - uc_income_reduction) > 0, but since 198cf21e the code also requires a non-child member (:216, screen &= _has_adult_member(...)), so the declared screen admits units the real one rejects. The second is the one from last round, put more carefully: the spec has training_rows and rows: spi_channel_screened, and rows can be read as the draw domain, but nothing declares the zeroing scope, which is the whole SPI channel. Either rows should say spi_channel or a separate field should declare the rewrite domain; the notes prose ("redraws the SPI reporter state") already says the right thing, the structured parameters should too.

2. Still open, and I would still hold merge on it. :267 if target.any(): is unchanged; only the training side refuses on emptiness (:248). Probed with an engine stub that fails the screen for every SPI benunit (ran it myself): the stage returns successfully, SPI reported UC in the fixture goes from 60.0 to 0.0, and the receipt records screened_spi_benunits: 0. A success-shaped receipt on a total wipe is the one outcome this stage should refuse; the fix is a one-line guard mirroring :248.

3. Still open. :289-290 still asserts what :549 established two lines earlier; it cannot fail.

4. Still open. spine_swap_signed_differences.jsonuc-reporter-benefit-unit-redraw-incidence still signs candidate_below / 0.0184 against a stated structural range of [0.038962, 0.067990] around 0.057359. Arithmetic checks (4428/113649, 7727/113649; lower delta 0.018397). The upper 0.0106 is declared reachable and unsigned. The would_claim_uc entry's 0.0553 is consistent with its evidence.

The two folded commits

The claimant fallback and the adult-member screen are the right fixes: the old total guard genuinely could not survive a qualifying-young-person-only unit, and uc_maximum_amount paying such a unit mechanically is exactly the kind of thing a screen must not trust. Child/adult is defined once (the engine's is_child_or_qualifying_young_person_for_universal_credit) and used consistently at the screen, the claimant selection and the landing, so there is no alignment gap between the three. Three things worth fixing before this lands on the licensed build again:

5. The receipt files child-only units under single_with_children. :584-588 classifies family type as single_with_children whenever children > 0 and not couple. A QYP-only unit has one child and no couple, so every child-only unit on the SPI channel that the new screen zeroes lands in the receipt as spi.single_with_children.demoted — the lone-parent cell the whole #832 story turns on. Verified by running a probe against the test fixture's stub engine: a 17-year-old alone in an SPI benunit carrying stage-2 UC of 300 exits the stage at 0 and the receipt reads single_with_children: {demoted: 1}. Question for the I6 reading rather than a claim: is any of the single_with_children near-wash in the transition receipt made of these units? A child_only family type (or excluding them from the transition table) would make that answerable.

6. The relocated landing guard is unreachable in the pipeline — same shape as finding 3. :556-565 refuses a positive draw on a child claimant. A claimant is a child only via the fallback at :401-408 (no non-child member), which is exactly the condition under which _has_adult_member is false, so screen is false, target is false, and draws is 0 for that unit. test_positive_draw_on_a_child_claimant_refuses_at_the_landing reaches the guard only by hand-feeding draws=[500.0, 500.0], which the stage cannot produce. Fine as defence-in-depth, but the commit message's "fail-closed fence" reads as if the pipeline can trip it, and it cannot.

7. No end-to-end test drives a child-only SPI benunit through the stage. The two new tests are real (the claimant one would have raised before 3de56b63), but both test helpers in isolation. One test that puts a QYP-only SPI unit carrying reported UC through redraw_spi_reported_uc and asserts the outcome would have caught finding 5, and is the test the licensed-build failure was asking for.

Checked and clear

ruff clean; test_uk_uc_reporter_redraw, test_uk_uc_capital_coherence, test_uk_battery_bindings, test_uk_source_stages, test_uk_signed_differences all pass locally (two skips need the policyengine-uk extra). The new commits still write only universal_credit_reported, so the four-column twin attribution is undisturbed; they only change which SPI rows are zeroed. Minor: _has_adult_member .fillna(False) screens out a benunit with no person rows silently rather than refusing, and working_age hard-codes 66 where the engine knows state pension age.


2 is the merge-blocker as before. 1 is now a spec-truthfulness fix rather than a behaviour change, and 5 matters because it corrupts the very cell the measurement session is reading.

@juaristi22

Copy link
Copy Markdown
Collaborator Author

Disposition of the second pass (at 198cf21e), finding by finding. All seven plus the two minors are accepted and fixed in one commit on top of the two folded licensed-build fixes; nothing is deferred except the one item that is María's adjudication.

1. Spec truthfulness — fixed at the spec, not the code. The declared screen is now claimable_screen, derived in the manifest as positive_pre_takeup_uc_award AND has_non_child_member (both derivations spelled out under the stage's derive op), so the spec admits exactly what the code admits. The rewrite domain is now its own structured field: rewrite_rows: spi_channel beside rows: spi_channel_screened (draw domain), with the schema branch, _assert_stage_parameters, and the stage notes updated to say the zeroing scope out loud. The stage-manifest projection, coverage-manifest sha and UK spec sha moved with it.

2. Empty screened SPI domain — refuses. if not target.any(): raise ValueError("… no screened SPI benefit units; refusing to zero the SPI channel."), mirroring the training guard, and the if target.any(): branch is gone. Your probe is now a test: a stub engine that fails the screen for every SPI unit raises instead of returning a success-shaped receipt over a wiped channel.

3. The tautological retention check — replaced by invariants the write did not derive. _assert_landing_invariants checks, from the person table after landing: the base-channel column is byte-identical to a pre-landing copy; every SPI benefit unit's summed amount equals its draw; and each reporting SPI unit carries exactly one positive person row. Each can fail on a mask/index/claimant-selection bug in a different code path from the write, and each has a test that makes it fail.

4. The universal_credit_reported signature — re-derived on the I5 actuals. spine-m measures 5,730 nonzero reporter records of 113,649 (share 0.050418), 0.006941 below the incumbent's 0.057359. The entry keeps candidate_below / 0.0184 as the distance to the structural floor and now states why the above side is unsigned: the built artifact does not reach it (a 100% draw on the 3,299 screened units would be needed against a ~46% base rate). Evidence anchor moved to a new Part F of the receipts doc carrying the spine-m actuals.

5. Child-only units misfiled as single_with_children — fixed. The transition receipt now files any unit with no non-child member under its own child_only label before the couple/children split, so the lone-parent cell the measurement reads is clean. The I6 question you posed (is any of the single_with_children near-wash made of these units?) becomes answerable on the next licensed build; on spine-m it was not separable, which is precisely the defect.

6. The landing guard's framing — corrected. Comment now says what it is: defence in depth against hand-fed draws and a future screen regression, unreachable through the stage's own screen. The commit message that called it a pipeline fence stands as history.

7. End-to-end child-only test — added. test_child_only_spi_benunit_end_to_end_is_zeroed_and_filed_as_child_only drives a 17-year-old QYP-only SPI unit carrying a chain fill of 300 through redraw_spi_reported_uc with a stub engine that pays it (the mechanical uc_maximum_amount case): exits at 0, counted in screen_failed_spi_benunits, receipt reads child_only: {demoted: 1}, and single_with_children is absent from the SPI receipt.

Minors, both taken. _has_adult_member refuses a benefit unit with no person rows instead of .fillna(False). Working age now comes from the engine's is_SP_age, materialized alongside the screen (the fourth UC_REPORTER_SCREEN_VARIABLES entry), not a hard-coded 66; a test pins that an engine-flagged SPA-age elder yields the younger adult as claimant.

One item that is an adjudication, not a fix. The superseded uc-reporter-claim-refresh-lift entry signs candidate_above (bound 0.0553), consistent with its I1 evidence as you noted — but spine-m measures would_claim_uc at 0.548741, 0.001951 below the incumbent's 0.550692 and inside the whole-spine ±0.02 acceptance band: the redraw's demotions outweigh the OR-refresh lift. The register's own doctrine says an in-band difference needs no signature. I have recorded the measured actual in the entry's provenance and in Part F, and left the signature untouched for María to rule: retire as in-band, or re-sign below.

Verification: UK+machinery funnel, ruff, and tools/ci_test_groups.py --verify green locally (the two documented licensed-cache regeneration tests remain the only exclusions); CI on the push.

juaristi22 added a commit that referenced this pull request Sep 2, 2026
…d blocker)

The first licensed spine-m build failed at uc_reporter_redraw: 'Every
benefit unit must contain an adult claimant candidate.' A 16-19
qualifying young person heading their own benefit unit is its only
member, so the engine's UC child flag leaves no ~uc_child candidate —
the licensed frame carries 238 all-under-18 benefit units, and PR CI can
never reach this path (secrets-free, no licensed builds).

The fallback now takes the unit's eldest member as its de-facto head,
and the fail-closed check moves to the landing, where a POSITIVE draw on
a child claimant refuses — the award screen keeps child-only units out
of the drawn domain, so a positive landing there is exactly the
corruption the old total guard feared. Regression tests for both halves.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@juaristi22
juaristi22 force-pushed the uk-uc-reporter-redraw-832 branch from 7a49e2d to f2271ee Compare September 2, 2026 11:15
juaristi22 added a commit that referenced this pull request Sep 2, 2026
…d blocker)

The first licensed spine-m build failed at uc_reporter_redraw: 'Every
benefit unit must contain an adult claimant candidate.' A 16-19
qualifying young person heading their own benefit unit is its only
member, so the engine's UC child flag leaves no ~uc_child candidate —
the licensed frame carries 238 all-under-18 benefit units, and PR CI can
never reach this path (secrets-free, no licensed builds).

The fallback now takes the unit's eldest member as its de-facto head,
and the fail-closed check moves to the landing, where a POSITIVE draw on
a child claimant refuses — the award screen keeps child-only units out
of the drawn domain, so a positive landing there is exactly the
corruption the old total guard feared. Regression tests for both halves.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@juaristi22
juaristi22 force-pushed the uk-uc-reporter-redraw-832 branch from f2271ee to cf16f45 Compare September 2, 2026 11:58
@vahid-ahmadi

Copy link
Copy Markdown
Contributor

Third pass at cf16f452 (Claude Code, high effort; detached worktree of the rebased branch, targeted suites run in both microcosm-build and microcosm-graph). Checked each of the nine dispositions against the code rather than the description, plus the fixture re-pin.

Dispositions

1. Spec truthfulness — verified. sources.yaml:1938-1939 derive has_non_child_member and claimable_screen; :1957-1958 carry rewrite_rows: spi_channel and screen: claimable_screen. The schema branch adds rewrite_rows to both required and properties, source_stages.json matches key-for-key, and _assert_stage_parameters (uc_reporter_redraw.py:780-803) pins all of it. The code's screen at :209-213 and :230-231 is literally what the spec now says. Spec sha and the 14 coverage-manifest shas re-pinned.

2. Empty screened domain — verified. :267-275 refuses; the if target.any(): wrapper is gone and :291-301 is unconditional. The new test's fail_all_spi stub sets reduction equal to maximum on every SPI unit while base units still pass, so it reaches this guard rather than the training one, and asserts the raise.

3. Landing invariants — two of three tested to failure. _assert_landing_invariants (:614-659) is a real improvement: base_before is a copy=True snapshot taken before the landing over a mask that cannot be reordered, so the byte-identical check is independent of the write, and the one-positive-row check is recomputed from the column. test_landing_invariants_refuse_base_writes_and_double_landings makes both of those fail. The third branch, "landed amounts that disagree with the draws" (:649-652), has no failing test: the double-landed case is 125 + 125 against a draw of 250, so it sums correctly and trips the one-row check instead. One more case with a wrong total (or a dropped claimant row) closes it. Worth saying plainly that this branch is a consistency check, not an oracle — draws feeds both the write and the check — which is fine, but it is the reason it needs its own failing test.

4. Incidence signature — disposition right, one sentence of the evidence wrong. Keeping candidate_below / 0.0184 is correct on the spine-m actuals (5,730 of 113,649, share 0.050418, matching Part F). But the magnitude_evidence now says the above side "needs a 100% reporter draw on the 3,299 screened SPI units". That is the range ceiling. Crossing the incumbent needs 0.057359 × 113,649 ≈ 6,519 reporter persons, i.e. 6,519 − 4,428 = 2,091 of 3,299 screened units, about 63%. The measured draw landed 1,302 (39.5%) and the base rate is ~46%, so the conclusion stands with margin; the sentence should say ~63%, since a reader checking the arithmetic will trip on it.

5. child_only receipt label — verified (:679-687, applied before the couple/children split).

6. Guard comment — verified (:602-606).

7. End-to-end child-only test — verified. It drives a 17-year-old QYP-only SPI unit carrying a chain fill of 300 through the stage with an engine that pays it, and asserts exit at 0, screen_failed_spi_benunits == 2, child_only: {demoted: 1}, and no single_with_children key in the SPI receipt. This is the test the licensed-build failure was asking for.

8. Minors — verified for wiring; one caveat. _has_adult_member refuses a unit with no person rows, tested. is_SP_age is the fourth screen variable in all four places (constant, yaml predictors, source_stages.json, the parameter assert) and drives working_age = adult & ~sp_age (:438), with the SPA-flagged-elder test. Caveat only: the stub sets is_SP_age = age >= 66, so equivalence with the old age < 66 rests on the engine's 2024 SPA being 66 for everyone; the one live-engine test is the skip without the licensed extra. It is the convention etb_vat.py and etb_services.py already use, so this is a note, not a finding.

9. uc-reporter-claim-refresh-lift — recorded as stated. The entry keeps candidate_above / 0.0553 and its provenance now carries the spine-m actual (0.548741, 0.001951 below the incumbent) and names the open adjudication. Agreed this is María's call, and the entry is honest as it stands.

The fixture re-pin (cf16f452)

Legitimate and fully accounted for. The two counts are different denominators: the roster is 29 stages (28 on main) and UK_SPINE_EXCLUSIONS removes two, so the graph has 27 (test_uk_graph.py:166, was 26). uk_spine.json goes 37 → 39 nodes, adding uc_reporter_redraw.boundary and uc_reporter_redraw (kernel uk.stage.uc_reporter_redraw@1, single rewrite output, mass: conserve) between hmrc_spi_income_spine and uc_capital_coherence.boundary. Of the 37 pre-existing nodes exactly two changed, both consequences of insertion: the coherence boundary's population base now points at the redraw boundary, and uc_capital_coherence.inputs swapped person.is_severely_disabled_for_benefits for universal_credit_reported — that is the shape-anchor rule in uk_runtime/graph.py (_stage_slices, :634-658) picking one cell from the immediately preceding stage. Nothing else moved. The re-pin was needed only because #836 landed the fixture on main underneath this PR.

Runs

ruff check clean on the changed files. Build suites: 281 passed, 1 skipped (the live-engine test). microcosm-graph full suite: 195 passed, 2 skipped. CI fully green on the push (23 checks).


Nothing blocking remains. The merge-blocker from the first two rounds (an empty screen wiping the channel with a clean receipt) is closed with a test that reaches the guard. Two cheap tidy-ups before merge: a failing-path test for the sum-equals-draw branch (3), and the ~63% correction to the incidence evidence (4). Item 9 is the adjudication María already owns.

juaristi22 and others added 7 commits September 2, 2026 16:26
I2-I4 of the approved plan: new uc_reporter_redraw stage at index 19
(temp-frame engine screen on positive pre-take-up award with FRS-carried
capital; benefit-unit RegimeGatedQRF seed 44 trained on the screened base
domain; single-claimant landing), the derived-surface sweep (schema branch,
op-kind allowlist, sources lockstep, roster/sha/binder pins, coverage
manifest), and the two bounded signed differences with I1 receipt anchors
(experiments/832-uc-reporter-receipts.md).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Codex <noreply@openai.com>
np.isclose's default band admitted ~[-1.00001, -0.99999] as the sentinel,
silently reading near-sentinel corruption as a declared absence. Exact
equality in the stage-entry check, the gate-binding domain predicates, and
the sentinel-parity check; near-sentinel regression tests at both layers.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…d blocker)

The first licensed spine-m build failed at uc_reporter_redraw: 'Every
benefit unit must contain an adult claimant candidate.' A 16-19
qualifying young person heading their own benefit unit is its only
member, so the engine's UC child flag leaves no ~uc_child candidate —
the licensed frame carries 238 all-under-18 benefit units, and PR CI can
never reach this path (secrets-free, no licensed builds).

The fallback now takes the unit's eldest member as its de-facto head,
and the fail-closed check moves to the landing, where a POSITIVE draw on
a child claimant refuses — the award screen keeps child-only units out
of the drawn domain, so a positive landing there is exactly the
corruption the old total guard feared. Regression tests for both halves.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The relocated landing guard caught the second half of the child-only
defect on the retry build: the engine computes a positive pre-take-up
award for a benefit unit whose only member is a 16-19 qualifying young
person, though such a unit cannot claim — so the award screen alone
admits child-only units into the drawn domain and the model lands
positive draws on child claimants. The screen now also requires a
non-child member; the landing guard stays as the fail-closed fence.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Spec truthfulness: the declared screen is claimable_screen (positive
pre-take-up award AND a non-child member) and the rewrite domain is its own
field (rewrite_rows: spi_channel) beside the draw domain; schema, drift fence
and stage notes follow. An empty screened SPI domain now refuses instead of
zeroing the channel. The tautological retention check is replaced by landing
invariants the write did not derive (base column byte-identical, per-unit
sums equal the draws, one positive row per reporting unit). The transition
receipt files child-only units under child_only rather than
single_with_children. The landing guard is framed as defence in depth. An
end-to-end QYP-only SPI unit test covers the licensed-build failure path.
Working age comes from the engine's is_SP_age; _has_adult_member refuses
person-less units. The universal_credit_reported signature is re-derived on
the spine-m actuals (Part F); the would_claim_uc entry records its measured
in-band actual pending adjudication.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The uc_reporter_redraw stage is declared in the UK spine graph (open input
surface, one rewritten person cell) and wired into the kernel registry and
the fixture tool; the hermetic fixture's descriptor and pinned graph move
with it. Regenerated with tools/graph_uk_spine_fixture.py at this head.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ass (#832)

María's ruling on the spine-m actuals: would_claim_uc lands 0.001951 below
the incumbent, inside the whole-spine acceptance band, so the #828-era
candidate_above signature (bound 0.0553) is retired rather than re-signed;
the register test now asserts its absence and Part F records the ruling.
The incidence entry's evidence states the real crossing point (about 2,091
of the 3,299 screened SPI units, ~63%, against the measured 39.5%) instead
of the range ceiling. The sum-equals-draw landing invariant gains its own
failing-path test.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@juaristi22
juaristi22 force-pushed the uk-uc-reporter-redraw-832 branch from cf16f45 to 7f3a9cd Compare September 2, 2026 15:07
@juaristi22
juaristi22 merged commit 47c7422 into main Sep 2, 2026
23 checks passed
juaristi22 added a commit that referenced this pull request Sep 2, 2026
The licensed acceptance was re-run on the rebased tree: twins e9r-twin-a and
e9r-twin-b (28 stages) are payload-identical and pass the spine battery, the
control is rebuilt from main 47c7422, attribution again isolates exactly the
six E9 columns with E5 byte-equal, the e9 identity receipt passes, and the
six columns are byte-identical to the pre-rebase twins. The engine round-trip
figures move slightly with the reporter redraw upstream (mean monthly 66.81 /
50.66; at-cap 0.145 / 0.263; unique-row incidence z 1.84 / 2.00).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Sep 2, 2026
…h age_tail (#785, #845)

The graph's CREATE node declared person.age as float64 while the legacy
frs_spine root produces int64, so every graph stage saw a float age from
node 0 and the legacy plan an integer until age_tail rewrote it. The
interface forbids a rewrite from changing its base's dtype, so the honest
declaration is int64 end to end: the root cell is declared int64, age_tail's
rewrite is declared int64, and the transform preserves an integer input dtype
(its bands and within-band offsets are integral). frs_education's explicit
int64 cast stays as the stage's own contract. The exported age column changes
dtype (float64 to int64) and nothing else: the payload expectation declares
that as a dtype-only change through a new per-group surfaces allowlist in the
classifier, so a value difference on age stays unexpected. The H2 fixture is
re-pinned; the receipts checklist names the graph-executed post-#835 build as
the before artifact, since the legacy-executed spine-m carries executor
normalizations that are not this change's.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Sep 2, 2026
…h age_tail (#785, #845)

The graph's CREATE node declared person.age as float64 while the legacy
frs_spine root produces int64, so every graph stage saw a float age from
node 0 and the legacy plan an integer until age_tail rewrote it. The
interface forbids a rewrite from changing its base's dtype, so the honest
declaration is int64 end to end: the root cell is declared int64, age_tail's
rewrite is declared int64, and the transform preserves an integer input dtype
(its bands and within-band offsets are integral). frs_education's explicit
int64 cast stays as the stage's own contract. The exported age column changes
dtype (float64 to int64) and nothing else: the payload expectation declares
that as a dtype-only change through a new per-group surfaces allowlist in the
classifier, so a value difference on age stays unexpected. The H2 fixture is
re-pinned; the receipts checklist names the graph-executed post-#835 build as
the before artifact, since the legacy-executed spine-m carries executor
normalizations that are not this change's.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants