Raw archive integrity: 12 manifest-declared objects were never in ledger-raw (restored); IRS re-published 2022 IRA tables; OBR blocks unattended fetch

[MaxGhenis]

Found while backfilling ledger-raw → chronicle-raw for #143 (mechanism 3, dual-run). Every object was fetched by its manifest-recorded key and verified against the sha256 embedded in the key before being written to the new bucket. Result: 144 of the 156 manifest-declared raw objects existed in ledger-raw; 12 did not exist at all, even though their manifests record a storage.r2 URI in ledger-raw:

Package	File	Recovered from
census_pep/census-pep-2024-national-age-sex	nc-est2024-agesex-res.csv	publisher, byte-exact
census_pep/census-pep-2024-state-age-sex	sc-est2024-alldata6.csv	publisher, byte-exact
census_pep/census-pep-county-population-2024	co-est2024-alldata.csv	publisher, byte-exact
cms_nhe/cms-nhe-historical-service-source	national-health-expenditures-…-cy-1960-2024.zip	publisher, byte-exact
dwp/dwp-uc-deductions-march-2025-february-2026	universal-credit-deductions-march-2025-to-february-2026.ods	publisher, byte-exact
eurostat/eurostat-ilc-di01	ilc_di01.json	publisher, byte-exact
eurostat/eurostat-ilc-li02	ilc_li02.json	publisher, byte-exact
irs_soi/soi-county-2022	22incyallnoagi.csv	publisher, byte-exact
irs_soi/soi-ira-roth-contributions-2022	22in06ira.xlsx	tracked repo copy (see 2)
irs_soi/soi-ira-traditional-contributions-2022	22in05ira.xlsx	tracked repo copy (see 2)
obr/obr-efo-aggregates-march-2026	efo_aggregates.xlsx	tracked repo copy (see 3)
obr/obr-efo-economy-march-2026	efo_economy.xlsx	tracked repo copy (see 3)

All 12 are now present in both buckets at the recorded key with the recorded hash (additive repair: nothing in ledger-raw was rewritten or moved; the recorded URIs are simply true now). A readback from chronicle-raw was hash-verified.

Three things to fix

1. Manifests can record R2 custody that never happened. PR CI is secrets-free, so nothing checks that a storage.r2 URI resolves. Proposal: inventory-artifacts gains a --verify-r2 mode (HEAD/get by key) run by the trusted-base workflow or a scheduled job, and publish-raw writes the storage.r2 block only after a successful upload (never from a dry run).
2. IRS re-published the 2022 IRA contribution tables. The live 22in05ira.xlsx / 22in06ira.xlsx are valid workbooks with core modified 2026-06-04 and different bytes (11,365 / 11,697 bytes vs recorded 13,932 / 13,617). The recorded bytes were restored from the tracked copies, so the fact lineage is intact, but the publisher's current release differs: register a new release revision (per the fact-identity ADR, same vintage + new bytes ⇒ new release_revision) and re-derive the IRA facts to see whether any value changed.
3. OBR returns 403 to non-browser fetchers, so fetch-artifact cannot refresh OBR files unattended. Either a browser-like User-Agent in the fetcher (if OBR's terms allow) or a documented manual-fetch step with hash verification.

Related: #143 (bucket cutover), #221 (registration tooling).

[MaxGhenis]

Addendum: a YAML walk over the tracked manifests on main finds 157 distinct raw keys (137 manifest files), one more than the regex sweep the first pass used. The extra one, irs_soi/soi-w2-statistics-2020 → 20in04w2all.xlsx, was also absent from ledger-raw; re-fetched from the IRS byte-exact and written to both buckets. So the phantom count is 13 of 157, and chronicle-raw now holds all 157.

[MaxGhenis]

Second pass, after recounting on main with a YAML walk (186 distinct keys across 154 manifest files; the first sweep ran against a stale clone index): 18 more objects copied from ledger-raw, and 12 more phantoms (manifests recording an R2 URI that never existed), all in packages merged since mid-August: the 9 Statbel fiscal-income tables (BE), DfE funded early-education, HMRC CGT size-of-gain, HMRC tax-free childcare, ONS households-by-type. 9 were re-fetched byte-exact from the publishers; 3 came from the tracked repo copies after the publisher fetch produced different bytes or was blocked (DfE headline figures, HMRC CGT table, ONS households). Running total: 25 phantom objects of 186, all now present in both buckets at the recorded keys; chronicle-raw holds all 186. The pattern is clear: publish-raw is not being run for new packages, so item 1 above (verify storage.r2 resolves, and only write the block after a real upload) is the fix that matters.