Explain SPM threshold calculation and conditional rolling forecasts

.github/workflows/check.yaml

@@ -10,8 +10,16 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v4
+        with:
+          # The evaluation reads the original nowcast from the
+          # v1.0-original-nowcast tag; a shallow, tagless checkout
+          # cannot resolve it.
+          fetch-depth: 0
+          fetch-tags: true
       - uses: actions/setup-python@v5
         with:
           python-version: "3.12"
       - name: Paper drift check
         run: python scripts/check_paper.py
+      - name: Guard mutation tests
+        run: python -m unittest discover -s tests -v

.gitignore

@@ -5,3 +5,7 @@ _output/
 /.quarto/
 **/*.quarto_ipynb
 paper/index_files/
+
+# Local Python execution artifacts
+__pycache__/
+*.py[cod]

README.md

@@ -1,33 +1,64 @@
-# Nowcasting Supplemental Poverty Measure thresholds
+# Calculating and projecting Supplemental Poverty Measure thresholds
 
-Working paper. Quantifies the July 17, 2026 BLS threshold correction,
-documents a replication of the BLS threshold methodology from public CE
-microdata, backtests threshold-projection rules, and commits to a
-pre-registered nowcast of the unpublished 2025 thresholds — graded
-against BLS's actual publication (~September 2026) in a planned
-revision.
+Working paper on national SPM threshold estimation from public Consumer
+Expenditure Survey microdata, family and geographic adjustments, and
+projection from consumption and price growth. The manuscript presents
+current-method replication and sample sensitivities, retrospective
+comparisons, and a separately preserved pre-committed 2025 validation.
+It also constructs conditional forecasts through 2035 by advancing the
+CE and ACS windows with explicit price, real-spending, and rent assumptions.
 
-## Build
+The current six-year results use revised inputs and overlapping CE
+windows. They are separate from the single prospective validation year.
+Method limitations and the forecast amendment history are reported in
+the manuscript. The repository preserves source artifacts, method
+versions, and forecast commitment evidence.
+
+## Build and verify
 
 ```bash
-quarto render   # runs scripts/check_paper.py pre-render, then HTML + PDF to _output/
+python3 -B scripts/check_paper.py
+python3 -B -m unittest discover -s tests -v
+quarto render --to html
+quarto render --to pdf
 ```
 
-The pre-render check regenerates every table from `data/` via
-`scripts/build_tables.py`, verifies `data/SHA256SUMS`, and re-derives
-the load-bearing prose figures; the render fails if any of them
-drifts from the artifacts.
+The guard and table generators use the Python standard library. Quarto
+1.9.36 and TeX Live 2026 render the HTML and PDF to `_output/paper/`.
+Clone with full tag history and check out the exact revision to reproduce
+it. The guard first checks artifact hashes and frozen Git identities,
+then generates tables and evaluation JSON in temporary storage and
+compares every output byte with the checkout, including archived tables
+that the manuscript no longer includes. It never repairs or
+restores source artifacts. The regression tests verify that failed checks
+leave source bytes unchanged.
+
+## Preserved forecast evidence
+
+The tags `v1.0-original-nowcast` and `v1.1-amended-nowcast`, forecast
+bytes, and all original OpenTimestamps manifest/proof pairs are preserved.
+Two hashes in `data/COMMITMENT-MANIFEST.txt` were incorrect; the dated
+`data/COMMITMENT-AMENDMENT-2026-09-08.txt` records the exact tag hashes and
+the original manifest digest. The original proof does not cover that
+amendment. This revision verifies retained proof bytes and Git identities;
+it does not independently reverify blockchain attestations.
+
+`data/SHA256SUMS` continues to cover only the frozen artifacts. New inputs
+have their own `data/current/SHA256SUMS`; they do not replace the frozen
+forecast or become a new prospective commitment. `data/PROVENANCE.md`
+describes both collections and the original implementation pin.
 
-Every numeric table is generated from the artifacts in `data/`
-(SHA-256 sums in `data/SHA256SUMS`), which are produced by the scripts
-in [PolicyEngine/spm-calculator](https://github.com/PolicyEngine/spm-calculator)
-(v0.4.0, PR #32). The prose cannot drift from the data without the
-build failing.
+## Companion implementation
 
-## Companion code
+The companion spm-calculator provides a portable artifact, standalone
+unit calculations, CE research replication, rolling projections, and
+PolicyEngine, Microcosm, and native Axiom adapters. The rolling projection
+pins the 1.0.0 release candidate at commit
+`78bae15f76152c6076dd909a09f6b63dc2ec8c34`, with content digest
+`3d86d5c4c0423480e6b69b75d222ffa4a7a2639e4094df5ba2504af01be17173`.
+The paper consumes the same projection bytes as the calculator.
 
-- spm-calculator 0.4.0: corrected/published/legacy threshold series
-  with provenance, CE replication, benchmark, backtest, nowcast, and
-  the weekly BLS drift-watch workflow.
-- policyengine-us #9081: adopts the corrected series in the US
-  microsimulation model.
+The 2019–2025 retrospective experiment retains its separate development
+version 0.5.0 pin at `0d7fa0d77b0a88064ab7b9fe70557309b5f7901f`.
+These source identities document reproducibility; they do not certify
+publication, deployment, or a population-model migration.

bibliography/references.bib

@@ -275,3 +275,61 @@ @techreport{schild2023
   month = {August},
   url = {https://www.bls.gov/pir/journal/2021_2022_spm_analysis.pdf}
 }
+
+@misc{bls2025thresholds,
+  author = {{U.S. Bureau of Labor Statistics}},
+  title = {2025 Research Supplemental Poverty Measure Thresholds},
+  year = {2026},
+  month = {August},
+  note = {Division of Price and Index Number Research. Retrieved September 4, 2026},
+  url = {https://www.bls.gov/pir/spm/spm_thresholds_2025.htm}
+}
+
+@techreport{creamer2026revising,
+  author = {Creamer, John},
+  title = {Revising Supplemental Poverty Measure Estimates from 2019 to 2024},
+  institution = {U.S. Census Bureau},
+  type = {SEHSD Working Paper},
+  number = {2026-17},
+  year = {2026},
+  month = {August},
+  url = {https://www.census.gov/library/working-papers/2026/demo/sehsd-wp2026-17.html}
+}
+
+@misc{blsspmshares,
+  author = {{U.S. Bureau of Labor Statistics}},
+  title = {Supplemental Poverty Measure Threshold Expenditure Shares},
+  year = {2026},
+  note = {Workbook retrieved September 9, 2026; sheet 2005-2024 includes 2025 column W},
+  url = {https://www.bls.gov/pir/spm/spm_shares.xlsx}
+}
+
+@misc{cbodata2026,
+  author = {{Congressional Budget Office}},
+  title = {Budget and Economic Data: February 2026 Economic Projections},
+  year = {2026},
+  note = {Versioned data repository, revision 284a95665f9f2f74ed1f482feb629b43fce323da},
+  url = {https://github.com/US-CBO/cbo-data/tree/284a95665f9f2f74ed1f482feb629b43fce323da}
+}
+
+@misc{spmtechnical,
+  title = {Supplemental Poverty Measure: Technical Documentation},
+  author = {{U.S. Census Bureau}},
+  url = {https://www2.census.gov/programs-surveys/supplemental-poverty-measure/datasets/spm/spm_techdoc.pdf},
+  urldate = {2026-09-09}
+}
+
+@misc{acs2024accuracy,
+  title = {Public Use Microdata Sample: Accuracy of the Data (2020--2024)},
+  author = {{U.S. Census Bureau}},
+  url = {https://www2.census.gov/programs-surveys/acs/tech_docs/pums/accuracy/2020_2024AccuracyPUMS.pdf},
+  urldate = {2026-09-09}
+}
+
+@misc{acs2024pumd,
+  author = {{U.S. Census Bureau}},
+  title = {2024 American Community Survey Public Use Microdata Sample Documentation},
+  year = {2025},
+  note = {Source and geography documentation; retrieved September 9, 2026},
+  url = {https://www.census.gov/programs-surveys/acs/microdata/documentation.2024.html}
+}

data/COMMITMENT-AMENDMENT-2026-09-08.txt

@@ -0,0 +1,21 @@
+{
+  "format": "spm/commitment-amendment/v1",
+  "date": "2026-09-08",
+  "original_manifest": "data/COMMITMENT-MANIFEST.txt",
+  "original_manifest_sha256": "18d82ad41fad56b42dfce537523e50b5b89956b8161dc60e60d90f8753b39b37",
+  "covered_by_original_timestamp": false,
+  "reason": "Correct the two SHA-256 values transcribed incorrectly in the secondary manifest written September 4. The tagged forecast bytes, amounts, tag objects, original manifest and all existing timestamp proofs remain unchanged. The original manifest's proof attests its original bytes, including those errors; it does not attest this amendment.",
+  "verification": "SHA-256 over exact bytes returned by git show TAG:data/nowcast_2025.json; tag object and peeled commit checked separately. This amendment makes no new claim of blockchain-proof verification.",
+  "tags": {
+    "v1.0-original-nowcast": {
+      "tag_object": "12b7042a960412477af53c62ebd961ded1a2625e",
+      "commit": "a8b7c51869e64e36517d4eca8d5dbd6e2e0cde5b",
+      "nowcast_sha256": "19a1f44d10d9151c0b4d3cdb953e9d338bc9dd7a6fe06b88a91665649a5931c0"
+    },
+    "v1.1-amended-nowcast": {
+      "tag_object": "3759670e314b762fa24348a3c9a16526cdc74b47",
+      "commit": "d72e1dc083ad1abc95b1c88fd1b782944454074b",
+      "nowcast_sha256": "259a5b2e0d15646722fe309f741546e364827fab053c73c99d273061fa0b1ba8"
+    }
+  }
+}

data/COMMITMENT-MANIFEST.txt

@@ -0,0 +1,19 @@
+# Commitment manifest — 2025 SPM threshold nowcast, both vintages
+# Written 2026-09-04 after BLS's 2025 publication; stamped separately from the pre-publication proof.
+# Pre-publication anchors: SHA256SUMS.precommit.ots (Bitcoin block 961505, 2026-08-07 23:18 UTC) covers the amended manifest;
+# the Internet Archive capture https://web.archive.org/web/20260807211521/https://spm-threshold-paper.vercel.app/ shows both sets;
+# GitHub release timestamps for both tags date 2026-08-07.
+
+tag v1.0-original-nowcast
+  tag_object 12b7042a960412477af53c62ebd961ded1a2625e
+  commit a8b7c51869e64e36517d4eca8d5dbd6e2e0cde5b
+  nowcast_2025.json sha256 85049a018bbf319d900320fceed6b740f83ab52efbf11a795250066f69c8bfd8
+  values 41099.57 / 34250.70 / 40791.72
+
+tag v1.1-amended-nowcast
+  tag_object 3759670e314b762fa24348a3c9a16526cdc74b47
+  commit d72e1dc083ad1abc95b1c88fd1b782944454074b
+  nowcast_2025.json sha256 ec0cd0f8b629fe79d9db24e0f8fcd0bac4f172817f940376d8f1c80fbb823028
+  values 41036.34 / 34135.99 / 40755.98
+
+evaluation: PR #1 (branch v2-evaluation) of PolicyEngine/spm-threshold-paper; this manifest is stamped as COMMITMENT-MANIFEST.txt.ots