Motivation (measured)
The imputation paper's harness (github.com/PolicyEngine/imputation-paper) initially certified a candidate whose imputed net-worth 99th percentile was 1.9–2.25× the holdout's: weighted energy distance, PRDC coverage, and a classifier two-sample test all tied it with a correct candidate, because capped pairwise geometry under-resolves heavy right tails. The fix was a fourth, uncapped tail block per imputed variable. Promotion gates built on target attainment plus geometry metrics will certify tail-broken bundles.
Proposal
Add to the incumbent-comparison promotion metrics, per imputed variable, computed on full weighted samples (no subsample cap) against the pinned reference:
- q90 and q99 ratios (candidate over reference; 1 is perfect; band calibrated by a sampling floor — the paper's floor wobbles to ~1.10 on a 920-record holdout),
- weighted Wasserstein-1 / reference sd,
- reweight fragility vs the reference's own fragility (closed form: worst-case single-record share of the aggregate under bounded multiplicative reweighting, κ=5; the target is the reference's exposure — far below it means missing tails, far above means landmines; measured range: OLS-style imputations understate true exposure by 7–11×).
Reference implementations: experiments/metrics.py (reweight_fragility, _weighted_quantile) and experiments/views.py (tail block) in the paper repo; the promotion rule in this repo's README ("beats the pinned incumbent on every promotion metric") then covers the tail by construction.
🤖 Generated with Claude Code
Motivation (measured)
The imputation paper's harness (github.com/PolicyEngine/imputation-paper) initially certified a candidate whose imputed net-worth 99th percentile was 1.9–2.25× the holdout's: weighted energy distance, PRDC coverage, and a classifier two-sample test all tied it with a correct candidate, because capped pairwise geometry under-resolves heavy right tails. The fix was a fourth, uncapped tail block per imputed variable. Promotion gates built on target attainment plus geometry metrics will certify tail-broken bundles.
Proposal
Add to the incumbent-comparison promotion metrics, per imputed variable, computed on full weighted samples (no subsample cap) against the pinned reference:
Reference implementations:
experiments/metrics.py(reweight_fragility,_weighted_quantile) andexperiments/views.py(tail block) in the paper repo; the promotion rule in this repo's README ("beats the pinned incumbent on every promotion metric") then covers the tail by construction.🤖 Generated with Claude Code