diff --git a/docs/adr/0003-employer-firm-extension.md b/docs/adr/0003-employer-firm-extension.md index b80236e7..5eb3280b 100644 --- a/docs/adr/0003-employer-firm-extension.md +++ b/docs/adr/0003-employer-firm-extension.md @@ -1,15 +1,58 @@ -# ADR 0003: Employer-firm extension — C1 spell schema and C2 canonical firm-size banding +# ADR 0003: Employer-firm extension — IC1 spell schema and IC2 canonical firm-size banding -**Status:** Accepted — C1 and C2 frozen 2026-07-16. From this +**Status:** Accepted — IC1 and IC2 frozen 2026-07-16. From this point the contracts change only by joint PR between workstreams A and B ([populace-dynamics#192](https://github.com/PolicyEngine/populace-dynamics/issues/192)). +**Amendment 1 (naming, no semantic change).** The interface +contracts were originally named `C1`/`C2`/`C3`. They are renamed +`IC1`/`IC2`/`IC3` — *interface contract* — with no change to any +column, band, code mapping, or gate definition. This is a +joint-PR change because it edits frozen contract text, not because +anything in the contracts moved; the numbering is preserved 1:1, so +every prior reference maps by prefixing `I`. + +**Why, and why this side moves.** In this repository a bare "C1" +already meant four different things: + +| sense | example | can it be renamed? | +|---|---|---| +| gate_w1 **fingerprints** `c1`/`c2` | `gates.yaml` `fingerprints.c1` (PPI↔NRA) | **No** — inside `gate_w1`, which is `locked: true`. Renaming needs a public amendment plus a fresh referee round | +| SSA Trustees **table** II.C1 | `data/external/ssa_tr_2014_ii_c1.*` | No — an external publisher's table label | +| RNG **substream** C3 | household-composition `nonfamily_bridge` | Unrelated component; renaming is churn for no gain | +| **interface contracts** C1/C2/C3 | this ADR | **Yes** — the only set that is repo-internal, pre-lock, and ours | + +The collision was flagged on #192 before the C3 referee round with +the note that it would confuse referees. It is fixed now rather +than later for one reason: `IC3` (the employer gate block) is +about to be written into `gates.yaml`, which already contains +`fingerprints.c1` and `fingerprints.c2`. Once a block named `C3` +locks alongside them, renaming either set costs an amendment and a +fresh referee round — the exact cost this table shows the +fingerprint side already carries. + +Prior discussion (issue #192, the ADR history, merged PR bodies) +uses the old names and is not rewritten; this note is the mapping. + +**Boundary: history keeps the old names, live documents are +renamed.** The plan (`docs/plans/employer-firm-plan.html`), cited by +the Context section below as the operative split, is a live document +and is renamed with this amendment, so a referee following the ADR's +own link does not meet unmapped names. The unrenamed senses in the +table above (the locked `gates.yaml` fingerprints, the SSA table +labels, the RNG substream) remain as they are, by the reasons given. +Production-source docstrings sealed by the published first-estimates +replay ceremony also retain their historical `C1`/`C2` wording. They +are not operative contract text and are interpreted through this +one-to-one mapping; cosmetic edits would invalidate the sealed replay +identity. New source text uses the `IC` names. + **Sign-off:** @vahid-ahmadi (Workstream B, author) · @daphnehanse11 (Workstream A) — the joint sign-off is recorded by the merge of the freeze PR: authorship by one workstream owner plus approval by the other. First scheduled amendment (pre-registered below): -the C1 ``hours_band``/monthly-hours column once phase-1 establishes +the IC1 ``hours_band``/monthly-hours column once phase-1 establishes SIPP's supportable hours granularity. ## Context @@ -17,15 +60,15 @@ SIPP's supportable hours granularity. The employer-firm plan (`docs/plans/employer-firm-plan.html`) splits the extension into workstream A (person side: SIPP spells, CPS hosts, imputation) and workstream B (firm side: external targets, banding, -calibration, register), meeting at three interface contracts. C1 (the -spell schema) and C2 (canonical firm-size banding and its semantics) +calibration, register), meeting at three interface contracts. IC1 (the +spell schema) and IC2 (canonical firm-size banding and its semantics) freeze in week 1. This ADR records both, plus the target/gate partition rule, folding in the four contract-affecting findings from the week-1 review on issue #192. ## Decision -### C2 — canonical firm-size banding +### IC2 — canonical firm-size banding 1. **Semantics (review finding F5).** The canonical firm-size variable means **administrative enterprise size**: total @@ -33,7 +76,7 @@ the week-1 review on issue #192. counts it. Survey labels are noisy measures of that quantity — CPS ASEC firm size (worker-reported, all locations, previous calendar year's longest job — under either the raw Census `NOEMP` - or the IPUMS `FIRMSIZE` coding; see C2.5) is the primary training + or the IPUMS `FIRMSIZE` coding; see IC2.5) is the primary training label; SIPP 2014+ `EJB1_EMPSIZE` (establishment size) is a proxy chain. SUSB is therefore the correct E1 reference. 2. **Bands are headcount bands.** Five canonical bands with edges at @@ -43,7 +86,7 @@ the week-1 review on issue #192. both QWI (20-49 / 50-249) and the detailed SUSB classes (40-49 / 50-74) support it. FTE-denominated thresholds (the ACA cut is 50 full-time equivalents at 30 hours/week, not headcount) are - resolved by a person-side hours join — out of C2 scope. + resolved by a person-side hours join — out of IC2 scope. 3. **Mappings are total but explicitly ambiguous where the source is coarse.** Every raw code from every source maps to exactly one `BandSpan` (a contiguous run of canonical bands with an `exact` @@ -65,7 +108,7 @@ the week-1 review on issue #192. replication (weighted code shares by year vs. SUSB) is committed as `runs/noemp_band_evidence_v1.json` with its build script and pinning tests (#211) — the reported-anchor convention, since it - is derived evidence rather than a source extract — for the C3 + is derived evidence rather than a source extract — for the IC3 record. 5. **The person-side coding is explicit, not inferred (seam with #194).** The raw Census ASEC person file carries `NOEMP` @@ -85,7 +128,7 @@ the week-1 review on issue #192. emit `CanonicalBand` directly), never feed `NOEMP` integers to the `ipums_firmsize` route. -### C1 — job-spell schema +### IC1 — job-spell schema One tidy table, written by workstream A, read by workstream B: @@ -96,7 +139,7 @@ One tidy table, written by workstream A, read by workstream B: | `start_period` | period | first period of the spell | | `end_period` | period | last period; open spells use a sentinel | | `industry` | str | NAICS major (sector) group | -| `firm_size_band` | enum | canonical band per C2 (`CanonicalBand`) | +| `firm_size_band` | enum | canonical band per IC2 (`CanonicalBand`) | | `class_of_worker` | enum | private / federal / state-local government / self-employed / unpaid family | | `earnings_share` | float | share of the person's period earnings from this job | | `primary_job` | bool | phase 0 is primary-job-only | @@ -111,7 +154,7 @@ One tidy table, written by workstream A, read by workstream B: from the SUSB/QWI calibration universe; self-employed spells have no defined `firm_size_band`. - **Geography joins from the person table.** QWI/J2J targets are - state-level; C1 deliberately carries no geography column. The + state-level; IC1 deliberately carries no geography column. The state of a spell is the host person's state at `start_period`, joined on `person_id` — the join key lives on the person table, not the spell table. @@ -122,9 +165,9 @@ One tidy table, written by workstream A, read by workstream B: compliance, issue #192 — the 80-hours-per-month test of 7 CFR 273.24 and the 3-in-36 countable-month clock need month-resolved hours, not spell start/end plus annual earnings). - C1 as frozen carries no hours column, so it **cannot yet serve + IC1 as frozen carries no hours column, so it **cannot yet serve monthly-hours consumers**; a `hours_band` (or monthly hours) - column is the first scheduled C1 amendment, to be added by joint + column is the first scheduled IC1 amendment, to be added by joint PR once workstream A's phase-1 spell imputation establishes what hours granularity SIPP can support. Consumers must not proxy monthly compliance from annual quantities in the meantime. @@ -146,10 +189,10 @@ phase-0 QRF therefore comes from a named bridge, not an implicit one: bridge, aged forward. 2. **Proxy chain:** SIPP 2014+ establishment size x tenure, mapped through the establishment-to-enterprise noise model implied by - the C2 semantics. + the IC2 semantics. 3. **Pre-registered caveat:** the ASEC reference-period mismatch (`FIRMSIZE` = last calendar year's longest job; tenure supplement - = current job) is carried into the C3 gate notes as a known + = current job) is carried into the IC3 gate notes as a known label-misalignment term. ### Target/gate partition rule @@ -161,7 +204,7 @@ firm-size x sector flow margins committed under `data/external/`; gates E1/E2/E7/E11 score on held-out dimensions of the same sources (the sex/age demographic axes of QWI, the firm-age axis, and the state axis) that calibration never touches. The exact cell lists lock -with C3 after the floor runs. +with IC3 after the floor runs. Three unit rules recorded now (issue #192 review, point 4; branch review finding 3): @@ -171,7 +214,7 @@ review finding 3): so calibrating person-spells to QWI cells carries a wedge on the order of the multiple-jobholding rate (~5%, time-varying). A job-count -> person-count adjustment is an explicit pre-registered - C3 item, not a footnote. + IC3 item, not a footnote. - **QWI publishes mean earnings (`EarnS`), never medians**; E7 is stated on means. - **J2J's employer universe is broader than SUSB/QWI's.** The @@ -182,7 +225,7 @@ review finding 3): sectors (notably 61 Educational Services and 62 Health Care). Any E11 cell definition must either restate J2J on a private-comparable basis or carry this scope difference as a pre-registered caveat; - the choice locks with C3. + the choice locks with IC3. ## Consequences @@ -193,8 +236,8 @@ review finding 3): workstreams push directly to each other's branches when useful (reader fixes, rebases, contract-text corrections — this has run in both directions and worked). The norm the freeze makes - explicit: a change that touches contract semantics (C1 columns, - C2 bands/codings, gate definitions) requires the *other* + explicit: a change that touches contract semantics (IC1 columns, + IC2 bands/codings, gate definitions) requires the *other* workstream owner's approval on the PR even when the commit was pushed directly, so pre-registration always records who decided, not just who typed. @@ -204,6 +247,6 @@ review finding 3): references, analogous to the NCHS/Census/ONS files — never scored model output. Raw microdata is never committed. - No change to `gates.yaml`. Employer gates E1-E12 lock as a new - block (C3) after noise-floor runs and a referee round, via the - standard amendment process; no one-shot candidate runs before C3 + block (IC3) after noise-floor runs and a referee round, via the + standard amendment process; no one-shot candidate runs before IC3 locks. diff --git a/docs/adr/0004-linkage-qc.md b/docs/adr/0004-linkage-qc.md new file mode 100644 index 00000000..1e776fd6 --- /dev/null +++ b/docs/adr/0004-linkage-qc.md @@ -0,0 +1,479 @@ +# ADR 0004: Employer-firm linkage QC requirements for IC3 + +**Status:** Proposed — input to the IC3 referee round; this document +locks no IC3 threshold. IC1 and IC2 are treated as frozen and immutable +for this work under the workstream directive and the freeze record in +[populace-dynamics#215](https://github.com/PolicyEngine/populace-dynamics/pull/215). +This ADR does not amend their schema, semantics, or readers. + +## Context + +The employer-firm plan on +[issue #192](https://github.com/PolicyEngine/populace-dynamics/issues/192) +requires noise floors before thresholds, a referee round before the IC3 +gate block locks, and no one-shot candidate run before that lock. The +same ordering must govern the worker-to-employer or worker-to-firm-type +assignment itself. A downstream E-cell cannot certify a model if the +links used to construct the cell have unknown quality. + +Here, **link** includes any accepted worker-to-employer, employer- +attachment, or worker-to-firm-type assignment consumed by an E-cell. +That includes an assigned IC2 `CanonicalBand` or industry type; it does +not turn a statistical imputation into an identified firm ID. A **link +unit** is the unit on which a decision is accepted or withheld. Paired +and run-level moments may require a stricter derived unit, as specified +below. + +The frozen contract seam remains the IC1 spell fields +`person_id`, `spell_id`, `start_period`, `end_period`, `industry`, +`firm_size_band`, `class_of_worker`, `earnings_share`, and +`primary_job`. The implemented IC2 seam is the banding functions in +`src/populace_dynamics/firms/banding.py`. Linkage-QC decisions, +adjudication labels, match scores, and linkage weights therefore live +in versioned sidecars keyed to IC1 rows; they are not new IC1 columns. +`spell_type` is also not an IC1 field. Where needed below, transition +type is derived from adjacent spell rows plus the versioned person-month +observation/nonemployment frame; IC1 spells alone do not identify exits, +entries, or censoring. + +### Evidence behind the precision-first rule + +LIFE-M scales carefully reviewed hand links with supervised learning. +Its published workflow used independent double review, three new +independent reviewers for disagreements, a held-out test half, and +ten-fold cross-validation within the training half. It selected +thresholds to maximize recall subject to a project-specific +**"97% precision rate"** +and evaluated that training-selected cutoff out of sample. This ADR +adopts that ordering and separation, not LIFE-M's numeric 97% choice. +See Bailey et al. (2023), sections IV.C-IV.D and Figure 5, and the +[LIFE-M linking description](https://life-m.org/linking/). + +The final abstract of Bailey, Cole, Henderson, and Massey (2020) +reports that trained reviewers rejected **"15 to 37 percent"** of +links from widely used automated methods and that the combined +problems studied attenuated an intergenerational-income-elasticity +estimate by **"up to 29 percent."** Sections III.B, IV, VI.C, and VII +show why these are not merely lost-sample problems: false links can be +systematic, estimates usually moved toward zero in their case study, +and removing false links brought estimates across algorithms together. +The authors consequently recommend putting more weight on precision +than on increasing the match count. Those numeric findings describe +their historical-data exercises, not an employer-firm floor to import. + +Bailey, Cole, and Massey (online 2019; print 2020), section III.B and +Table 4, separately show how application-specific inverse-propensity +weights can improve balance between a linked sample and its reference +population on included observables. They also state the load-bearing +limits: common support and the paper's unconfoundedness/properly +specified propensity-score assumption are required, and balance on +omitted or unobserved characteristics is not guaranteed. Weighting +therefore complements the precision floor; it does not repair false +positive links. + +## Decision + +### 1. Precision first, before every IC3 threshold + +1. **Every link-producing component used by a gated E-cell must have + an independent audit artifact.** For categorical assignments, the + artifact reports the audit-design- and survey-weighted matrix of + adjudicated true class by assigned class, including no-counterpart, + abstain/unlinked, and indeterminate outcomes. It also reports + accepted-assignment precision, + end-to-end and stage-specific recall, false-negative and abstention + rates, and all numerators and denominators, overall and for every + pre-registered gate-relevant stratum. If it reports a pairwise + false-positive rate, it must define the candidate-pair universe; + `1 - precision` is the false-discovery rate, not that pairwise rate. + Point estimates and one-sided confidence bounds are both required. +2. **The precision floor is pre-registered before E-cell thresholds.** + IC3 must name the assignment, eligible universe, link unit, floor + `P_floor`, confidence level, required strata, pooling rule, and + failure disposition before it names any threshold for an E-cell + that consumes that assignment. Recall is always published. Whether + recall also gates is an explicit IC3 referee decision, not an + after-the-fact response to results. +3. **Passing uses a confidence bound, not the observed proportion.** + The lower one-sided confidence bound for precision must be at least + `P_floor` overall and at every stratum or derived-unit level that + IC3 designates operative. Oversampled strata are combined only with + their recorded sample inclusion weights. +4. **No threshold shopping follows a linkage failure.** A failed or + unevaluable applicable floor makes the linked E-cell invalid. It is + not a model miss that can be cured by widening the E-cell tolerance, + dropping an inconvenient stratum, or selecting a different weighted + result. A registered candidate with any required invalid cell cannot + pass the employer block. +5. **The audit precedes the one-shot run.** Linkage-QC results and the + immutable adjudication manifest must be on the IC3 record before a + candidate may consume the matcher. A materially changed matcher, + cutoff, candidate-generation rule, source vintage, or target + vocabulary requires a new versioned audit or the pre-registered + transport test; it never inherits a pass silently. + +### 2. Hand-adjudication sample + +The IC3 block must register the following sample design before labels +are opened. + +1. **Frame and two audit arms.** Define the complete eligible universe, + candidate-generation rules, accepted-link rule, true no-counterpart + cases, and abstentions. Draw (a) an accepted-assignment arm, which + identifies false positives and precision, and (b) an eligible-universe + truth-search arm containing accepted, rejected, and no-link cases. + The second arm uses an independent exhaustive search or external + reference that can find a true counterpart omitted by candidate + generation; reviewing only the matcher's candidate set is not an + end-to-end recall study. Report candidate-generation recall separately + from selector/cutoff recall. Because a unit may enter both arms, IC3 + registers the dual-frame overlap and combined-inclusion estimator so + it is neither omitted nor counted twice. +2. **Stratification follows the moments the matcher feeds.** For the + accepted arm, stratify at minimum by the five assigned IC2 + `CanonicalBand` values (plus withheld or unresolved assignments), + assigned NAICS major industry, and the derived spell/transition class + relevant to the battery: stay, job-to-job, exit, or entry, crossed + with `primary_job` where that changes the estimand. Rare E11 origin- + destination size pairs and E12 firm types are deliberately + oversampled. Because rejected and no-link units lack an assigned or + known-true class before review, the universe arm uses a separate + pre-link stratification frame observed for every eligible unit, then + reports recall by adjudicated true band, industry, and transition + domain. IC3 publishes any pooling of sparse strata before adjudication + and retains every arm-specific inclusion probability. +3. **Target size is powered at the floor.** IC3 registers `P_floor`, a + substantively meaningful design precision `P_design > P_floor`, + one-sided size `alpha`, power `1 - beta`, and its multiplicity rule. + For independent, equal-probability accepted assignments within a + simple-random operative stratum, the target is the smallest number + `n` for which an integer critical count `c` exists such that + + `Pr[X >= c | X ~ Binomial(n, P_floor)] <= alpha` + + and + + `Pr[X >= c | X ~ Binomial(n, P_design)] >= 1 - beta`. + + Repeated spells, pairs, and runs are not independent Bernoulli draws. + A worker-, firm-, or time-clustered, unequal-probability, finite, or + dual-frame design instead uses design-based analytic or simulation + power with the registered clustering unit, inclusion weights, finite- + population correction where material, and anticipated design effect. + The target includes unusable-record, indeterminate, and nonresponse + inflation. If multiple strata must clear, IC3 distinguishes simultaneous + confidence coverage from an intersection-union pass rule and computes + the **joint** probability that every required stratum passes at + `P_design`; powering each stratum separately at `1 - beta` is not + enough. A round number without the applicable calculation is not a + target-size justification. The eligible-universe truth-search arm has + its own registered target, based on expected true-counterpart + prevalence and a desired recall-bound width or, if recall gates, an + analogous floor-and-power calculation. It must contain enough + independently searched true links to make false-negative uncertainty + informative. +4. **Blind, independent coding.** Two trained coders independently see + the same source evidence and candidate set, but not the matcher's + score, cutoff, accepted choice, downstream outcome, or the other + coder's decision. They code `link`, `no link`, or `insufficient + evidence` under a frozen manual. A third coder or standing panel + adjudicates disagreements without majority labels being disclosed + first. Before labels open, IC3 registers whether an indeterminate, + unusable-evidence, or nonresponse case counts conservatively as + incorrect, enters partial-identification bounds, or makes the floor + unevaluable; hard cases may not be dropped from denominators after + their labels are known. The artifact reports each rate, its effect on + precision and recall denominators, agreement, disagreement, coder/ + manual versions, and final dispositions. Coders first calibrate on + separate, vetted cases excluded from both audit arms; blinded + audit/gold repeats measure + continuing coder accuracy as well as agreement. The registered + precision test either propagates estimated reference-label error or + publishes the sensitivity bound needed to clear the floor. The label + is **hand-adjudicated reference truth**, not a claim of infallible + ground truth. +5. **Provenance is committed and immutable.** Commit a privacy-safe + manifest containing the frame query and vintage, stratum definitions, + random seed, selected-row hashes or access-controlled immutable IDs, + inclusion probabilities, evidence and coding-manual versions, coder + assignment protocol, adjudication rule, counts, and artifact hashes. + Raw restricted records and direct identifiers remain outside git. + Any replacement or exclusion is logged; the sample is never silently + refreshed. +6. **No train-test leakage.** Neither adjudication arm, its final labels, + nor disagreement dispositions may train, tune, select features for, + set a cutoff for, or otherwise adapt the matcher it scores. A leaked + sample is retired from evaluation and replaced under a new manifest. + +#### First-lock scope, ownership, and sidecar location + +The first IC3 lock applies this audit to **E4/E5 SIPP-internal employer +attachment only**. Within-panel `EJB` job-ID attachment is an assignment +for which hand-adjudicable reference evidence can exist. The full +five-band × NAICS-major × transition-class × `primary_job` grid is +phased behind the first lock. E9 transition classes, E11 ordered pairs, +and any later-promoted gate must clear a new audit at promotion time; +none inherits an E4/E5 pass. + +Workstream A (`@daphnehanse11`) owns the E4/E5 adjudication frame, +coding manual, coder-panel operation, and privacy-safe audit manifest. +Workstream B (`@vahid-ahmadi`) owns the firm-side sidecar schema, +versioning rules, and firm-side audit artifacts. Versioned sidecars +join IC1 on `person_id` and `spell_id`; they never amend IC1. Before the +first audit opens labels, the reusable schema and versioning contract +must be committed at +`docs/design/employer_linkage_qc_sidecar.schema.json`. Privacy-safe +manifests and results must use +`runs/employer_linkage_qc__v.json`. Restricted evidence, +direct identifiers, coder identities, and access-controlled lookup +keys remain outside git. + +### 3. Linkage-bias reweighting on observables + +1. **Name the target population and identification assumption.** Each + linked analysis declares its actual analysis unit (assignment, pair, + transition, run, or firm/type cluster), the eligible reference + population it is intended to represent, and the pre-link observables + `X` available for both linked and unlinked units. + Candidate observables include source/vintage, age, sex, state, + `class_of_worker`, `primary_job`, industry information known before + the assignment, earnings/tenure measures, missingness, and transition + opportunity. For each E-cell outcome `Y`, the registered identification + claim is usable-link inclusion independent of `Y` conditional on `X`, + plus positivity on the target support. It is an assumption to defend, + not a result of a balance test. Post-link outcomes cannot be used to + manufacture balance. +2. **Estimate and publish actual inclusion propensities.** Fit and + version `s_i = Pr(L_i = 1 | X_i)`, where `L_i` denotes inclusion in + the usable linked subsample of the complete eligible universe at the + E-cell's analysis unit. A person-level score does not automatically + weight a pair, run, or cluster; IC3 models that unit's inclusion or + pre-registers and justifies a joint construction from component + scores. Publish the model specification, training + population, out-of-sample diagnostics, propensity distributions for + linked and reference units, overlap/common-support checks, covariate + balance before and after weighting, weight distribution, and effective + sample size. Trimming, stabilization, normalization, and capping rules + are pre-registered. +3. **Use a weight derived for the sampling construction.** For actual + inclusion propensity `s_i`, full-population inverse-probability + weighting uses `1 / s_i`, subject to the registered base design. + Bailey, Cole, and Massey instead append a linked-sample copy to a + reference-population copy and fit + `r_i = Pr(D_i = linked copy | X_i)` in that stack. For this density- + ratio construction only, their normalized inverse-odds weight is + `[(1 - r_i) / r_i] [q / (1 - q)]`, where `q` is the linked-copy + share of the stack. `s_i` and `r_i` are not interchangeable. The + linkage adjustment multiplies the pre-existing survey/design/ + opportunity weight; it does not replace that base weight. IC3 derives + and registers the applicable form before seeing the E-cell. +4. **Publish weighted and unweighted with valid uncertainty.** Every + E-cell consuming links publishes both, labels the registered operative + version + (`unweighted` or `linkage_ipw`), and explains its estimand. The + `unweighted` label means base-weighted without the linkage adjustment, + not equal-record weighting that discards a source survey design. The + adjudication sample-inclusion weight and the linkage-propensity weight + are distinct and must not be conflated. Intervals and gate statistics + account for estimated propensities, base survey/design weights, any + audit weights entering the estimator, trimming or stabilization, and + repeated-person, firm/type, and time clustering. A point estimate and + effective sample size alone are insufficient. +5. **Treat overlap failure as scope failure.** Extreme weights, absent + common support, or material residual imbalance are reported, not + hidden by ad hoc trimming. If the registered weighting diagnostic + fails, the weighted cell is invalid. The unweighted diagnostic remains + visible but cannot be substituted as the gate after results are known. + Support-based trimming changes the target population, which must be + renamed and reported rather than presented as the original estimand. + These weights mitigate selection on included observables only; they + do not correct a wrong link, establish balance on unobservables, or + turn a firm type into an observed firm identity. + +### 4. Battery wiring and invalidation + +An overall floor failure invalidates every linked cell using that +matcher version. A required-stratum failure invalidates every cell whose +estimand includes that stratum, unless IC3 pre-registers a genuinely +disjoint matcher and estimand. A passing marginal link floor does not by +itself certify a pair or a run: IC3 must either audit the derived unit +directly or register and justify a conservative composition rule. + +| cell | linkage unit and required QC | weighting and failure disposition | +|---|---|---| +| **E4 — retention pairs** | Audit endpoint assignments and, if IC3 makes it operative, the derived same-employer/same-attribute decision. The audit strata include age, industry, IC2 band, transition month, and `primary_job` status used by the cell. | Publish pair-opportunity estimates unweighted and with linkage-IPW. If any IC3-designated endpoint or pair-level floor fails, all affected E4 retention cells are invalid. | +| **E5 — attachment runs** | If IC3 designates a run-level floor, audit the full multi-window run label, including false continuation and false break errors. A per-month pass alone cannot certify a run because error compounds with length. | Weight the eligible run opportunity, not each observed linked month as if independent. If any IC3-designated endpoint or run-level floor fails, the affected E5 run-length cells are invalid. | +| **E9 — earnings-change coherence** | Derive stay, job-to-job, exit, and entry from adjacent IC1 spells plus the versioned person-month observation/nonemployment frame. Audit any IC3-designated transition floor; for job-to-job cells, audit both origin and destination firm-size/industry assignments. | Define propensity and composite weight on the eligible transition opportunity, then publish both versions within class. A failed IC3-designated origin, destination, or transition-class floor invalidates the corresponding E9 cells; the referee cannot replace them post hoc with a different definition. | +| **E11 — firm-size flow ladder** | For any per-record audit, the unit is an origin-destination job-to-job pair and the joint ordered IC2-band assignment is audited. An unresolved `BandSpan` is not a correct categorical assignment merely because it contains the eventual band. A separately registered held-out aggregate destination-band distribution may gate under the boundary below. | Model inclusion and weight at the ordered-pair opportunity for per-record claims. A failed IC3-designated origin, destination, or transition-class floor invalidates the corresponding per-record E11 cells. An aggregate E11 gate certifies only reproduction of its registered aggregate distribution. | +| **E12 — variance and coworker structure** | Phase 2 must audit worker-to-firm-type assignment and any generated same-firm or coworker co-assignment at the exact unit the E12 estimand uses. Type agreement alone cannot validate a claim about an identified firm. | Model inclusion at the worker pair, co-assignment, or cluster unit used by the decomposition; a worker-only propensity is insufficient without a justified composition. Publish both versions. If truth, floor, or support fails, every E12 cell using it is invalid and phase 2 is a no-go. | + +E3 and E8 do not ordinarily require a worker-to-firm link, and E10 +re-runs the existing locked PSID earnings gates without a new noise +floor. They are not blanket exemptions: if a final IC3 implementation +constructs any of them from accepted employer or firm-type assignments, +the precision-first law applies. Linkage failure never weakens E10 or +changes an existing PSID threshold. + +### 5. Phase scope and real seams + +#### Phase 1 — spell hazards, no two-sided register + +Phase 1 has within-panel employer attachment and firm attributes on +spells, not an observed two-sided worker-firm roster. The SIPP reader's +`EJB{n}_JOBID` is a within-panel attachment key. Its spell collapse +currently carries raw `empsize_code`; `sipp_empsize_to_canonical` in +`banding.py` preserves source-band ambiguity through `BandSpan`, while +the frozen IC1 seam ultimately carries one `CanonicalBand`. Therefore: + +1. QC scores the **final accepted assignment consumed by the E-cell**, + after any ambiguity resolution, not the raw SIPP code or a claim that + an ambiguous span is exact. +2. An exact SIPP interval-to-band map establishes only numeric interval + nesting. SIPP measures establishment size, so it does not by itself + validate administrative enterprise size. +3. Sidecars join to IC1 with `person_id` and `spell_id`. They do not add + `job_id`, `firm_id`, match score, adjudication status, or weights to + frozen IC1. +4. E4/E5 score retention and attachment, E9 scores transition-conditioned + earnings changes, and E11 scores firm-size flows only after their + applicable link and reweighting requirements above are evaluable. +5. A QRF-imputed enterprise-size band on a CPS host has no admissible + per-record truth frame. CPS `NOEMP` is the training label and has a + reference-period mismatch; SIPP measures establishment rather than + enterprise size; public LEHD/SUSB data have no person-level link to + CPS; and pre-redesign SIPP is an aged self-report on a different + sample. More fundamentally, a draw from a conditional distribution + is not a claim that an observed host has one adjudicable true class. + Therefore: + + > Per-record or individual-outcome cells conditioning on imputed + > firm-size bands are validated distributionally (calibration fit to + > SUSB margins plus held-out-axis stability) and are report-only in + > every phase; they gate only if an external person-level truth source + > materializes, at which point they enter through the standard + > promotion ceremony (new floor + ADR 0004 audit). + + This per-record status is permanent for the current evidence regime, + not a provisional deferral. Calibration fit is a build check and may + not be described as independent validation. + + A genuinely held-out aggregate distribution such as E11's + destination-size margin may gate when it is disjoint from every + calibration target and IC3 locks its exact statistic, population, cell + list, floor, and automatic demotion rule before fitting. Such a gate + certifies aggregate-distribution reproduction only. It does **not** + validate any worker's assigned band, true worker-employer linkage, + worker sorting, coworker structure, or firm effects. This boundary + implements the HIPSM-first synthetic-firm decision recorded on + [issue #282](https://github.com/PolicyEngine/populace-dynamics/issues/282): + aggregate validation is admissible now, while true-linked validation + remains a future promotion requirement. + +The pre-IC3 floor draft on +[PR #212](https://github.com/PolicyEngine/populace-dynamics/pull/212) +does not satisfy or conflict with this ADR: it estimates sampling noise +in E3/E4/E5/E8/E9 after the linkage inputs are defined. Its half-splits +cannot reveal a common linkage bias. The seam artifact on +[PR #214](https://github.com/PolicyEngine/populace-dynamics/pull/214) +likewise remains a separate prerequisite: it compares SIPP and J2J rate +levels, but does not estimate precision or recall for a worker-to-firm- +type assignment. + +#### Phase 2 — BLM firm types, still not observed firms + +Phase 2 proposes a BLM-style register of firm **types** (industry x +size x state), not identified enterprises or a public worker-firm +roster. E12 is especially sensitive to false assignments because a bad +worker-to-firm or coworker link moves covariance between the within- and +between-firm components. Bailey, Cole, Henderson, and Massey (2020), +section VI.C and Figure 7, show that false links often attenuated their +intergenerational-income-elasticity estimates and that removing them +reconciled estimates. Their broader result also warns that systematic +error can make the bias algorithm-dependent rather than always +attenuating. + +For that reason, a phase-2 E12 story must identify an admissible +hand-adjudication frame for the actual assignment unit and clear its +precision floor. If public margins cannot support that truth, IC3 records +the limitation as a phase-2 no-go; calibration fit to aggregates is not +a substitute. The register may support firm-type policy claims only at +the level it identifies. It may not relabel type agreement as firm- +identity or coworker validation. + +### 6. Items for the IC3 referee round — deliberately unresolved + +The referee round must decide and pre-register the following. This ADR +does not resolve them: + +1. The numeric precision floor or floors; `P_design`, `alpha`, power, + confidence interval, multiplicity correction, and whether recall has + an operative floor. +2. Within the first-lock E4/E5 scope fixed in section 2, the permissible + evidence, exact truth labels, privacy-safe manifest details, and + numeric coder-panel design; for later phases, the E9/E11 frames and + whether an E12 truth source exists at all. Imputed CPS enterprise-size + bands follow section 5's permanent report-only degradation rule and + are not an unresolved hand-adjudication frame. +3. Beyond E11's required ordered pair and the actual co-assignment unit + used by E12, which endpoint, pair, transition, and run levels gate; + how a passing endpoint result composes, if at all; and which sparse + strata may be pooled before the powered sample target is calculated. +4. The target reference population, propensity model and observables, + overlap and balance tolerances, weight stabilization/trimming rule, + and the pre-registered operative weighting for E4, E5, E9, E11, and + E12. +5. The exact calibration/gate cell partition, including held-out axes; + the job-count-to-person-count adjustment for phase 0; the private- + comparable versus caveat treatment of J2J's broader employer + universe; and the ASEC firm-size/tenure reference-period mismatch. +6. PR #212's measurement choices: E3 quantile gaps versus weighted-ECDF + gaps, and E9 stay IQR versus a broader distributional distance. The + integer and wave-constant heaping degeneracies remain on the record. +7. PR #214's cross-wave SIPP job-ID consistency check and the final + ruling on J2J rate levels, SIPP persistence, and seam-aware hazard + estimation. +8. The E12 estimand and entity: firm type versus generated pseudo-firm, + the minimum evidence for within/between variance and coworker + correlation, and the phase-2 identification/go-no-go standard. +9. Long-window tenure evidence from PSID/NLSY and any transport test for + applying one adjudication result across source vintages or populations. + +The settled IC1 fields, IC2 semantics and five bands, explicit +`NOEMP`/`FIRMSIZE` coding, class-of-worker universe, and person-table +geography join are outside this list. Reopening them requires the joint +IC1/IC2 amendment process, not the IC3 referee round. + +## Consequences + +- IC3 gains a linkage-quality input gate before its model-fit gates. + E4, E5, E9, E11, and E12 cannot certify a candidate from unaudited or + floor-failing assignments. +- Every link-consuming cell exposes the observable-selection question + by publishing linkage-IPW and unweighted estimates together, with one + operative version chosen before the candidate result is seen. +- The required evidence is carried in sidecars and reported artifacts; + IC1, IC2, `gates.yaml`, readers, and banding code are unchanged. +- Exact floor values, E-cell thresholds, and operative weighting choices + remain decisions for the IC3 referee round. + +## References + +- Bailey, Martha, Peter Z. Lin, A. R. Shariq Mohammed, Paul Mohnen, + Jared Murray, Mengying Zhang, and Alexa Prettyman. 2023. + ["The Creation of LIFE-M: The Longitudinal, Intergenerational Family + Electronic Micro-Database Project."](https://doi.org/10.1080/01615440.2023.2239699) + *Historical Methods* 56 (3): 138-159. +- Bailey, Martha J., Connor Cole, Morgan Henderson, and Catherine Massey. + 2020. ["How Well Do Automated Linking Methods Perform? Lessons from US + Historical Data."](https://doi.org/10.1257/jel.20191526) + *Journal of Economic Literature* 58 (4): 997-1044. The 29-percent + figure above follows the + [final AEA abstract](https://www.aeaweb.org/articles?id=10.1257/jel.20191526), + while the + [earlier deposited author manuscript](https://pmc.ncbi.nlm.nih.gov/articles/PMC8294155/) + reports 20 percent in its abstract, introduction, and section VI.C. +- Bailey, Martha, Connor Cole, and Catherine Massey. 2019 online / 2020 + print. ["Simple Strategies for Improving Inference with Linked Data: A + Case Study of the 1850-1930 IPUMS Linked Representative Historical + Samples."](https://doi.org/10.1080/01615440.2019.1630343) + *Historical Methods* 53 (2): 80-93. diff --git a/docs/plans/employer-firm-plan.html b/docs/plans/employer-firm-plan.html index 413453ea..56480a6b 100644 --- a/docs/plans/employer-firm-plan.html +++ b/docs/plans/employer-firm-plan.html @@ -255,11 +255,11 @@

Targets, register & calibration

INTERFACE CONTRACTS — frozen week 1, changed only by joint PR
    -
  • C1 · Spell schema. One table: person_id, spell_id, start_period, end_period, industry (major), firm_size_band, earnings_share, primary_job. A writes it, B reads it. Firm-size bands use the canonical banding B defines (C2). Multi-job resolved primary-job-only in phase 0.
    • -
  • C2 · Canonical firm-size banding + semantics. B proposes the band set reconcilable across NOEMP / SIPP-establishment / SUSB-enterprise, and the decision of what the variable means (administrative firm size, per review F5). A trains to it; documented in the ADR.
    • -
  • C3 · Gate pre-registration. Jointly authored employer gate block (E1–E12 thresholds after floor runs), split ownership as above, one referee round, locked before any candidate runs. Neither side's model work may start a one-shot run until C3 locks.
    • +
  • IC1 · Spell schema. One table: person_id, spell_id, start_period, end_period, industry (major), firm_size_band, earnings_share, primary_job. A writes it, B reads it. Firm-size bands use the canonical banding B defines (IC2). Multi-job resolved primary-job-only in phase 0.
    • +
  • IC2 · Canonical firm-size banding + semantics. B proposes the band set reconcilable across NOEMP / SIPP-establishment / SUSB-enterprise, and the decision of what the variable means (administrative firm size, per review F5). A trains to it; documented in the ADR.
    • +
  • IC3 · Gate pre-registration. Jointly authored employer gate block (E1–E12 thresholds after floor runs), split ownership as above, one referee round, locked before any candidate runs. Neither side's model work may start a one-shot run until IC3 locks.
-

Sync points: week 1 (freeze C1/C2), week 4 (lock C3), week 10 (joint phase-2 go/no-go with Max). Everything else is asynchronous — A can build readers/imputation against fixture spells; B can build the target pipeline and register against a synthetic spell file conforming to C1.

+

Sync points: week 1 (freeze IC1/IC2), week 4 (lock IC3), week 10 (joint phase-2 go/no-go with Max). Everything else is asynchronous — A can build readers/imputation against fixture spells; B can build the target pipeline and register against a synthetic spell file conforming to IC1.

Precedents — what similar projects did

@@ -277,8 +277,8 @@

Precedents — what similar projects did

Milestones

- - + + diff --git a/scripts/build_noemp_band_evidence.py b/scripts/build_noemp_band_evidence.py index dc1a23cd..9bcc886c 100644 --- a/scripts/build_noemp_band_evidence.py +++ b/scripts/build_noemp_band_evidence.py @@ -3,7 +3,7 @@ REPORTED ANCHOR, NOT A GATE RUN. Like the mortality/claiming/ disability floors, this reads no gate and decides nothing on its own; it is committed evidence pinned by a reproduction test. It -records the empirical basis for the C2 banding decision's treatment +records the empirical basis for the IC2 banding decision's treatment of CPS ASEC firm size: **the 2019+ data dictionaries' relabeling of NOEMP codes 2/3 (from 10-49 / 50-99 to 10-24 / 25-99) never happened in the instrument.** diff --git a/tests/test_firms_banding.py b/tests/test_firms_banding.py index ae8e1f3f..3c474b64 100644 --- a/tests/test_firms_banding.py +++ b/tests/test_firms_banding.py @@ -1,4 +1,4 @@ -"""Tests for the canonical firm-size banding (contract C2). +"""Tests for the canonical firm-size banding (contract IC2). Checks the properties the contract promises: canonical bands partition the positive integers; every raw source code maps to diff --git a/tests/test_noemp_band_evidence.py b/tests/test_noemp_band_evidence.py index 1a98c64a..4f1c1c44 100644 --- a/tests/test_noemp_band_evidence.py +++ b/tests/test_noemp_band_evidence.py @@ -1,7 +1,7 @@ """Pin the NOEMP band-label evidence artifact (issue #192). The committed ``runs/noemp_band_evidence_v1.json`` records the -discontinuity test behind the C2 decision to read ASEC NOEMP codes +discontinuity test behind the IC2 decision to read ASEC NOEMP codes 2/3 as 10-49 / 50-99 in every year. These tests pin the artifact's internal consistency, and — when the ASEC files are staged — reproduce it from the raw data.
Workstream A — Daphne (person side)Workstream B — Vahid (firm side)
SIPP job-level reader (label-verified, family.py pattern); CPS NOEMP/tenure loaders; ADR drafted jointly · freeze C1/C2Target pipeline: SUSB/BDS/QWI/J2J/JOLTS extracts committed with provenance notes; canonical banding proposal (C2)
SIPP noise-floor runs; seam-vs-J2J reconciliation run; draft E3–E5/E8–E10 thresholdsAggregate-side floor studies; target/gate partition; draft E1/E2/E6/E7/E11 thresholds · joint: referee round, lock C3
SIPP job-level reader (label-verified, family.py pattern); CPS NOEMP/tenure loaders; ADR drafted jointly · freeze IC1/IC2Target pipeline: SUSB/BDS/QWI/J2J/JOLTS extracts committed with provenance notes; canonical banding proposal (IC2)
SIPP noise-floor runs; seam-vs-J2J reconciliation run; draft E3–E5/E8–E10 thresholdsAggregate-side floor studies; target/gate partition; draft E1/E2/E6/E7/E11 thresholds · joint: referee round, lock IC3
Phase-0 QRF imputation of spells + attributes onto CPSCalibration of the imputed file to partitioned QWI/SUSB cells; E1/E7 evidence artifacts
Phase-1 transition candidates registered; one-shot runs against locked gatesBLM firm-type register prototype; E12 feasibility study (are published AKM/coworker moments sufficient targets?)
Joint: phase-2 go/no-go review with Max, based on committed gate evidence + the E12 identification story