Skip to content

Workstream A: label-verified CPS January tenure-supplement loader (PTST1TN) + E3 evidence tabulation #198

Description

@daphnehanse11

Part of #192, Workstream A week-1 deliverable ("CPS ASEC NOEMP + tenure-supplement loaders"). Companion to #193/#194 (the NOEMP side); this issue covers the job-tenure side.

What

A label-verified reader for the CPS January Job Tenure supplement (the biennial Displaced Worker / Employee Tenure / Occupational Mobility supplement) in src/populace_dynamics/data/, same discipline as the NOEMP loader: dictionary-adjudicated variable maps with provenance, loud refusal on anything unverified — plus the weighted tenure-by-age tabulation that is the E3 gate's evidence base (tenure P25/P50/P75 by age band, reference: CPS tenure supplement).

Verified facts (Census January 2024 technical documentation, cpsjan24.pdf, read 2026-07-14)

  • PTST1TN is the primary variable: Census's edited recode for employer tenure "expressed in years, with two implied decimals," incorporating usable responses from the raw items PEST1A (amount), PEST1B (periodicity), and PEST3 (exact months when tenure < 3 years). Usability per the dictionary: PEST1A in 1–99 (PEST3 1–35) and age minus tenure ≥ 14 years. Valid entries 0–3100 (topcoded at 31.00 years); −1 NIU, −2 DK, −3 refused, −9 no response. Preferring the recode over reconstructing from the raw trio is deliberate: the edit rules live with Census, not with us.
  • PWTENWGT is the tenure-supplement weight (10 digits, 4 implied decimals) — not the basic weight and not PWSUPWGT (the displaced-worker weight).
  • The question wording is "How long have you been working CONTINUOUSLY … for your present employer" — the current job at the January interview, which is the reference-period mismatch with ASEC firm size (preceding year's longest job) pre-registered in ADR 0003; the loader documents it and carries interview month/year.
  • The same person file carries PRTAGE, PESEX, GESTFIPS, PEMLR, and PEIO1COW (class of worker, main job) — enough for the E3 age bands and for universe alignment with the calibration side.
  • The supplement asks no firm-size question (verified for the Employer–firm relationships: US-first plan, evaluation gates, and two-workstream split #192 review) — this loader supplies the tenure margin only; the size × tenure joint comes from the SIPP 2008-panel bridge per ADR 0003.
  • Trap for the unwary, documented in the module: HETENURE/HXTENURE on the same file are housing tenure (own/rent), not job tenure.

Scope

  • data/cps_tenure.py: read staged jan{yy}pub.csv person files (natively published by Census for 2020, 2022, 2024 — verified to carry PTST1TN/PWTENWGT; 2018 and earlier are fixed-width .dat only and enter later via a documented conversion, as with pre-2017 ASEC), staged under ~/PolicyEngine/cps-tenure-data with an env override. Per-year dictionary verification (cpsjan20/22/24.pdf) before a year is accepted; unsupported years refused.
  • Output: one row per supplement person with a usable tenure response — tenure_years (PTST1TN / 100), topcode flag, age, sex, state FIPS, class of worker (raw + label), employment status, weight; nonresponse codes (−1/−2/−3/−9) excluded with counts surfaced, never silently dropped.
  • tenure_tabulation(...): PWTENWGT-weighted P25/P50/P75 of tenure by age band (and optionally sex/class-of-worker) per year — the E3 evidence artifact and the input to Workstream A's noise-floor work.
  • Read-time verification in the NOEMP-loader style: required columns, value domains (0–3100 ∪ {−1,−2,−3,−9}), the age−tenure ≥ 14 usability invariant, weight non-negativity, path/year consistency for jan{yy} filenames.

Non-goals

Acceptance

  • Fixture tests for both the value path (implied decimals, topcode flag) and every refusal path; a skipif-gated real-data test.
  • Reading a fixture with age 30 and tenure 20 years fails the age−tenure ≥ 14 invariant loudly.
  • Tabulation reproduces fixture-known weighted quantiles.
  • Module docstring records dictionary provenance for each supported year.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions