Skip to content

Evaluation harness: support aged targets (declared fact-ratio levels) in precomputed checkpoints #171

Description

@MaxGhenis

Found publishing Microcosm-BE v0.5 (2025 simulation) to the cross-dataset page. v0.5's release surface contains 233 aged targets — 2025 levels derived as published fact × declared FPB fact-ratio with both record ids and the factor in the manifest aging block. The precomputed-checkpoint adapter (#164) enforces two constraints that are jointly unsatisfiable for such releases:

  1. every checkpoint row's period must equal its Chronicle facts' period (precomputed checkpoint row 'statbel_b1_decile_01_total_tax' period does not match its Chronicle facts — the row is a 2025 level over …income_year2023… facts);
  2. the checkpoint row set must cover the authoritative Microcosm release surface (filtering the aged rows out fails with differs from the authoritative Microcosm release surface; missing=[233 rows]).

The US path solves this with first-class aging transforms (microcosm_aging, period treatment aligned_fact). The BE precomputed path needs the equivalent: accept a per-row alignment declaration (numerator/denominator fact ids + factor, mirroring the release manifest's aging block), verify the arithmetic against the named facts, and surface it as the existing aligned_fact period treatment with calibration_exposure intact — never silently accepting a mismatched period.

Until then the datasets page stays on the v0.4 run (evaluation-f28ca06a…, unaged surface) while the calibration-fit page serves v0.5; the v0.5 checkpoints are in population-rerun/out/v05b/evaluation/ ready to re-run once supported.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions