Add optional mortality and ageing population graph

.github/workflows/tests.yml

@@ -37,16 +37,37 @@ jobs:
       - name: Run tests (shard ${{ matrix.shard }}/4)
         run: pytest -q --splits 4 --group ${{ matrix.shard }} --splitting-algorithm duration_based_chunks --durations-path .test_durations
 
+  graph-integration:
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ['3.13', '3.14']
+    steps:
+      - uses: actions/checkout@v6
+      - uses: actions/setup-python@v6
+        with:
+          python-version: ${{ matrix.python-version }}
+          cache: 'pip'
+      - name: Install the pinned optional graph integration
+        run: pip install -e ".[graph]" pytest
+      - name: Require typed graph capabilities
+        run: python -c "from populace_dynamics.graph._compat import require_graph; require_graph()"
+      - name: Run graph and existing mortality regressions
+        run: pytest -q tests/test_graph_mortality.py tests/test_m6_engine_refit.py tests/test_m6_engine_steps.py
+
   # Fan-in jobs keeping the branch-protection context names
   # ("pytest (3.11)" / "pytest (3.13)") stable across the shard split.
   pytest:
     name: pytest (${{ matrix.python-version }})
-    needs: pytest-shard
+    needs: [pytest-shard, graph-integration]
     if: always()
     runs-on: ubuntu-latest
     strategy:
       matrix:
         python-version: ['3.14']
     steps:
       - name: Verify all shards passed
-        run: test "${{ needs.pytest-shard.result }}" = "success"
+        run: |
+          test "${{ needs.pytest-shard.result }}" = "success"
+          test "${{ needs.graph-integration.result }}" = "success"

docs/population-graph.md

@@ -0,0 +1,128 @@
+# First population graph: mortality and ageing
+
+The optional `populace_dynamics.graph` package fits the existing M6 mortality
+model on a historical synthetic panel, applies the fitted artifact to a
+separate starting population, and adds surviving observations for the next
+year. It produces an engineering report and a content-addressed execution
+manifest. It does not change the existing projection loop, candidate
+registries, scientific gates, or committed evidence.
+
+## Dependencies and execution
+
+The graph example requires Python **3.13 or 3.14**, NumPy 2+, pandas 2.3+,
+and the Microcosm graph/frame revisions containing typed artifact edges and
+`microcosm.graph.randomness.keyed_uniform`. The default Dynamics installation
+continues to support Python 3.10–3.14 without importing Microcosm. The entry
+point checks capabilities and gives installation guidance when they are absent.
+
+The `graph` extra pins both graph and Frame to core commit
+`3ff92b0aea14407d09479bff5623dc7d1a92d008`. An older package merely sharing
+the version number `0.1.0` is insufficient. In an isolated Python 3.13 or 3.14
+environment, install with `uv pip install '.[graph]'`. The core change must be
+reviewed before this dependent integration is released; replace the Git pins
+with a compatible published release when one exists. CI installs this exact
+extra, refuses missing capabilities, and runs the integration on both supported
+Python versions. Do not modify the existing scientific gate environment. No
+rules engine, restricted microdata, or optional forest fitter is needed here.
+
+Run from that environment, choosing an output directory:
+
+```sh
+python -m populace_dynamics.graph --synthetic --output-dir ./mortality-example
+```
+
+The command creates small synthetic inputs under `mortality-example/inputs`.
+It preserves existing input files so that edits can test cache invalidation.
+Repeated execution reuses the verified store under `mortality-example/store`.
+The report, manifest, fitted JSON model, entity tables, and next-period slice
+are written inside the chosen output directory. A failed engineering or
+fixture verdict exits nonzero while retaining the diagnostics.
+
+Four explicit source paths can replace the generated inputs:
+
+```sh
+python -m populace_dynamics.graph \
+  --training ./inputs/training.json --rates ./inputs/rates.json \
+  --initial ./inputs/initial.json --holdout ./inputs/holdout.json \
+  --boundary-year 2014 --external-vintage-year 2014 \
+  --experiment-id comparison-a --replicate 0 --base-seed 0 \
+  --output-dir ./mortality-example
+```
+
+These inputs still exercise the synthetic engineering contract. The example
+does not confer validity on a real-population projection. Source JSON rejects
+duplicate members and nonfinite values. Each source is declared separately;
+holdout bytes are available only to evaluation. Domain kernels read the
+content-verified JSON directly: the registered source marker deliberately
+does not pretend an external rate table or a holdout report is a population.
+
+## Executable ownership
+
+The graph has two CREATE roots, each carrying a `person_period` observation
+entity and `person` and `period` groups. `person` retains stable identities;
+`period.period` is the immutable mass-partition label. The training root
+contains exposure records; the initial root contains recipients. Their only
+connection is the explicitly typed mortality-model artifact.
+
+The fit node calls `prepare_mortality_refit_inputs` and
+`fit_mortality_model`. Event year, required interview year, and declared
+external vintage retain the existing cutoff checks. The JSON model contains
+validated contiguous age bands, sex-specific probabilities, fit boundary,
+external vintage, and retained row count. The manifest binds its producer
+to source identities and implementation digests. The fitter's external-rate
+factor cancels in its fitted-window level, so this is not evidence of
+independent external calibration.
+
+Application calls `apply_mortality` with a graph-specific context. Every
+uniform is keyed by the original person identity, process, year, and draw
+index under the chosen experiment/replicate/seed. It does not use the legacy
+ID-sorted ordinal registry. Reordering, splitting, or adding unrelated
+people preserves the existing people's draws. Fit and application declare
+platform-specific bitwise numeric behavior conservatively; cross-platform
+equality is not claimed.
+
+EXPAND calls `advance_age` on survivors, adds their next-period observations
+with lineage to the original observations, and attaches them to one newly
+admitted period group. A same-version rewrite node claims the materialized
+age values. Historical ages and memberships remain unchanged. The temporary
+`year` returned by `advance_age` is never written over a carried observation
+column. No new person, birth, or immigrant is implied by admission of the
+period group.
+
+Typed person-period weights are the single authority. Every survivor carries
+the same trajectory weight into the new period. The declared mass receipt
+shows historical mass unchanged and new-period mass equal to surviving
+weight. Total stored observation mass therefore grows by the additional
+period. If everyone dies, the graph adds no observations and no orphan period
+group; the report explicitly records next-period mass zero.
+
+## Evaluation and limits
+
+The report separates `engineering_verdict` (survivor/age parity and population
+structure) from `fixture_verdict` (the independently sourced synthetic
+death-rate and age expectations). It records weighted expected, observed,
+and generated deaths, row counts, period mass, node/model identities, and
+cache reuse. The fixture death-rate tolerance is an input named
+`fixture_max_abs_death_rate_gap`; it is not a scientific acceptance threshold.
+Changing all held-out outcomes to deaths fails that fixture check while
+leaving fitting, application, draws, and accounting unchanged.
+
+Household accounting is explicitly unsupported and refused by the Python
+entry point. Household weight sharing, marriage, births, immigration,
+alignment replay, repeated years, and the full M6 loop remain later work.
+No certified data release or scientific candidate is produced by this graph.
+
+## Tests
+
+```sh
+python -m pytest -q tests/test_graph_mortality.py \
+  tests/test_m6_engine_refit.py tests/test_m6_engine_steps.py
+```
+
+The integration tests cover direct execution with an independently injected
+uniform vector, JSON validation, cutoff and holdout isolation, fitted-artifact
+reuse, changed fitting weights, row/chunk/person invariance, cold/warm stores,
+and zero/all-survivor expansion. They skip the optional runtime cases when
+the required core capabilities are unavailable; the JSON and dependency
+boundary tests still run. Importing `populace_dynamics.graph` remains safe
+under Python 3.10–3.12.

pyproject.toml

@@ -47,6 +47,12 @@ dependencies = [
 ]
 
 [project.optional-dependencies]
+# Typed artifacts are not yet in the published 0.1.0 packages. Pin both
+# graph and Frame to the same reviewed core revision; this extra needs 3.13+.
+graph = [
+    "microcosm-frame @ git+https://github.com/PolicyEngine/microcosm.git@3ff92b0aea14407d09479bff5623dc7d1a92d008#subdirectory=packages/microcosm-frame",
+    "microcosm-graph @ git+https://github.com/PolicyEngine/microcosm.git@3ff92b0aea14407d09479bff5623dc7d1a92d008#subdirectory=packages/microcosm-graph",
+]
 dev = [
     "pytest>=7.4.0",
     "black>=23.7.0",

scripts/first_estimates_birth_evidence.py

@@ -156,6 +156,15 @@
     Path("src/populace_dynamics/estimates/anchor_context_registry.py"),
     Path("src/populace_dynamics/estimates/anchor_context_rehearsal.py"),
     Path("src/populace_dynamics/estimates/anchor_context_report.py"),
+    # The opt-in graph integration is outside the historical reducer and
+    # registered production call paths.  Keep exact file exclusions, with
+    # import-reachability coverage, rather than changing any evidence pin.
+    Path("src/populace_dynamics/graph/__init__.py"),
+    Path("src/populace_dynamics/graph/__main__.py"),
+    Path("src/populace_dynamics/graph/_compat.py"),
+    Path("src/populace_dynamics/graph/model.py"),
+    Path("src/populace_dynamics/graph/runtime.py"),
+    Path("src/populace_dynamics/graph/synthetic.py"),
 )
 POST_REVIEW_SHARED_SOURCE_BLOBS = {
     Path(

src/populace_dynamics/graph/__init__.py

@@ -0,0 +1,18 @@
+"""Optional synthetic population-graph integration.
+
+Importing this package does not import Microcosm or change legacy execution.
+The graph entry point checks Python and the installed graph capabilities.
+"""
+
+
+def run_mortality_graph(**kwargs):
+    """Run the existing mortality/ageing operations through Microcosm."""
+    from ._compat import require_graph
+
+    require_graph()
+    from .runtime import run_mortality_graph as run
+
+    return run(**kwargs)
+
+
+__all__ = ["run_mortality_graph"]

src/populace_dynamics/graph/__main__.py

@@ -0,0 +1,66 @@
+"""Run the mortality graph with explicit inputs and output placement."""
+
+import argparse
+from pathlib import Path
+
+from . import run_mortality_graph
+from .synthetic import write_synthetic_inputs
+
+
+def parser():
+    result = argparse.ArgumentParser(description=__doc__)
+    result.add_argument("--output-dir", type=Path, required=True)
+    result.add_argument("--synthetic", action="store_true")
+    for name in ("training", "rates", "initial", "holdout"):
+        result.add_argument(f"--{name}", type=Path)
+    result.add_argument("--boundary-year", type=int, default=2014)
+    result.add_argument("--external-vintage-year", type=int, default=2014)
+    result.add_argument("--experiment-id", default="mortality")
+    result.add_argument("--replicate", type=int, default=0)
+    result.add_argument("--base-seed", type=int, default=0)
+    return result
+
+
+def main(argv=None):
+    arg_parser = parser()
+    args = arg_parser.parse_args(argv)
+    sources = {
+        name: getattr(args, name)
+        for name in ("training", "rates", "initial", "holdout")
+    }
+    if args.synthetic:
+        if any(sources.values()):
+            arg_parser.error("--synthetic cannot be combined with input paths")
+        if args.boundary_year != 2014:
+            arg_parser.error(
+                "the supplied synthetic fixture has boundary year 2014"
+            )
+        sources = write_synthetic_inputs(args.output_dir / "inputs")
+    elif not all(sources.values()):
+        arg_parser.error("supply all four input paths or --synthetic")
+    try:
+        run = run_mortality_graph(
+            **sources,
+            output_dir=args.output_dir,
+            boundary_year=args.boundary_year,
+            external_vintage_year=args.external_vintage_year,
+            experiment_id=args.experiment_id,
+            replicate=args.replicate,
+            base_seed=args.base_seed,
+        )
+    except (ImportError, ValueError) as error:
+        arg_parser.exit(2, f"{error}\n")
+    print(
+        f"{args.output_dir / 'report.json'}: engineering={run.report['engineering_verdict']}, fixture={run.report['fixture_verdict']}"
+    )
+    return (
+        0
+        if run.report["engineering_verdict"]
+        == run.report["fixture_verdict"]
+        == "pass"
+        else 1
+    )
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

src/populace_dynamics/graph/_compat.py

@@ -0,0 +1,36 @@
+"""An explicit dependency boundary for the optional graph example."""
+
+import importlib
+import sys
+
+_GUIDANCE = (
+    "The population graph needs Microcosm's typed model-artifact and keyed "
+    "randomness interfaces. Install the reviewed microcosm-graph and "
+    "microcosm-frame revisions together; see docs/population-graph.md. "
+    "Legacy Dynamics does not require these packages."
+)
+
+
+def _python_version():
+    return sys.version_info[:2]
+
+
+def require_graph():
+    """Refuse unsupported Python or a graph lacking the required interfaces."""
+    if _python_version() < (3, 13):
+        raise ImportError(
+            "The optional population graph requires Python >=3.13."
+        )
+    try:
+        decl = importlib.import_module("microcosm.graph.decl")
+        kernel = importlib.import_module("microcosm.graph.kernel")
+        randomness = importlib.import_module("microcosm.graph.randomness")
+    except (ImportError, SyntaxError) as error:
+        raise ImportError(_GUIDANCE) from error
+    for name in ("ArtifactType", "ArtifactInput", "ArtifactOutput"):
+        if not hasattr(decl, name):
+            raise ImportError(_GUIDANCE)
+    if not hasattr(kernel.SeedSource, "KEYED") or not hasattr(
+        randomness, "keyed_uniform"
+    ):
+        raise ImportError(_GUIDANCE)