Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 36 additions & 1 deletion docs/graph-acceptance.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,8 @@ its owner.
| B3 | **Storage-preserving patch.** Patching owned positions preserves the incumbent column's dtype (nullable `boolean` stays nullable `boolean`; float bits including negative zero survive) and leaves every non-owned position byte-identical. This is the WIC guard, made structural. | leg 2 §3.3 | same |
| B4 | **Inputs are immutable.** A kernel receives read-only views; an in-place write raises inside the kernel and the node fails. | leg 1 finding 5 | same |
| B5 | **Null means absence.** A node declares each owned cell as *produced* or *absent*. A kernel writing a non-null value into an absent-declared cell is rejected. | `DESIGN.md:128-134` | same |
| B6 | **Entrants are declared.** An `EXPAND` node with `entrants=True` may return rows with null lineage; the executor requires the kernel to materialize every carried column for such a row (dtype-checked), records them as entrants rather than copies in the lineage receipt, and refuses null lineage on a node without the declaration. `entrants=True` with `mass='conserve'` is a compile error. | Dynamics: immigrant cohorts (microcosm-dynamics#412, #218) | Max's session; amendment 11 |
| B7 | **Entrant persons carry their stratum.** An entrant row on the person entity takes its stratum from `KernelResult.strata` (indexed by its new id); an entrant person absent from it, a label for a copied or incumbent person, or a label for an unknown id rejects the node. Entrant persons join incumbent or entrant groups through the materialized membership columns, and the mass ledger counts them from the node that admits them. | Dynamics: immigrant cohorts are persons (microcosm-dynamics#412, #218) | Max's session; amendment 14 |

## C. Seeds and factorization

Expand All @@ -54,16 +56,18 @@ its owner.
| C2 | **Removal invariance.** Removing a node that nothing depends on, or adding a new leaf node, changes no other node's key or output. This is the `0347a009` replay: five targets removed, zero survivors re-modeled. | F5 | same |
| C3 | **Declared predecessors only.** A chained target's predictors are exactly its declared predecessors. The executor hands a kernel only its declared slices, so an undeclared read is impossible rather than merely detected. | F5, leg 3 §legibility | same |
| C4 | **Seed from identity.** A node's RNG seed is a pure function of its node key. Two nodes with identical declarations, inputs, and kernels in different graphs draw identical values. No positional RNG consumption exists anywhere in the shard (static check). | F4, `docs/spec-engine.md:254-282` | same |
| C5 | **Tolerance is declared.** A kernel claiming `tolerance_bound` numerics without a `Tolerance` is refused at registration, and a bitwise kernel may not carry one. The tolerance is recorded in every receipt, and a kernel reading a cell sees its owner's declared tolerance in `KernelContext.tolerances`; a gate comparing against anything else says so in its evidence. | H2 (arm64/x86 one-ulp weights); microcosm-dynamics#412 | Max's session; amendment 13 |

## D. Weights and mass

| Id | Property | Closes | Owner |
|---|---|---|---|
| D1 | **Weight transitions are typed nodes.** `design → importance → calibrated` are the only legal transitions; the executor rejects a regression and rejects a transition declared on inherited (non-explicit) weights. | F9 (leg 1 finding 1) | María / Max's session |
| D2 | **Mass ledger.** Every population-changing node (select, concat, clone, reweight) emits a mass record with before/after totals and per-stratum mass. Under `conserve`, a stratum losing mass fails the node. `select` cannot drop mass silently. | F9 | same |
| D2 | **Mass ledger.** Every population-changing node (select, concat, clone, reweight) emits a mass record with before/after totals and per-stratum mass. Mass is weighted person mass per stratum, within each declared partition (amendment 12). Under `conserve`, a stratum losing mass fails the node. An expansion that conserves its weight entity's mass while changing group composition changes person mass and must say so: `declared`, with a receipt stating the invariant it does hold (ruled 2026-09-02 on #844). `select` cannot drop mass silently. | F9 | same |
| D3 | **Cap anchored to design.** A calibration node's `max_weight_ratio` is asserted against the declared anchor across composed stages; a selection-then-refit chain that ships a record above `R × design` fails. | F9 (#493) | same |
| D4 | **Filters are binary.** A target filter containing NaN or a non-binary value is rejected at compile. | F9 | same |
| D5 | **Uncertainty travels.** A target's declared standard error reaches the calibration kernel's inputs; a kernel that ignores a declared `se` must say so in its capability record. | scoreboard row 5 (leg 1 finding 7) | same |
| D6 | **Mass is partitioned.** With `Graph.mass_partition` set, the ledger reports per stratum within each partition value, `conserve` holds within each partition, and a node that moves mass between partitions under `conserve` fails. Every `CREATE` node declares the partition column with a partition dtype, or compilation fails. A row contributes mass only to the partitions it exists in. | Dynamics: person-period residency (microcosm-dynamics#412) | Max's session; amendment 12 |

## E. Store and resume

Expand Down Expand Up @@ -201,6 +205,37 @@ Amendments so far (each re-locked):
(python storage); a population entering the graph with `object`
strings is normalized at `CREATE`. Parity fixtures compare identities
after the same normalization on the legacy side, and say so.
11. **Entrants are declared.** `Node.entrants=True` (EXPAND only) lets a
kernel add rows that copy no base row: their lineage is null, the
kernel materializes every carried column for them, the executor
records them as entrants, and the node's mass policy cannot be
`conserve`. Raised by the dynamics program (immigrant cohorts through
the scheduled-entries seam, microcosm-dynamics#412 / #218); Max ruled
go 2026-09-02; adopted 2026-09-02.
12. **Mass is partitioned.** `Graph.mass_partition = (entity, column)`
partitions mass accounting (per stratum within each partition value;
`conserve` per partition). Every `CREATE` node declares the column
with a dtype in `PARTITION_DTYPES`. The field is normative: the
executor folds it into every structural node's key, so structural keys
move once when a graph adopts it. Raised by the dynamics program for
person-period residency; adopted 2026-09-02.
13. **Tolerance is declared.** `Capabilities.tolerance: Tolerance | None`
(`rtol`, `atol`, `ulps`) is required for `tolerance_bound` kernels and
forbidden for bitwise ones; `KernelContext.tolerances` hands each
reader the declared tolerance of every input cell's owner. Raised by
the H2 parity finding (root weights differ by one ulp between arm64
and x86) and the dynamics review; adopted 2026-09-02.

14. **Entrant persons carry their stratum.** `KernelResult.strata` (EXPAND
kernels on an `entrants=True` node only) names the stratum of every
entrant person by its new id; the executor requires exactly the entrant
persons there. Raised by the implementation of amendment 11, which
found the frozen result had no channel for a new person's mandatory
stratum and left person entrants fail-closed; adopted 2026-09-02.

Adding a normative field with a default changes the canonical projection
of every node that carries it, so node keys moved with amendments 11 and
13's sibling field `entrants`; no released artifact pins a graph key yet.

## Ownership

Expand Down
4 changes: 2 additions & 2 deletions docs/graph-interface.lock
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
b6fbf50e0253e6f04a2684fcac295fb27a4608f6a272bedb86c43f087fd36b8e decl.py
8ac8328c086bfbea0e36032e9c7fa9cb1ed62f63a1036bca4eb528c11b9b1890 kernel.py
bdd74c8598a8537f7ed95ffa712726c8b01d8b8b840e5d9e7006b2cc705bd0ff decl.py
b4315d44f7e54a845d7efd7ac1239a813ddd312ce642869b9e15a2b66f62ea45 kernel.py
9 changes: 9 additions & 0 deletions packages/microcosm-fit/src/microcosm/fit/kernels.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
KernelResult,
Numeric,
SeedSource,
Tolerance,
source_hash,
)

Expand All @@ -48,6 +49,13 @@
)
"""Distributions whose versions form part of ``fit.qrf@1``'s identity."""

#: How far ``fit.qrf@1`` numbers may move between machines. The forest stack
#: promises no cross-platform bit stability (charter H1 records the claim as
#: ``tolerance_bound``); this bound is provisional until measured on the H1
#: fixture across arm64 and x86_64 (amendment 13 follow-up), and parity in
#: the locked environment is still asserted byte for byte.
FIT_QRF_TOLERANCE = Tolerance(rtol=1e-6)


QRF_EXECUTOR_SEED_HIGH = 2**31 - 1
"""Exclusive upper bound for the one seed drawn from ``KernelContext.rng``."""
Expand Down Expand Up @@ -114,6 +122,7 @@ def __init__(self, seed_source: SeedSource = SeedSource.EXECUTOR) -> None:
numeric=Numeric.TOLERANCE_BOUND,
seed_source=seed_source,
dependencies=FIT_QRF_DEPENDENCIES,
tolerance=FIT_QRF_TOLERANCE,
)

def implementation_hash(self) -> str:
Expand Down
2 changes: 2 additions & 0 deletions packages/microcosm-fit/tests/test_kernels.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@
Owned,
SeedSource,
Slice,
Tolerance,
source_hash,
)

Expand Down Expand Up @@ -233,6 +234,7 @@ def test_capabilities_protocol_and_wrapped_source_hash() -> None:
numeric=Numeric.TOLERANCE_BOUND,
seed_source=SeedSource.PARAM,
dependencies=FIT_QRF_DEPENDENCIES,
tolerance=Tolerance(rtol=1e-6),
)
assert QRF_EXECUTOR_KERNEL.capabilities.seed_source is SeedSource.EXECUTOR
assert QRF_PARAM_KERNEL.implementation_hash() == source_hash(
Expand Down
4 changes: 4 additions & 0 deletions packages/microcosm-graph/src/microcosm/graph/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
DTYPES,
GATE_OUTCOMES,
MASS_POLICIES,
PARTITION_DTYPES,
ROWS_ALL,
WEIGHT_KINDS,
CompiledGraph,
Expand Down Expand Up @@ -46,6 +47,7 @@
KernelRole,
Numeric,
SeedSource,
Tolerance,
source_hash,
)

Expand All @@ -54,6 +56,7 @@
"DTYPES",
"GATE_OUTCOMES",
"MASS_POLICIES",
"PARTITION_DTYPES",
"ROWS_ALL",
"WEIGHT_KINDS",
"Capabilities",
Expand Down Expand Up @@ -85,6 +88,7 @@
"RunManifest",
"SOURCE_CODECS",
"SeedSource",
"Tolerance",
"Slice",
"SourceCodec",
"SourceCodecRegistry",
Expand Down
77 changes: 75 additions & 2 deletions packages/microcosm-graph/src/microcosm/graph/decl.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,17 @@
known at compile time: a mask that is not ``bool`` or ``boolean`` is a
compile error (charter D4). Nulls inside a nullable mask are a run-time
rejection by the executor.
- An ``EXPAND`` node copies rows: every new row names the base row it
copies, so lineage is total. A node that declares ``entrants=True`` may
also add rows that copy nothing (births not patterned on a parent,
immigrant cohorts); the kernel materializes every carried column for
such a row, the executor records them as entrants rather than copies,
and the node cannot claim to conserve mass (amendment 11).
- Mass is accounted as weighted person mass per stratum. A graph may name
a partition column (:attr:`Graph.mass_partition`, e.g. a period on a
person-period population); the ledger then reports per stratum within
each partition and ``conserve`` holds within each partition, so a row
contributes mass only to the partitions it exists in (amendment 12).

This file is a frozen interface (see ``docs/graph-acceptance.md``).
"""
Expand All @@ -42,6 +53,7 @@
"GATE_OUTCOMES",
"MASK_DTYPES",
"MASS_POLICIES",
"PARTITION_DTYPES",
"ROWS_ALL",
"WEIGHT_KINDS",
"CompiledGraph",
Expand Down Expand Up @@ -82,6 +94,9 @@
#: Mass policies a weight transition or structural node may declare.
MASS_POLICIES = frozenset({"conserve", "free", "declared"})

#: The dtypes a mass-partition column may have.
PARTITION_DTYPES = frozenset({"int32", "int64", "string"})

#: The closed set of gate outcomes (charter F4). ``unreached`` is also the
#: outcome of a release whose required human decisions are absent.
GATE_OUTCOMES = ("pass", "fail", "evidence_absent", "not_applicable", "unreached")
Expand Down Expand Up @@ -269,6 +284,11 @@ class Node:
sources: Names of :class:`SourceRef` entries this node reads.
weights: A declared weight-kind transition, if any.
mass: Mass policy for structural nodes that change rows or weights.
entrants: ``EXPAND`` nodes only: the kernel may add rows that copy
no base row. Such a row has null lineage, the kernel supplies
every carried column for it, and the executor records it as an
entrant. Entrants add mass, so the node's mass policy cannot be
``conserve``.
description: Descriptive; never hashed.
citation: Descriptive; never hashed.
"""
Expand All @@ -284,6 +304,7 @@ class Node:
sources: tuple[str, ...] = ()
weights: WeightTransition | None = None
mass: str = "conserve"
entrants: bool = False
description: str = ""
citation: str = ""

Expand Down Expand Up @@ -333,6 +354,17 @@ def __post_init__(self) -> None:
)
elif self.base is not None:
raise GraphError(f"Node {self.id!r}: only structural nodes have a base.")
if not isinstance(self.entrants, bool):
raise GraphError(f"Node {self.id!r}: entrants must be a boolean.")
if self.entrants and self.structural is not StructuralDelta.EXPAND:
raise GraphError(
f"Node {self.id!r}: only an EXPAND node may admit entrants."
)
if self.entrants and self.mass == "conserve":
raise GraphError(
f"Node {self.id!r}: entrants add mass, so an entrant-admitting "
"node cannot declare mass='conserve'."
)
if self.weights is not None and self.structural is not StructuralDelta.REWEIGHT:
raise GraphError(
f"Node {self.id!r}: a weight transition changes the population "
Expand Down Expand Up @@ -386,18 +418,40 @@ class Graph:
into node keys (a node's identity is its computation).
sources: External inputs by name.
nodes: Every node. Declaration order carries no meaning.
mass_partition: ``(entity, column)`` of a column that partitions
mass accounting, or ``None``. When set, every ``CREATE`` node
declares the column with a dtype in :data:`PARTITION_DTYPES`,
the executor's ledger reports per stratum within each partition
value, and ``conserve`` holds within each partition. Normative:
it enters the key of every structural node.
"""

country: str
sources: tuple[SourceRef, ...]
nodes: tuple[Node, ...]
mass_partition: tuple[str, str] | None = None

def __post_init__(self) -> None:
_nonempty("Graph.country", self.country)
if len({s.name for s in self.sources}) != len(self.sources):
raise GraphError("Graph repeats a source name.")
if len({n.id for n in self.nodes}) != len(self.nodes):
raise GraphError("Graph repeats a node id.")
if self.mass_partition is not None:
if (
not isinstance(self.mass_partition, tuple)
or len(self.mass_partition) != 2
or not all(isinstance(part, str) for part in self.mass_partition)
):
raise GraphError(
"Graph.mass_partition must be an (entity, column) pair of strings."
)
_nonempty("Graph.mass_partition entity", self.mass_partition[0])
_nonempty("Graph.mass_partition column", self.mass_partition[1])

def normative(self) -> dict[str, object]:
"""The graph-level facts that enter every structural node's key."""
return {"mass_partition": self.mass_partition}

def node(self, node_id: str) -> Node:
for node in self.nodes:
Expand Down Expand Up @@ -436,8 +490,9 @@ def compile_graph(graph: Graph) -> CompiledGraph:
GraphError: A cell with two owners or none (ownership is total and
exclusive), an unknown source or population, a structural node
whose base is not structural, a row mask whose declared dtype is
not boolean, a cycle, or a graph with several structural nodes
and a node that omits ``population``.
not boolean, a mass-partition column that a ``CREATE`` node does
not declare with a partition dtype, a cycle, or a graph with
several structural nodes and a node that omits ``population``.
"""

by_id = {node.id: node for node in graph.nodes}
Expand Down Expand Up @@ -494,6 +549,24 @@ def compile_graph(graph: Graph) -> CompiledGraph:
owners[key] = node.id
dtypes[key] = owned.dtype

if graph.mass_partition is not None:
entity, column = graph.mass_partition
for node in structural:
if node.structural is not StructuralDelta.CREATE:
continue
dtype = dtypes.get((node.id, entity, column))
if dtype is None:
raise GraphError(
f"Graph.mass_partition names {entity}.{column}, which CREATE "
f"node {node.id!r} does not declare; partitions must exist "
"from the first version."
)
if dtype not in PARTITION_DTYPES:
raise GraphError(
f"Graph.mass_partition {entity}.{column} is declared {dtype!r}; "
f"a partition column must be one of {sorted(PARTITION_DTYPES)}."
)

def declared_dtype(version: str, entity: str, column: str) -> str | None:
"""The owner-declared dtype of a column as visible in ``version``."""
while True:
Expand Down
Loading
Loading