diff --git a/changelog.d/axiom-concept-coverage.added.md b/changelog.d/axiom-concept-coverage.added.md new file mode 100644 index 000000000..b732d2e16 --- /dev/null +++ b/changelog.d/axiom-concept-coverage.added.md @@ -0,0 +1 @@ +Add an optional real-Axiom input inventory and a shared, closed concept-coverage diagnostic that preserves unknown metadata, requires explicit evidence bindings, and cannot certify a population schema. diff --git a/docs/axiom-concept-coverage.md b/docs/axiom-concept-coverage.md new file mode 100644 index 000000000..a924f6de2 --- /dev/null +++ b/docs/axiom-concept-coverage.md @@ -0,0 +1,117 @@ +# Axiom input concept-coverage diagnostic + +This is a **diagnostic, not a population schema, calibration plan, or +certification gate**. It makes unavailable contracts visible before a population +build tries to use them. It introduces no country-specific Python operator and +does not change Chronicle source facts. + +The generic `InputInventoryProvider` protocol is optional; it does not expand the +required `RulesEngine` protocol. `AxiomEngine.input_inventory()` compiles a fresh +program through the real Axiom dense runtime and records root-input names per +mapped entity, canonical request addresses, and accepted aliases. It excludes +derived outputs. A relation-bearing module fails closed because this adapter +does not yet wire related-entity batches. Its scope is the mapped root entities +and all versions in the module, not every Belgian policy or a selected output's +minimal dependency closure. + +`entity` records the operational frame table; `engine_entity` records its dense +root. These are not evidence for a source publisher's statistical entity or +universe. Input `dtype`, `unit`, `period`, `definition`, semantic `concept_id`, and +`required` remain null where the runtime does not supply them. In particular, +being a root input does not establish that an input is mandatory in every case. +Some runtime inputs are configuration or externally supplied derived quantities, +not observed microdata: the Belgian pilot includes communal/agglomeration tax +rates, a supplied tax amount, and a tax-base flag. This inventory does not decide +which fields to store, derive, impute, or supply as configuration. +The complete typed-input contract remains +[axiom-rules-engine#62](https://github.com/TheAxiomFoundation/axiom-rules-engine/issues/62). + +## Run it + +Install the real Axiom Python wrapper and native dense extension into the local +environment, then pass explicit absolute canonical `rulespec-` roots: + +```bash +uv run --no-sync python tools/inventory_axiom_concepts.py \ + --module /path/to/rulespec-be/be/statutes/income_tax/individual/pilot_worker_oracle_pipeline.yaml \ + --rulespec-root /path/to/rulespec-be \ + --group-entity household +``` + +The command prints a closed v1 JSON manifest. It does not open a microdata file, +materialize taxes, infer metadata from spelling, or create semantic matches by +column name. Missing Axiom software is an error, never a fallback evaluator. + +## Evidence and dataset assessment + +`microcosm.build.concept_coverage.build_concept_coverage` also accepts an optional +Frame, explicit `column_bindings`, and consumer-authored `fact_bindings`. +Its string-only manifest refuses non-string Frame column labels rather than +coercing distinct labels into apparent matches. + +- No Frame: `not_supplied`; every input column status is `unassessed`. +- Frame but no explicit binding: still `unassessed`, even if names match. +- Explicit binding plus Frame: `present` or `absent` according to column names. + Values, distributions, missingness, units, and observed/imputed origin are not + assessed. The schema digest fingerprints column names, **not dataset content**. +- Extra columns remain permitted. Demography, wealth, predictors, future reforms, + and presently unencoded concepts need not be current-law inputs to belong in + the population. + +Every binding pins both the Frame entity and the native engine entity, along +with the runtime slot and canonical address, plus the inventory's +engine/module/root fingerprints. A binding cannot move between native root +entities even when files and request addresses are unchanged. Stale pins fail; non-canonical +aliases must be replaced by the catalog's canonical address. Conflicting target +concepts on one runtime slot fail instead of merging. + +Every fact binding independently records source concept, fact ID, source artifact +digest/vintage, target concept/legal vintage, both statistical scopes, a declared +transformation, and evidence (URI, digest, locator, claim). The scopes include +statistic, entity, universe, unit, geography, period, stock/flow classification, +stock reference date, and income-year versus assessment-year accounting basis. +Entity and universe definitions require document pins for non-unresolved +assertions. An asserted exact match requires equal complete statistical scope +and an identity transformation. Names do not create bindings. + +The builder preserves `asserted_relationship` separately from +`effective_relationship`. Missing target semantics produce effective +`unresolved` with reason `target_semantics_unavailable`, never a proxy invented +from ignorance. Even with future complete metadata, v0 emits unresolved with +`semantic_equivalence_unverified`: it does not fetch documents or adjudicate +legal/statistical equivalence. Authoring an exact/proxy assertion does not change +engine metadata, create a coverage score, or promote a certification flag. + +## Provenance and limits + +The manifest fingerprints the entry module, every YAML file plus toolchain pin +in each explicit RuleSpec root, actual imported wrapper/native files, adapter, +and diagnostic builder. Relative paths and bytes determine tree fingerprints; +relocating an identical root does not change identity. Compilation uses a fresh +adapter, and changes between before/after fingerprints abort discovery. Keep +the source checkouts quiescent during the run, as the Axiom loader requires. + +Per-entity discovery records complete enumeration and its runtime input count, +or `no_derived_program` with a null count. The latter is not successful zero-input +enumeration. Missing discovery, unsupported relations, failed compilation, or +inconsistent counts refuse an artifact. The Belgian pilot enumerates seven +Person inputs; it has no Household program. `blocking_gaps` exposes unavailable +metadata, unassessed dataset columns, and unresolved evidence without claiming +that this module is all of Belgian law. + +Inputs sort by canonical address, entity, and slot. The content hash uses +`microcosm-json-v1`: sorted JSON object keys, UTF-8, no NaN, compact separators, +and exclusion of `content_sha256` itself. Runtime versions and platform are +informational context. This diagnostic is unsigned; do not use a corpus release +key to sign it. + +These are reproducibility fingerprints, not signed source authenticity or +release certification. They neither establish the native binary's source-build +attestation nor replace the corpus's signature chain. PR CI tests synthetic +contract fixtures; it does not certify a population or publish any artifact. + +Belgium is the first inventory example, not the first proven cross-constellation +concordance. The [pilot snapshot](evidence/be-concept-coverage/README.md) records +the exact runtime/source pins and the separate scoped target-inventory audit. +No evidence bindings or dataset are supplied in that snapshot. Linking concepts +exposes missing evidence; it does not manufacture it. diff --git a/docs/evidence/be-concept-coverage/README.md b/docs/evidence/be-concept-coverage/README.md new file mode 100644 index 000000000..1fb861e20 --- /dev/null +++ b/docs/evidence/be-concept-coverage/README.md @@ -0,0 +1,66 @@ +# Belgian pilot input inventory snapshot + +Generated on 2026-08-29 by the shared diagnostic tool, using the real Axiom dense +compiler. [pilot-inputs.json](pilot-inputs.json) contains seven Person inputs, +their runtime-provided canonical addresses, and six unknown metadata fields per +input. It supplies **no dataset and no evidence bindings**. Household compilation +reports `no_derived_program` and a null count, not successful zero enumeration. +It is not a complete Belgian-law inventory or a certified population schema. + +The two identical runs produced content digest +`8b6c379afbf21e590c259722207ea07e41043be6ef335b6f77fed9a49b553086`. +The manifest fingerprints the actual adapter/builder files, installed engine +wrapper/native package, RuleSpec entry module, and canonical root YAML/toolchain. +The producer-code digests identify the files used for this dated snapshot, not +an attestation of any published release. + +## Source and build pins + +- Microcosm base: `ba73e2f43c6f1dcb3533faeab26ecc18260bd712`. +- Axiom source: `bb4b5684870547756078a62f1866a77c5b56f7f3`. +- RuleSpec-BE source: `b105e2b3a3086ddd2de447d58a9b951346870dd1`. +- Entry module: `be/statutes/income_tax/individual/pilot_worker_oracle_pipeline.yaml`. +- Native wheel: `axiom_rules_engine_dense-0.1.0-cp314-cp314-macosx_11_0_arm64.whl`, + SHA-256 `7928e395942ae071054197891a656f4744864e57c07554a9d0bf7d6635fe0f90`. +- Effective native Cargo lock SHA-256: + `e40add4c7ddeb39fa0005f379abc438bf9d2d927ac4570e8f6bf45d671a9e1b2`. + +The pinned source's extension lock was stale: its local path dependency named +engine version 0.1.0 while the source crate declared 0.2.2. The initial `--locked` +build refused. `maturin build --release --offline` changed that one local-crate +version entry; no external dependency version changed. The wrapper/native +distribution labels remain 0.1.0, so they must not substitute for the file +fingerprints. The manifest leaves the unexposed runtime core version null. + +Build/runtime outputs live outside tracked source. This snapshot is unsigned +diagnostic evidence, not an attested binary build or signed-corpus release. + +## Reproduction + +Install the pinned wrapper and built native wheel into a Python 3.14 environment +containing these Microcosm sources. The exact command surface is: + +```bash +uv run --no-sync python tools/inventory_axiom_concepts.py \ + --module /absolute/path/rulespec-be/be/statutes/income_tax/individual/pilot_worker_oracle_pipeline.yaml \ + --rulespec-root /absolute/path/rulespec-be \ + --group-entity household +``` + +The Work Bonus input's canonical owner is its imported `work_bonus` module, not +the pilot entry module. The other six addresses belong to the pilot module. +Those addresses came from the runtime catalog, not string construction. Runtime +inputs include configuration rates and supplied derived quantities as well as +remuneration; this diagnostic does not require every input to be observed data. + +## Separate target-inventory observation + +The local pilot artifact `microcosm_be_v051_chronicle_targets.json`, SHA-256 +`250051e7c3e50a4aee5181857c72d78f4ec38c365ff7dd1fb6c00ae6def7c227`, +declares 956 targets and 91 validations. The session's scoped inventory audit +found no wealth-stock targets. A reproducible name-screen returned no target +names containing `wealth`, `net_worth`, `asset`, `mortgage`, `debt`, or `hfcs`. +This is an audit of that specific artifact, not a classification engine or a +claim that Belgian publishers lack wealth statistics. The input diagnostic does +not ingest or republish that target file, and no source-to-input equivalence is +asserted here. Income and saving flows do not establish wealth-stock coverage. diff --git a/docs/evidence/be-concept-coverage/pilot-inputs.json b/docs/evidence/be-concept-coverage/pilot-inputs.json new file mode 100644 index 000000000..76cfc333a --- /dev/null +++ b/docs/evidence/be-concept-coverage/pilot-inputs.json @@ -0,0 +1,288 @@ +{ + "artifact_kind": "concept_coverage_diagnostic", + "blocking_gaps": [ + "dataset_column_coverage_unassessed", + "dataset_not_supplied", + "input_metadata_unavailable", + "no_derived_program:household", + "population_schema_and_certification_not_assessed", + "semantic_evidence_unresolved" + ], + "canonicalization": "microcosm-json-v1", + "column_bindings": [], + "content_sha256": "8b6c379afbf21e590c259722207ea07e41043be6ef335b6f77fed9a49b553086", + "dataset": { + "assessment": "column_presence_only", + "columns": null, + "extra_columns": "permitted", + "presence_only": true, + "schema_sha256": null, + "status": "not_supplied" + }, + "entity_discovery": [ + { + "engine_entity": "Household", + "entity": "household", + "root_input_count": null, + "status": "no_derived_program" + }, + { + "engine_entity": "Person", + "entity": "person", + "root_input_count": 7, + "status": "complete" + } + ], + "fact_bindings": [], + "fingerprints": [ + { + "name": "microcosm.frame.adapters.axiom", + "role": "adapter", + "sha256": "f8413fb3765588ecb46c592c52f9728c3b2a0161e1d9b967c60ab60a33f1a849" + }, + { + "name": "microcosm.build.concept_coverage", + "role": "diagnostic_builder", + "sha256": "b819345144b69ab198b38f26ed79f8c5c745a21139d2780e9e6aeeca9da537b2" + }, + { + "name": "axiom_rules_engine_dense", + "role": "engine_native", + "sha256": "3b22567418f7faf9e82d04e0d1144c5f45a7fa0423663754df2a7646d04b3541" + }, + { + "name": "axiom_rules_engine", + "role": "engine_wrapper", + "sha256": "b5a7e21e0b62d372a917985600decb7faf3203ffd01f3ca22f5da9996faa7d1c" + }, + { + "name": "rulespec-be/be/statutes/income_tax/individual/pilot_worker_oracle_pipeline.yaml", + "role": "entry_module", + "sha256": "1388e420002ce0d4a3a51e45993407c4e8a9052b9c9fb8cadd7ebf75aa71e880" + }, + { + "name": "rulespec-be", + "role": "rulespec_root_yaml_and_toolchain", + "sha256": "7cef80a8007d7d6126f1335c26dc79e9d0430b82c2b8ee1a8db39bb5d8ff0754" + } + ], + "inputs": [ + { + "canonical_request_name": "be:regulations/social_security/workers/work_bonus#input.belgium_worker_work_bonus_supplied_reference_annual_remuneration", + "column_status": "unassessed", + "data_origin": "unassessed", + "engine_entity": "Person", + "entity": "person", + "metadata": { + "concept_id": null, + "definition": null, + "dtype": null, + "period": null, + "required": null, + "unit": null + }, + "metadata_gaps": [ + "concept_id", + "definition", + "dtype", + "period", + "required", + "unit" + ], + "name": "belgium_worker_work_bonus_supplied_reference_annual_remuneration", + "request_names": [ + "be:regulations/social_security/workers/work_bonus#input.belgium_worker_work_bonus_supplied_reference_annual_remuneration" + ], + "semantic_status": "unassessed" + }, + { + "canonical_request_name": "be:statutes/income_tax/individual/pilot_worker_oracle_pipeline#input.belgium_pit_agglomeration_additional_tax_rate", + "column_status": "unassessed", + "data_origin": "unassessed", + "engine_entity": "Person", + "entity": "person", + "metadata": { + "concept_id": null, + "definition": null, + "dtype": null, + "period": null, + "required": null, + "unit": null + }, + "metadata_gaps": [ + "concept_id", + "definition", + "dtype", + "period", + "required", + "unit" + ], + "name": "belgium_pit_agglomeration_additional_tax_rate", + "request_names": [ + "be:statutes/income_tax/individual/pilot_worker_oracle_pipeline#input.belgium_pit_agglomeration_additional_tax_rate" + ], + "semantic_status": "unassessed" + }, + { + "canonical_request_name": "be:statutes/income_tax/individual/pilot_worker_oracle_pipeline#input.belgium_pit_article_23_worker_remuneration", + "column_status": "unassessed", + "data_origin": "unassessed", + "engine_entity": "Person", + "entity": "person", + "metadata": { + "concept_id": null, + "definition": null, + "dtype": null, + "period": null, + "required": null, + "unit": null + }, + "metadata_gaps": [ + "concept_id", + "definition", + "dtype", + "period", + "required", + "unit" + ], + "name": "belgium_pit_article_23_worker_remuneration", + "request_names": [ + "be:statutes/income_tax/individual/pilot_worker_oracle_pipeline#input.belgium_pit_article_23_worker_remuneration" + ], + "semantic_status": "unassessed" + }, + { + "canonical_request_name": "be:statutes/income_tax/individual/pilot_worker_oracle_pipeline#input.belgium_pit_article_466_tax_share_on_nonprofessional_movable_income", + "column_status": "unassessed", + "data_origin": "unassessed", + "engine_entity": "Person", + "entity": "person", + "metadata": { + "concept_id": null, + "definition": null, + "dtype": null, + "period": null, + "required": null, + "unit": null + }, + "metadata_gaps": [ + "concept_id", + "definition", + "dtype", + "period", + "required", + "unit" + ], + "name": "belgium_pit_article_466_tax_share_on_nonprofessional_movable_income", + "request_names": [ + "be:statutes/income_tax/individual/pilot_worker_oracle_pipeline#input.belgium_pit_article_466_tax_share_on_nonprofessional_movable_income" + ], + "semantic_status": "unassessed" + }, + { + "canonical_request_name": "be:statutes/income_tax/individual/pilot_worker_oracle_pipeline#input.belgium_pit_article_466bis_hypothetical_total_tax_if_treaty_exempt_foreign_professional_income_were_belgian", + "column_status": "unassessed", + "data_origin": "unassessed", + "engine_entity": "Person", + "entity": "person", + "metadata": { + "concept_id": null, + "definition": null, + "dtype": null, + "period": null, + "required": null, + "unit": null + }, + "metadata_gaps": [ + "concept_id", + "definition", + "dtype", + "period", + "required", + "unit" + ], + "name": "belgium_pit_article_466bis_hypothetical_total_tax_if_treaty_exempt_foreign_professional_income_were_belgian", + "request_names": [ + "be:statutes/income_tax/individual/pilot_worker_oracle_pipeline#input.belgium_pit_article_466bis_hypothetical_total_tax_if_treaty_exempt_foreign_professional_income_were_belgian" + ], + "semantic_status": "unassessed" + }, + { + "canonical_request_name": "be:statutes/income_tax/individual/pilot_worker_oracle_pipeline#input.belgium_pit_article_466bis_treaty_exempt_foreign_professional_income_base_applies", + "column_status": "unassessed", + "data_origin": "unassessed", + "engine_entity": "Person", + "entity": "person", + "metadata": { + "concept_id": null, + "definition": null, + "dtype": null, + "period": null, + "required": null, + "unit": null + }, + "metadata_gaps": [ + "concept_id", + "definition", + "dtype", + "period", + "required", + "unit" + ], + "name": "belgium_pit_article_466bis_treaty_exempt_foreign_professional_income_base_applies", + "request_names": [ + "be:statutes/income_tax/individual/pilot_worker_oracle_pipeline#input.belgium_pit_article_466bis_treaty_exempt_foreign_professional_income_base_applies" + ], + "semantic_status": "unassessed" + }, + { + "canonical_request_name": "be:statutes/income_tax/individual/pilot_worker_oracle_pipeline#input.belgium_pit_communal_additional_tax_rate", + "column_status": "unassessed", + "data_origin": "unassessed", + "engine_entity": "Person", + "entity": "person", + "metadata": { + "concept_id": null, + "definition": null, + "dtype": null, + "period": null, + "required": null, + "unit": null + }, + "metadata_gaps": [ + "concept_id", + "definition", + "dtype", + "period", + "required", + "unit" + ], + "name": "belgium_pit_communal_additional_tax_rate", + "request_names": [ + "be:statutes/income_tax/individual/pilot_worker_oracle_pipeline#input.belgium_pit_communal_additional_tax_rate" + ], + "semantic_status": "unassessed" + } + ], + "mapped_entities": [ + "household", + "person" + ], + "readiness": { + "certified": false, + "population_schema_ready": false, + "reason": "diagnostic_only" + }, + "root_input_count": 7, + "runtime": { + "core_version": null, + "engine": "axiom", + "native_distribution_version": "0.1.0", + "numpy": "2.4.6", + "platform": "macOS-26.5.1-arm64-arm-64bit-Mach-O", + "python": "3.14.4", + "wrapper_distribution_version": "0.1.0" + }, + "schema_version": 1, + "scope": "mapped_entity_root_inputs_all_module_versions" +} diff --git a/packages/microcosm-build/src/microcosm/build/concept_coverage.py b/packages/microcosm-build/src/microcosm/build/concept_coverage.py new file mode 100644 index 000000000..5b580caeb --- /dev/null +++ b/packages/microcosm-build/src/microcosm/build/concept_coverage.py @@ -0,0 +1,596 @@ +"""Input concept-coverage diagnostics, never a population/certification gate. + +Executable input discovery is adapter-owned. Consumer-authored bindings preserve +publisher scope and evidence; matching names do not establish equivalence. This +v0 inventories missing contracts without filling them with guessed metadata. +""" + +from collections.abc import Mapping, Sequence +from copy import deepcopy +from dataclasses import asdict +from datetime import date +from hashlib import sha256 +from pathlib import Path +from typing import Any + +from jsonschema import Draft202012Validator, ValidationError + +from microcosm.build.spec_engine.canonical import CANONICALIZER_ID, sha256_json +from microcosm.frame import Frame +from microcosm.frame.rules import InputInventoryProvider + +_TEXT = {"type": "string", "pattern": r"\S"} +_MAYBE_TEXT = {"anyOf": [_TEXT, {"type": "null"}]} +_SHA256 = {"type": "string", "pattern": "^[0-9a-f]{64}$"} + + +def _object(properties: dict[str, Any]) -> dict[str, Any]: + return { + "type": "object", + "additionalProperties": False, + "required": list(properties), + "properties": properties, + } + + +def _array(items: dict[str, Any]) -> dict[str, Any]: + return {"type": "array", "items": items, "uniqueItems": True} + + +_KEY = _object({"entity": _TEXT, "name": _TEXT}) +_INPUT_KEY = _object( + { + "entity": _TEXT, + "engine_entity": _TEXT, + "name": _TEXT, + "canonical_request_name": _TEXT, + } +) +_FINGERPRINT = _object({"role": _TEXT, "name": _TEXT, "sha256": _SHA256}) +_DEFINITION = _object({"uri": _TEXT, "sha256": _SHA256, "locator": _TEXT}) +_SCOPE = _object( + { + **{ + key: _MAYBE_TEXT + for key in ( + "statistic", + "entity", + "universe", + "unit", + "geography", + "period", + ) + }, + "measurement_kind": {"enum": ["stock", "flow", "not_applicable", None]}, + "reference_instant": _MAYBE_TEXT, + "accounting_basis": { + "enum": [ + "income_year", + "assessment_year", + "calendar_year", + "tax_year", + "instant", + "not_applicable", + None, + ] + }, + "entity_definition": {"anyOf": [_DEFINITION, {"type": "null"}]}, + "universe_definition": {"anyOf": [_DEFINITION, {"type": "null"}]}, + } +) +_EVIDENCE = _object({"uri": _TEXT, "sha256": _SHA256, "locator": _TEXT, "claim": _TEXT}) +_COLUMN_BINDING = _object( + {"input": _INPUT_KEY, "artifact_fingerprints": _array(_FINGERPRINT), "column": _KEY} +) +_FACT_BINDING = _object( + { + "input": _INPUT_KEY, + "artifact_fingerprints": _array(_FINGERPRINT), + "asserted_relationship": {"enum": ["exact", "proxy", "unresolved"]}, + "source": _object( + { + "fact_id": _TEXT, + "concept_id": _TEXT, + "vintage": _TEXT, + "artifact_sha256": _SHA256, + "scope": _SCOPE, + } + ), + "target": _object( + { + "concept_id": _MAYBE_TEXT, + "legal_vintage": _TEXT, + "scope": _SCOPE, + } + ), + "transformation": _object( + { + "kind": {"enum": ["identity", "declared_conversion", "unresolved"]}, + "description": _TEXT, + } + ), + "evidence": _array(_EVIDENCE), + } +) +_CLASSIFIED_BINDING = _object( + { + **_FACT_BINDING["properties"], + "effective_relationship": {"const": "unresolved"}, + "classification_reason": { + "enum": [ + "asserted_unresolved", + "target_semantics_unavailable", + "semantic_equivalence_unverified", + ] + }, + } +) +_METADATA = _object( + { + **{ + key: _MAYBE_TEXT + for key in ("dtype", "unit", "period", "definition", "concept_id") + }, + "required": {"type": ["boolean", "null"]}, + } +) +MANIFEST_SCHEMA = _object( + { + "schema_version": {"const": 1}, + "artifact_kind": {"const": "concept_coverage_diagnostic"}, + "canonicalization": {"const": CANONICALIZER_ID}, + "scope": {"const": "mapped_entity_root_inputs_all_module_versions"}, + "fingerprints": _array(_FINGERPRINT), + "mapped_entities": _array(_TEXT), + "entity_discovery": _array( + _object( + { + "entity": _TEXT, + "engine_entity": _TEXT, + "status": {"enum": ["complete", "no_derived_program"]}, + "root_input_count": {"type": ["integer", "null"], "minimum": 0}, + } + ) + ), + "root_input_count": {"type": "integer", "minimum": 0}, + "runtime": _object( + { + key: _MAYBE_TEXT + for key in ( + "engine", + "wrapper_distribution_version", + "native_distribution_version", + "core_version", + "python", + "platform", + "numpy", + ) + } + ), + "dataset": _object( + { + "status": {"enum": ["not_supplied", "schema_inspected"]}, + "assessment": {"const": "column_presence_only"}, + "presence_only": {"const": True}, + "schema_sha256": {"anyOf": [_SHA256, {"type": "null"}]}, + "columns": {"anyOf": [_array(_KEY), {"type": "null"}]}, + "extra_columns": {"const": "permitted"}, + } + ), + "inputs": _array( + _object( + { + "name": _TEXT, + "entity": _TEXT, + "engine_entity": _TEXT, + "canonical_request_name": _MAYBE_TEXT, + "request_names": _array(_TEXT), + "metadata": _METADATA, + "metadata_gaps": _array(_TEXT), + "column_status": {"enum": ["unassessed", "present", "absent"]}, + "data_origin": {"const": "unassessed"}, + "semantic_status": { + "enum": ["unassessed", "unverified_assertions"] + }, + } + ) + ), + "column_bindings": _array(_COLUMN_BINDING), + "fact_bindings": _array(_CLASSIFIED_BINDING), + "blocking_gaps": _array(_TEXT), + "readiness": _object( + { + "certified": {"const": False}, + "population_schema_ready": {"const": False}, + "reason": {"const": "diagnostic_only"}, + } + ), + "content_sha256": _SHA256, + } +) + + +def _validate_schema(schema: dict[str, Any], payload: object) -> None: + try: + Draft202012Validator(schema).validate(payload) + except ValidationError as exc: + if exc.validator == "uniqueItems": + raise ValueError( + "Invalid concept-coverage manifest: duplicate entries" + ) from exc + raise ValueError(f"Invalid concept-coverage manifest: {exc.message}") from exc + + +def _key(item: Mapping[str, Any]) -> tuple[str, str]: + return item["entity"], item["name"] + + +def _input_sort_key(item: Mapping[str, Any]) -> tuple[str, str, str]: + return item["canonical_request_name"] or "", item["entity"], item["name"] + + +def _classification(binding: dict, item: dict) -> tuple[str, str]: + # v0 checks assertion consistency, not legal/statistical equivalence. Even + # full future metadata would still need a semantic adjudication mechanism. + if binding["asserted_relationship"] == "unresolved": + return "unresolved", "asserted_unresolved" + if item["metadata_gaps"]: + return "unresolved", "target_semantics_unavailable" + return "unresolved", "semantic_equivalence_unverified" + + +def _blocking_gaps(report: dict) -> list[str]: + gaps = {"population_schema_and_certification_not_assessed"} + if any(item["metadata_gaps"] for item in report["inputs"]): + gaps.add("input_metadata_unavailable") + if any(item["canonical_request_name"] is None for item in report["inputs"]): + gaps.add("canonical_input_addresses_unavailable") + if report["dataset"]["status"] == "not_supplied": + gaps.add("dataset_not_supplied") + if any(item["column_status"] == "unassessed" for item in report["inputs"]): + gaps.add("dataset_column_coverage_unassessed") + if any(item["column_status"] == "absent" for item in report["inputs"]): + gaps.add("explicitly_bound_dataset_columns_absent") + if report["inputs"]: + gaps.add("semantic_evidence_unresolved") + for item in report["entity_discovery"]: + if item["status"] == "no_derived_program": + gaps.add(f"no_derived_program:{item['entity']}") + return sorted(gaps) + + +def _unique_index(items: Sequence[Mapping[str, Any]], label: str) -> dict: + result = {} + for item in items: + key = _key(item) + if key in result: + raise ValueError(f"duplicate {label}: {key}") + result[key] = item + return result + + +def _column_status(key: tuple[str, str], dataset: dict, bindings: dict) -> str: + if dataset["status"] == "not_supplied" or key not in bindings: + return "unassessed" + columns = {_key(item) for item in dataset["columns"]} + return "present" if _key(bindings[key]["column"]) in columns else "absent" + + +def build_concept_coverage( + engine: InputInventoryProvider, + *, + dataset: Frame | None = None, + column_bindings: Sequence[Mapping[str, Any]] = (), + fact_bindings: Sequence[Mapping[str, Any]] = (), +) -> dict[str, Any]: + """Discover inputs and make missing evidence explicit, without evaluating tax. + + A supplied Frame is inspected only for column presence, using explicit + input-to-column bindings. Its values, dtypes and weights are not inspected; + column presence does not identify observed versus imputed origin. No data + supplied, or no explicit binding, means unassessed, never measured absent. + Schema digests identify column names only, not the microdata artifact. + + Asserted relationships are preserved separately from machine-effective + classification. Missing target semantics means unresolved, not proxy. + v0 never adjudicates semantic equivalence, so it cannot emit effective exact + or proxy merely because an assertion has well-formed evidence fields. + Nothing here edits Chronicle or certifies a build. + """ + if not callable(getattr(engine, "input_inventory", None)): + raise TypeError("Rules adapter does not support input discovery.") + if dataset is not None: + for entity in dataset.entities: + if any( + not isinstance(column, str) for column in dataset.table(entity).columns + ): + raise ValueError( + f"Concept coverage cannot represent non-string column labels on {entity!r}." + ) + inventory = engine.input_inventory() + columns = ( + sorted( + [ + {"entity": entity, "name": column} + for entity in dataset.entities + for column in dataset.table(entity).columns + ], + key=_key, + ) + if dataset is not None + else None + ) + dataset_report = { + "status": "schema_inspected" if dataset is not None else "not_supplied", + "assessment": "column_presence_only", + "presence_only": True, + "schema_sha256": sha256_json(columns) if columns is not None else None, + "columns": columns, + "extra_columns": "permitted", + } + column_records = deepcopy(list(column_bindings)) + fact_records = deepcopy(list(fact_bindings)) + _validate_schema(_array(_COLUMN_BINDING), column_records) + _validate_schema(_array(_FACT_BINDING), fact_records) + for binding in column_records + fact_records: + binding["artifact_fingerprints"].sort( + key=lambda item: (item["role"], item["name"]) + ) + column_index = _unique_index( + [{**item["input"], "column": item["column"]} for item in column_records], + "column binding", + ) + asserted = {_key(item["input"]) for item in fact_records} + inputs = [] + for record in inventory.inputs: + item = asdict(record) + item["request_names"] = sorted(item["request_names"]) + metadata = {key: item.pop(key) for key in _METADATA["properties"]} + item.update( + metadata=metadata, + metadata_gaps=sorted( + key for key, value in metadata.items() if value is None + ), + column_status=_column_status(_key(item), dataset_report, column_index), + data_origin="unassessed", + semantic_status=( + "unverified_assertions" if _key(item) in asserted else "unassessed" + ), + ) + inputs.append(item) + report = { + "schema_version": 1, + "artifact_kind": "concept_coverage_diagnostic", + "canonicalization": CANONICALIZER_ID, + "scope": "mapped_entity_root_inputs_all_module_versions", + "fingerprints": sorted( + [dict(item) for item in inventory.fingerprints] + + [ + { + "role": "diagnostic_builder", + "name": "microcosm.build.concept_coverage", + "sha256": sha256(Path(__file__).read_bytes()).hexdigest(), + } + ], + key=lambda item: (item["role"], item["name"]), + ), + "mapped_entities": sorted(inventory.mapped_entities), + "entity_discovery": sorted( + [dict(item) for item in inventory.entity_discovery], + key=lambda item: item["entity"], + ), + "root_input_count": len(inputs), + "runtime": dict(inventory.runtime), + "dataset": dataset_report, + "inputs": sorted(inputs, key=_input_sort_key), + "column_bindings": sorted( + column_records, key=lambda item: _input_sort_key(item["input"]) + ), + "fact_bindings": sorted( + fact_records, + key=lambda item: ( + *_input_sort_key(item["input"]), + item["source"]["fact_id"], + ), + ), + "readiness": { + "certified": False, + "population_schema_ready": False, + "reason": "diagnostic_only", + }, + } + input_index = _unique_index(inputs, "input") + for binding in report["fact_bindings"]: + key = _key(binding["input"]) + if key not in input_index: + raise ValueError(f"binding refers to unknown input {key}") + effective, reason = _classification(binding, input_index[key]) + binding.update(effective_relationship=effective, classification_reason=reason) + report["blocking_gaps"] = _blocking_gaps(report) + report["content_sha256"] = sha256_json(report) + validate_concept_coverage(report) + return report + + +def validate_concept_coverage(payload: Mapping[str, Any]) -> None: + """Validate closed JSON shape, digest, and fail-closed cross-field semantics.""" + report = dict(payload) + _validate_schema(MANIFEST_SCHEMA, report) + content = {key: value for key, value in report.items() if key != "content_sha256"} + if sha256_json(content) != report["content_sha256"]: + raise ValueError("concept-coverage content digest mismatch") + inputs = _unique_index(report["inputs"], "input") + if report["inputs"] != sorted(report["inputs"], key=_input_sort_key): + raise ValueError("inputs are not sorted by canonical address") + if report["root_input_count"] != len(inputs): + raise ValueError("root input count disagrees with inventory") + _validate_discovery(report, inputs) + column_index = _unique_index( + [ + {**item["input"], "column": item["column"]} + for item in report["column_bindings"] + ], + "column binding", + ) + dataset = report["dataset"] + if dataset["status"] == "not_supplied": + if dataset["columns"] is not None or dataset["schema_sha256"] is not None: + raise ValueError("An unsupplied dataset cannot carry a measured schema.") + elif ( + dataset["columns"] is None + or sha256_json(dataset["columns"]) != dataset["schema_sha256"] + ): + raise ValueError("dataset schema digest mismatch") + for binding in report["column_bindings"] + report["fact_bindings"]: + key = _key(binding["input"]) + if key not in inputs: + raise ValueError( + f"binding refers to unknown input {_key(binding['input'])}" + ) + if binding["input"]["engine_entity"] != inputs[key]["engine_entity"]: + raise ValueError( + "binding engine entity disagrees with the compiled input context" + ) + if ( + binding["input"]["canonical_request_name"] + != inputs[key]["canonical_request_name"] + ): + raise ValueError( + "binding uses a non-canonical alias or unknown request address" + ) + expected_pins = [ + item + for item in report["fingerprints"] + if item["role"] != "diagnostic_builder" + ] + if not expected_pins or binding["artifact_fingerprints"] != expected_pins: + raise ValueError("binding artifact fingerprint mismatch") + asserted = {_key(item["input"]) for item in report["fact_bindings"]} + for key, item in inputs.items(): + if item["entity"] not in report["mapped_entities"]: + raise ValueError(f"Input {key} is outside mapped entity scope.") + if ( + item["canonical_request_name"] is not None + and item["canonical_request_name"] not in item["request_names"] + ): + raise ValueError("canonical request name is absent from accepted names") + gaps = sorted(name for name, value in item["metadata"].items() if value is None) + if gaps != item["metadata_gaps"]: + raise ValueError("metadata gap report disagrees with unknown fields") + if item["column_status"] != _column_status(key, dataset, column_index): + raise ValueError("column status disagrees with explicit binding and schema") + expected = "unverified_assertions" if key in asserted else "unassessed" + if item["semantic_status"] != expected: + raise ValueError("semantic status disagrees with recorded assertions") + seen_facts = set() + target_concepts = {} + for binding in report["fact_bindings"]: + key = _key(binding["input"]) + fact_key = (*key, binding["source"]["fact_id"]) + if fact_key in seen_facts: + raise ValueError(f"duplicate fact binding: {fact_key}") + seen_facts.add(fact_key) + concept = binding["target"]["concept_id"] + if concept is not None: + if key in target_concepts and target_concepts[key] != concept: + raise ValueError( + "conflicting target concept identities for one runtime slot" + ) + target_concepts[key] = concept + relationship = binding["asserted_relationship"] + if relationship != "unresolved" and not binding["evidence"]: + raise ValueError(f"{relationship} binding needs pinned evidence") + effective, reason = _classification(binding, inputs[key]) + if ( + binding["effective_relationship"] != effective + or binding["classification_reason"] != reason + ): + raise ValueError( + "effective relationship disagrees with available semantics" + ) + for side in ("source", "target"): + _validate_scope(binding[side]["scope"], relationship) + if relationship != "exact": + continue + source_scope = binding["source"]["scope"] + target = binding["target"] + target_scope = target["scope"] + if any( + source_scope[field] != target_scope[field] + for field in source_scope + if field not in {"entity_definition", "universe_definition"} + ): + raise ValueError("exact binding needs complete equal statistical scope") + if binding["transformation"]["kind"] != "identity": + raise ValueError( + "asserted exact binding requires an identity transformation" + ) + if report["blocking_gaps"] != _blocking_gaps(report): + raise ValueError("blocking gaps disagree with observed capability/evidence") + + +def _validate_discovery(report: dict, inputs: dict) -> None: + discovery = {item["entity"]: item for item in report["entity_discovery"]} + if len(discovery) != len(report["entity_discovery"]) or set(discovery) != set( + report["mapped_entities"] + ): + raise ValueError( + "per-entity discovery must cover each mapped entity exactly once" + ) + if not any(item["status"] == "complete" for item in discovery.values()): + raise ValueError("No mapped entity was successfully enumerated.") + for entity, item in discovery.items(): + records = [value for value in inputs.values() if value["entity"] == entity] + if item["status"] == "no_derived_program": + if item["root_input_count"] is not None or records: + raise ValueError( + "An uncompiled entity cannot claim enumerated inputs or zero count." + ) + elif item["root_input_count"] is None or item["root_input_count"] != len( + records + ): + raise ValueError("runtime root input count disagrees with entity inventory") + if any(value["engine_entity"] != item["engine_entity"] for value in records): + raise ValueError("runtime entity metadata disagrees with input entity") + canonical = [ + value["canonical_request_name"] + for value in records + if value["canonical_request_name"] is not None + ] + if len(set(canonical)) != len(canonical): + raise ValueError("multiple runtime slots share one canonical input address") + names = [(item["role"], item["name"]) for item in report["fingerprints"]] + if not any(role != "diagnostic_builder" for role, _ in names) or len(names) != len( + set(names) + ): + raise ValueError("missing or conflicting artifact fingerprints") + if report["fingerprints"] != sorted( + report["fingerprints"], key=lambda item: (item["role"], item["name"]) + ): + raise ValueError("artifact fingerprints are not canonically ordered") + + +def _validate_scope(scope: dict, relationship: str) -> None: + if scope["measurement_kind"] == "stock" and scope["reference_instant"] is None: + raise ValueError("stock scope requires a reference instant") + if scope["measurement_kind"] != "stock" and scope["reference_instant"] is not None: + raise ValueError("only a stock scope may carry a reference instant") + if scope["reference_instant"] is not None: + instant = scope["reference_instant"] + try: + if len(instant) != 10: + raise ValueError + date.fromisoformat(instant) + except ValueError as exc: + raise ValueError( + "stock reference instant must be an ISO calendar date" + ) from exc + if relationship == "unresolved": + return + for field in ("entity_definition", "universe_definition"): + if scope[field] is None: + raise ValueError(f"{relationship} scope requires a pinned {field}") + if relationship == "exact" and any( + value is None for key, value in scope.items() if key != "reference_instant" + ): + raise ValueError("asserted exact binding requires complete statistical scope") diff --git a/packages/microcosm-build/tests/fixtures/rulespec-zz/zz/policies/tests/shared_input.yaml b/packages/microcosm-build/tests/fixtures/rulespec-zz/zz/policies/tests/shared_input.yaml new file mode 100644 index 000000000..6251dd2f1 --- /dev/null +++ b/packages/microcosm-build/tests/fixtures/rulespec-zz/zz/policies/tests/shared_input.yaml @@ -0,0 +1,28 @@ +format: rulespec/v1 +module: + summary: Synthetic input-identity regression fixture; no tax or benefit computation. +units: + - name: EUR + kind: currency + minor_units: 2 +rules: + - name: person_echo + kind: derived + entity: Person + dtype: Money + unit: EUR + period: Year + source: synthetic review fixture + versions: + - effective_from: '2025-01-01' + formula: shared_input + - name: household_echo + kind: derived + entity: Household + dtype: Money + unit: EUR + period: Year + source: synthetic review fixture + versions: + - effective_from: '2025-01-01' + formula: shared_input diff --git a/packages/microcosm-build/tests/test_spec_concept_coverage.py b/packages/microcosm-build/tests/test_spec_concept_coverage.py new file mode 100644 index 000000000..c6c9a867f --- /dev/null +++ b/packages/microcosm-build/tests/test_spec_concept_coverage.py @@ -0,0 +1,497 @@ +"""A diagnostic cannot turn missing semantic evidence into population readiness.""" + +from copy import deepcopy +from dataclasses import replace +from pathlib import Path + +import numpy as np +import pandas as pd +import pytest + +from microcosm.build.concept_coverage import ( + build_concept_coverage, + validate_concept_coverage, +) +from microcosm.frame import EntitySchema, Frame, WeightKind, Weights +from microcosm.frame.adapters.axiom import AxiomEngine +from microcosm.frame.rules import EngineInput, InputInventory + +ADDRESS = "zz:statutes/example#input.earnings" +FINGERPRINTS = [ + {"role": "entry_module", "name": "example.yaml", "sha256": "a" * 64}, + { + "role": "rulespec_root_yaml_and_toolchain", + "name": "rulespec-zz", + "sha256": "b" * 64, + }, +] + + +class InventoryProvider: + """Protocol fixture only; no tax calculation or replacement Axiom evaluator.""" + + def input_inventory(self): + return InputInventory( + inputs=( + EngineInput( + name="earnings", + entity="person", + engine_entity="Person", + canonical_request_name=ADDRESS, + request_names=(ADDRESS, "zz:statutes/another#input.earnings"), + ), + ), + fingerprints=tuple(deepcopy(FINGERPRINTS)), + mapped_entities=("person", "household"), + entity_discovery=( + { + "entity": "person", + "engine_entity": "Person", + "status": "complete", + "root_input_count": 1, + }, + { + "entity": "household", + "engine_entity": "Household", + "status": "no_derived_program", + "root_input_count": None, + }, + ), + runtime={ + "engine": "protocol_fixture", + "wrapper_distribution_version": None, + "native_distribution_version": None, + "core_version": None, + "python": None, + "platform": None, + "numpy": None, + }, + ) + + +@pytest.fixture +def dataset(): + return Frame( + { + "person": pd.DataFrame( + { + "person_id": [1], + "person_household_id": [1], + "earnings": [100.0], + "wealth_for_future_reforms": [200.0], + } + ), + "household": pd.DataFrame({"household_id": [1]}), + }, + EntitySchema(group_entities=("household",)), + {"household": Weights(np.array([1.0]), WeightKind.DESIGN)}, + ) + + +def column_binding(name="earnings"): + return { + "input": { + "entity": "person", + "engine_entity": "Person", + "name": "earnings", + "canonical_request_name": ADDRESS, + }, + "artifact_fingerprints": deepcopy(FINGERPRINTS), + "column": {"entity": "person", "name": name}, + } + + +def fact_binding(relationship="proxy"): + scope = { + "statistic": "total", + "entity": "person", + "universe": "resident workers", + "unit": "EUR", + "geography": "BE", + "period": "income-year:2025", + "measurement_kind": "flow", + "reference_instant": None, + "accounting_basis": "income_year", + "entity_definition": { + "uri": "https://example.org/entities", + "sha256": "d" * 64, + "locator": "Person definition", + }, + "universe_definition": { + "uri": "https://example.org/universes", + "sha256": "e" * 64, + "locator": "Resident workers definition", + }, + } + return { + "input": { + "entity": "person", + "engine_entity": "Person", + "name": "earnings", + "canonical_request_name": ADDRESS, + }, + "artifact_fingerprints": deepcopy(FINGERPRINTS), + "asserted_relationship": relationship, + "source": { + "fact_id": "chronicle:example/fact", + "concept_id": "publisher:gross-earnings", + "vintage": "publication:2026-01-01", + "artifact_sha256": "b" * 64, + "scope": scope, + }, + "target": { + "concept_id": None, + "legal_vintage": "rulespec:revision-1", + "scope": deepcopy(scope), + }, + "transformation": { + "kind": "identity", + "description": "This is an authored identity claim, not verified equivalence.", + }, + "evidence": [ + { + "uri": "https://example.org/publisher/definition", + "sha256": "c" * 64, + "locator": "Table 1 definition", + "claim": "Publisher defines gross earnings for resident workers.", + } + ], + } + + +def test_missing_metadata_and_no_dataset_remain_unknown(): + report = build_concept_coverage(InventoryProvider()) + item = report["inputs"][0] + assert set(item["metadata"].values()) == {None} + assert item["column_status"] == "unassessed" + assert report["dataset"]["status"] == "not_supplied" + assert report["readiness"] == { + "certified": False, + "population_schema_ready": False, + "reason": "diagnostic_only", + } + assert report["fact_bindings"] == [] + validate_concept_coverage(report) + + +def test_matching_names_do_not_automatically_bind_columns(dataset): + report = build_concept_coverage(InventoryProvider(), dataset=dataset) + assert report["dataset"]["status"] == "schema_inspected" + assert report["inputs"][0]["column_status"] == "unassessed" + assert report["column_bindings"] == [] + + +@pytest.mark.parametrize( + "name,status", [("earnings", "present"), ("missing", "absent")] +) +def test_only_explicit_binding_can_measure_column_presence(dataset, name, status): + report = build_concept_coverage( + InventoryProvider(), dataset=dataset, column_bindings=[column_binding(name)] + ) + assert report["inputs"][0]["column_status"] == status + assert report["inputs"][0]["data_origin"] == "unassessed" + assert report["dataset"]["extra_columns"] == "permitted" + + +def test_column_binding_without_dataset_does_not_measure_absence(): + report = build_concept_coverage( + InventoryProvider(), column_bindings=[column_binding("missing")] + ) + assert report["inputs"][0]["column_status"] == "unassessed" + + +def test_proxy_evidence_is_an_unverified_assertion_not_readiness(): + report = build_concept_coverage(InventoryProvider(), fact_bindings=[fact_binding()]) + assert report["inputs"][0]["semantic_status"] == "unverified_assertions" + assert report["inputs"][0]["metadata"]["concept_id"] is None + assert report["readiness"]["population_schema_ready"] is False + + +@pytest.mark.parametrize("relationship", ["exact", "proxy"]) +def test_non_unresolved_bindings_require_pinned_evidence(relationship): + binding = fact_binding(relationship) + binding["evidence"] = [] + with pytest.raises(ValueError, match="evidence"): + build_concept_coverage(InventoryProvider(), fact_bindings=[binding]) + + +@pytest.mark.parametrize("relationship", ["exact", "proxy"]) +def test_unknown_semantics_preserve_assertion_but_machine_classify_unresolved( + relationship, +): + report = build_concept_coverage( + InventoryProvider(), fact_bindings=[fact_binding(relationship)] + ) + binding = report["fact_bindings"][0] + assert binding["asserted_relationship"] == relationship + assert binding["effective_relationship"] == "unresolved" + assert binding["classification_reason"] == "target_semantics_unavailable" + + +@pytest.mark.parametrize( + "field", ["statistic", "entity", "universe", "unit", "geography", "period"] +) +def test_exact_statistical_scope_mismatch_fails_closed(field): + binding = fact_binding("exact") + binding["target"]["scope"][field] = "different" + with pytest.raises(ValueError, match="scope"): + build_concept_coverage(InventoryProvider(), fact_bindings=[binding]) + + +def test_unknown_and_duplicate_input_bindings_rejected(): + binding = column_binding() + binding["input"]["name"] = "computed_tax" + with pytest.raises(ValueError, match="unknown input"): + build_concept_coverage(InventoryProvider(), column_bindings=[binding]) + with pytest.raises(ValueError, match="duplicate"): + build_concept_coverage( + InventoryProvider(), column_bindings=[column_binding(), column_binding()] + ) + + +def test_manifest_is_closed_and_digest_is_verified(): + report = build_concept_coverage(InventoryProvider()) + for path in [(), ("inputs", 0, "metadata"), ("dataset",)]: + changed = deepcopy(report) + target = changed + for key in path: + target = target[key] + target["invented"] = True + with pytest.raises(ValueError): + validate_concept_coverage(changed) + changed = deepcopy(report) + changed["inputs"][0]["name"] = "changed" + with pytest.raises(ValueError, match="digest"): + validate_concept_coverage(changed) + + +def test_provenance_and_content_digest_are_deterministic(): + first = build_concept_coverage(InventoryProvider()) + assert build_concept_coverage(InventoryProvider()) == first + assert len(first["content_sha256"]) == 64 + assert ( + next( + item["sha256"] + for item in first["fingerprints"] + if item["role"] == "entry_module" + ) + == "a" * 64 + ) + + +def test_readiness_cannot_be_promoted_by_payload_edit(): + report = build_concept_coverage(InventoryProvider()) + report["readiness"]["certified"] = True + with pytest.raises(ValueError): + validate_concept_coverage(report) + + +@pytest.mark.parametrize("binding_type", ["column", "fact"]) +def test_bindings_cannot_be_replayed_against_different_artifact(binding_type): + binding = column_binding() if binding_type == "column" else fact_binding() + binding["artifact_fingerprints"][0]["sha256"] = "f" * 64 + argument = "column_bindings" if binding_type == "column" else "fact_bindings" + with pytest.raises(ValueError, match="fingerprint mismatch"): + build_concept_coverage(InventoryProvider(), **{argument: [binding]}) + + +def test_noncanonical_alias_binding_is_rejected_not_merged(): + binding = fact_binding() + binding["input"]["canonical_request_name"] = "zz:statutes/another#input.earnings" + with pytest.raises(ValueError, match="non-canonical alias"): + build_concept_coverage(InventoryProvider(), fact_bindings=[binding]) + + +def test_conflicting_target_concepts_at_one_slot_fail_closed(): + first = fact_binding() + first["target"]["concept_id"] = "concept:gross" + second = deepcopy(first) + second["source"]["fact_id"] = "chronicle:another/fact" + second["target"]["concept_id"] = "concept:net" + with pytest.raises(ValueError, match="conflicting target concept"): + build_concept_coverage(InventoryProvider(), fact_bindings=[first, second]) + + +def test_stock_without_reference_instant_is_rejected(): + binding = fact_binding() + binding["source"]["scope"]["measurement_kind"] = "stock" + with pytest.raises(ValueError, match="reference instant"): + build_concept_coverage(InventoryProvider(), fact_bindings=[binding]) + + +def test_stock_flow_and_assessment_income_year_cannot_be_asserted_exact(): + for field, value in ( + ("measurement_kind", "stock"), + ("accounting_basis", "assessment_year"), + ): + binding = fact_binding("exact") + binding["source"]["scope"][field] = value + if field == "measurement_kind": + binding["source"]["scope"]["reference_instant"] = "2025-12-31" + with pytest.raises(ValueError, match="scope"): + build_concept_coverage(InventoryProvider(), fact_bindings=[binding]) + + +def test_asserted_exact_requires_identity_transformation(): + binding = fact_binding("exact") + binding["transformation"]["kind"] = "declared_conversion" + with pytest.raises(ValueError, match="identity transformation"): + build_concept_coverage(InventoryProvider(), fact_bindings=[binding]) + + +@pytest.mark.parametrize("field", ["entity_definition", "universe_definition"]) +def test_assertions_need_pinned_population_definitions(field): + binding = fact_binding() + binding["source"]["scope"][field] = None + with pytest.raises(ValueError, match=field): + build_concept_coverage(InventoryProvider(), fact_bindings=[binding]) + + +def test_unsupported_discovery_refuses_instead_of_reporting_zero(): + with pytest.raises(TypeError, match="does not support input discovery"): + build_concept_coverage(object()) + + +@pytest.mark.parametrize("count", [0, 2]) +def test_partial_discovery_cannot_drop_or_invent_an_input(count): + class PartialProvider(InventoryProvider): + def input_inventory(self): + original = super().input_inventory() + discovery = deepcopy(original.entity_discovery) + discovery[0]["root_input_count"] = count + return replace(original, entity_discovery=discovery) + + with pytest.raises(ValueError, match="root input count"): + build_concept_coverage(PartialProvider()) + + +def test_zero_inputs_requires_successful_runtime_enumeration(): + class EmptyProvider(InventoryProvider): + def input_inventory(self): + original = super().input_inventory() + discovery = deepcopy(original.entity_discovery) + discovery[0]["root_input_count"] = 0 + return replace(original, inputs=(), entity_discovery=discovery) + + report = build_concept_coverage(EmptyProvider()) + assert report["root_input_count"] == 0 + assert report["inputs"] == [] + assert report["entity_discovery"][0]["root_input_count"] is None + assert report["entity_discovery"][1]["status"] == "complete" + + +def test_completely_failed_discovery_cannot_look_like_zero_inputs(): + class FailedProvider(InventoryProvider): + def input_inventory(self): + original = super().input_inventory() + discovery = tuple( + {**item, "status": "no_derived_program", "root_input_count": None} + for item in original.entity_discovery + ) + return replace(original, inputs=(), entity_discovery=discovery) + + with pytest.raises(ValueError, match="successfully enumerated"): + build_concept_coverage(FailedProvider()) + + +def test_no_program_is_not_successfully_enumerated_zero(): + class FalseZeroProvider(InventoryProvider): + def input_inventory(self): + original = super().input_inventory() + discovery = deepcopy(original.entity_discovery) + discovery[1]["root_input_count"] = 0 + return replace(original, entity_discovery=discovery) + + with pytest.raises(ValueError, match="uncompiled entity"): + build_concept_coverage(FalseZeroProvider()) + + +def test_discovery_without_provenance_cannot_look_complete(): + class UnpinnedProvider(InventoryProvider): + def input_inventory(self): + return replace(super().input_inventory(), fingerprints=()) + + with pytest.raises( + ValueError, match="missing or conflicting artifact fingerprints" + ): + build_concept_coverage(UnpinnedProvider()) + + +def _dataset_with_column(label): + return Frame( + { + "person": pd.DataFrame( + {"person_id": [1], "person_household_id": [1], label: [10.0]} + ), + "household": pd.DataFrame({"household_id": [1]}), + }, + EntitySchema(group_entities=("household",)), + {"household": Weights(np.array([1.0]), WeightKind.DESIGN)}, + ) + + +def test_integer_column_label_cannot_be_coerced_into_a_string_match(): + dataset = _dataset_with_column(123) + assert 123 in dataset.table("person").columns + assert "123" not in dataset.table("person").columns + with pytest.raises(ValueError, match="non-string column"): + build_concept_coverage( + InventoryProvider(), + dataset=dataset, + column_bindings=[column_binding("123")], + ) + + +def test_actual_string_column_label_remains_a_valid_explicit_match(): + dataset = _dataset_with_column("123") + report = build_concept_coverage( + InventoryProvider(), dataset=dataset, column_bindings=[column_binding("123")] + ) + assert report["inputs"][0]["column_status"] == "present" + + +@pytest.mark.parametrize("binding_kind", ["column", "fact"]) +def test_real_axiom_binding_cannot_replay_between_native_root_entities(binding_kind): + """The same address/file pins can name inputs on different native roots.""" + pytest.importorskip("axiom_rules_engine") + pytest.importorskip("axiom_rules_engine_dense") + root = (Path(__file__).parent / "fixtures/rulespec-zz").resolve() + module = root / "zz/policies/tests/shared_input.yaml" + schema = EntitySchema(group_entities=("household",)) + normal = AxiomEngine(module, schema, rulespec_roots=(root,)) + swapped = AxiomEngine( + module, + schema, + rulespec_roots=(root,), + entity_names={"person": "Household", "household": "Person"}, + ) + before = normal.input_inventory() + after = swapped.input_inventory() + original = next(item for item in before.inputs if item.entity == "person") + remapped = next(item for item in after.inputs if item.entity == "person") + assert original.engine_entity == "Person" + assert remapped.engine_entity == "Household" + assert original.canonical_request_name == remapped.canonical_request_name + assert before.fingerprints == after.fingerprints + binding = ( + column_binding("shared_input") if binding_kind == "column" else fact_binding() + ) + binding["input"] = { + "entity": original.entity, + "engine_entity": original.engine_entity, + "name": original.name, + "canonical_request_name": original.canonical_request_name, + } + binding["artifact_fingerprints"] = [dict(item) for item in before.fingerprints] + argument = "column_bindings" if binding_kind == "column" else "fact_bindings" + dataset = _dataset_with_column("shared_input") + first = build_concept_coverage(normal, dataset=dataset, **{argument: [binding]}) + person = next(item for item in first["inputs"] if item["entity"] == "person") + assert person["engine_entity"] == "Person" + assert first[argument][0]["input"]["engine_entity"] == "Person" + if binding_kind == "column": + assert person["column_status"] == "present" + else: + assert first[argument][0]["effective_relationship"] == "unresolved" + with pytest.raises(ValueError, match="engine entity"): + build_concept_coverage(swapped, dataset=dataset, **{argument: [binding]}) diff --git a/packages/microcosm-frame/src/microcosm/frame/adapters/axiom.py b/packages/microcosm-frame/src/microcosm/frame/adapters/axiom.py index dec26045a..c9228647c 100644 --- a/packages/microcosm-frame/src/microcosm/frame/adapters/axiom.py +++ b/packages/microcosm-frame/src/microcosm/frame/adapters/axiom.py @@ -59,7 +59,10 @@ compiled upstream, not behind a protocol change. """ +import platform from collections.abc import Mapping, Sequence +from hashlib import sha256 +from importlib.metadata import PackageNotFoundError, version from pathlib import Path from typing import Any @@ -68,7 +71,7 @@ from microcosm.frame.bundle import Frame from microcosm.frame.materialize import engine_tables, put_frame_table, read_frame_table -from microcosm.frame.rules import ExportContract +from microcosm.frame.rules import EngineInput, ExportContract, InputInventory from microcosm.frame.schema import EntitySchema, VariableMetadata __all__ = ["AxiomEngine", "AxiomEntityTableDataset", "BE_SCHEMA"] @@ -245,6 +248,170 @@ def entity_schema(self) -> EntitySchema: """Return the frame entity schema (no engine import required).""" return self._schema + def input_inventory(self) -> InputInventory: + """Compile an input-only diagnostic through the real dense runtime. + + Canonical request addresses come from the compiled catalog, never from + slot-name concatenation. The current runtime supplies no typed input + semantics (engine issue #62); missing metadata stays unknown. Root + membership is operational entity evidence, not a statistical universe. + Related-entity batches are not supported by this adapter, so a module + with relations fails rather than reporting partial input coverage. + + Fresh compilation prevents cached programs being paired with newer + source fingerprints. Full canonical-root YAML and toolchain digests + conservatively bind imported content, not just the entry module. + """ + adapter = AxiomEngine( + self._module, + self._schema, + rulespec_roots=self._rulespec_roots, + entity_names=self._entity_names, + arithmetic=self._arithmetic, + ) + # Validate the engine's canonical authority boundary before reading + # any root contents for provenance. The measured compilation below is + # still fresh and bracketed by before/after content fingerprints. + adapter._program(self._schema.person_entity, missing_ok=True) + adapter._programs.clear() + adapter._metadata = None + before = adapter._inventory_fingerprints() + inputs: list[EngineInput] = [] + discovery = [] + for entity in self._schema.entities: + program = adapter._program(entity, missing_ok=True) + if program is None: + discovery.append( + { + "entity": entity, + "engine_entity": self._entity_names[entity], + "status": "no_derived_program", + "root_input_count": None, + } + ) + continue + if program.relations: + raise NotImplementedError( + "Input inventory does not support related-entity inputs; " + "a relation-bearing module cannot report complete coverage." + ) + catalog = program.input_catalog + aliases = program.input_request_names + root_inputs = program.root_inputs + discovery.append( + { + "entity": entity, + "engine_entity": program.root_entity, + "status": "complete", + "root_input_count": len(root_inputs), + } + ) + for name in sorted(root_inputs): + canonical = catalog.get(name) + accepted = tuple(sorted(aliases.get(name, ()))) + if canonical is not None and canonical not in accepted: + raise ValueError(f"Inconsistent Axiom input catalog for {name!r}.") + inputs.append( + EngineInput( + name=name, + entity=entity, + engine_entity=program.root_entity, + canonical_request_name=canonical, + request_names=accepted, + ) + ) + after = adapter._inventory_fingerprints() + if before != after: + raise ValueError("Axiom sources changed during input discovery.") + return InputInventory( + inputs=tuple(inputs), + fingerprints=after, + mapped_entities=tuple(self._schema.entities), + entity_discovery=tuple(discovery), + runtime={ + "engine": "axiom", + "wrapper_distribution_version": _distribution_version( + "axiom-rules-engine" + ), + "native_distribution_version": _distribution_version( + "axiom-rules-engine-dense" + ), + "core_version": None, + "python": platform.python_version(), + "platform": platform.platform(), + "numpy": np.__version__, + }, + ) + + def _inventory_fingerprints(self) -> tuple[dict[str, str], ...]: + engine = self._import_engine() + import axiom_rules_engine_dense + + fingerprints = [] + for root in self._rulespec_roots: + files = sorted( + path + for path in root.rglob("*") + if path.is_file() and path.suffix.lower() in {".yaml", ".yml"} + ) + toolchain = root / ".axiom/toolchain.toml" + if toolchain.is_file(): + files.append(toolchain) + fingerprints.append( + { + "role": "rulespec_root_yaml_and_toolchain", + "name": root.name, + "sha256": _content_tree_digest(root, files), + } + ) + module_name = next( + ( + f"{root.name}/{self._module.relative_to(root).as_posix()}" + for root in self._rulespec_roots + if self._module.is_relative_to(root) + ), + self._module.name, + ) + fingerprints.append( + { + "role": "entry_module", + "name": module_name, + "sha256": sha256(self._module.read_bytes()).hexdigest(), + } + ) + for role, module in ( + ("engine_wrapper", engine), + ("engine_native", axiom_rules_engine_dense), + ): + path = Path(module.__file__) + files = ( + [path] + if path.suffix != ".py" + else [ + item + for item in path.parent.rglob("*") + if item.is_file() + and item.suffix in {".py", ".so", ".pyd", ".dylib"} + ] + ) + fingerprints.append( + { + "role": role, + "name": module.__name__, + "sha256": _content_tree_digest(path.parent, files), + } + ) + fingerprints.append( + { + "role": "adapter", + "name": "microcosm.frame.adapters.axiom", + "sha256": sha256(Path(__file__).read_bytes()).hexdigest(), + } + ) + return tuple( + sorted(fingerprints, key=lambda item: (item["role"], item["name"])) + ) + # ------------------------------------------------------------------ # Materialization # ------------------------------------------------------------------ @@ -685,6 +852,27 @@ def _read(cls, path: Path) -> tuple[dict[str, pd.DataFrame], int]: return tables, time_period +def _distribution_version(name: str) -> str | None: + """Source-path imports can expose a runtime without installed dist metadata.""" + try: + return version(name) + except PackageNotFoundError: + return None + + +def _content_tree_digest(root: Path, files: Sequence[Path]) -> str: + """Hash relative paths and bytes with length framing, never absolute paths.""" + digest = sha256() + for path in sorted(files): + relative = path.relative_to(root).as_posix().encode() + content = path.read_bytes() + digest.update(len(relative).to_bytes(8, "big")) + digest.update(relative) + digest.update(len(content).to_bytes(8, "big")) + digest.update(content) + return digest.hexdigest() + + def _period_bounds(period: int | str) -> tuple[str, str, str]: """Map a kernel period to dense-execution (start, end, period_kind). diff --git a/packages/microcosm-frame/src/microcosm/frame/rules.py b/packages/microcosm-frame/src/microcosm/frame/rules.py index 34a3e1ca5..2fb006e3d 100644 --- a/packages/microcosm-frame/src/microcosm/frame/rules.py +++ b/packages/microcosm-frame/src/microcosm/frame/rules.py @@ -26,11 +26,60 @@ __all__ = [ "RulesEngine", "ExportContract", + "EngineInput", + "InputInventory", + "InputInventoryProvider", "assert_rules_engine_country", "materialize_rules_engine_predictors", ] +@dataclass(frozen=True) +class EngineInput: + """Observed executable input, not a complete population concept. + + ``entity`` is the adapter's operational frame entity. A request address + identifies an executable input; it does not establish semantic equivalence + to a publisher concept. Missing authored metadata remains ``None``. + """ + + name: str + entity: str + engine_entity: str + canonical_request_name: str | None + request_names: tuple[str, ...] + dtype: str | None = None + unit: str | None = None + period: str | None = None + definition: str | None = None + concept_id: str | None = None + required: bool | None = None + + +@dataclass(frozen=True) +class InputInventory: + """A module-scoped input observation with content fingerprints. + + This is not an export contract or a closed population schema. Inputs + outside the mapped entity surface and future-reform fields are not + adjudicated by this diagnostic. + """ + + inputs: tuple[EngineInput, ...] + fingerprints: tuple[Mapping[str, str], ...] + mapped_entities: tuple[str, ...] + entity_discovery: tuple[Mapping[str, Any], ...] + runtime: Mapping[str, str | None] + + +class InputInventoryProvider(Protocol): + """Optional metadata discovery; existing RulesEngine adapters need not opt in.""" + + def input_inventory(self) -> InputInventory: + """Discover executable inputs without guessing missing semantics.""" + ... + + @runtime_checkable class RulesEngine(Protocol): """Adapter interface to a tax-benefit rules engine. diff --git a/packages/microcosm-frame/tests/test_axiom_adapter.py b/packages/microcosm-frame/tests/test_axiom_adapter.py index cd9c07f32..3b4321ee4 100644 --- a/packages/microcosm-frame/tests/test_axiom_adapter.py +++ b/packages/microcosm-frame/tests/test_axiom_adapter.py @@ -15,6 +15,7 @@ import importlib.util import os +import shutil from pathlib import Path import numpy as np @@ -85,6 +86,84 @@ def _toy_bundle( return Frame({"person": person, "household": household}, BE_SCHEMA, weights) +@needs_engine +class TestInputInventory: + def test_real_runtime_names_addresses_entities_and_unknown_metadata(self): + inventory = AxiomEngine( + FIXTURE_MODULE, rulespec_roots=FIXTURE_RULESPEC_ROOTS + ).input_inventory() + by_name = {item.name: item for item in inventory.inputs} + assert "toy_taxable_income" in by_name + assert "toy_income_tax" not in by_name + assert by_name["toy_household_rent"].entity == "household" + assert by_name["toy_is_exempt"].dtype is None + for item in inventory.inputs: + assert item.canonical_request_name in item.request_names + assert "#input." in item.canonical_request_name + assert item.concept_id is None + assert item.definition is None + assert item.unit is None + assert item.period is None + assert item.required is None + assert sum( + item["root_input_count"] for item in inventory.entity_discovery + ) == len(inventory.inputs) + assert all(item["status"] == "complete" for item in inventory.entity_discovery) + assert inventory.runtime["engine"] == "axiom" + + def test_inventory_is_deterministic_and_recompiles_after_source_change( + self, tmp_path + ): + root = tmp_path / "rulespec-zz" + shutil.copytree(FIXTURE_RULESPEC_ROOT, root) + module = root / FIXTURE_MODULE.relative_to(FIXTURE_RULESPEC_ROOT) + engine = AxiomEngine(module, rulespec_roots=(root,)) + first = engine.input_inventory() + assert engine.input_inventory() == first + assert ( + "toy_taxable_income" in engine.variables() + ) # Prime materialization cache. + module.write_text(module.read_text().replace("toy_taxable_income", "new_input")) + second = engine.input_inventory() + assert "new_input" in {item.name for item in second.inputs} + assert "toy_taxable_income" not in {item.name for item in second.inputs} + first_pins = {item["role"]: item["sha256"] for item in first.fingerprints} + second_pins = {item["role"]: item["sha256"] for item in second.fingerprints} + assert first_pins["entry_module"] != second_pins["entry_module"] + assert ( + first_pins["rulespec_root_yaml_and_toolchain"] + != second_pins["rulespec_root_yaml_and_toolchain"] + ) + + def test_inventory_refuses_sources_that_change_while_discovering(self, monkeypatch): + counter = 0 + + def changing_fingerprints(self): + nonlocal counter + counter += 1 + return ( + {"role": "fixture", "name": "changed", "sha256": str(counter) * 64}, + ) + + monkeypatch.setattr( + AxiomEngine, "_inventory_fingerprints", changing_fingerprints + ) + with pytest.raises(ValueError, match="changed during input discovery"): + AxiomEngine( + FIXTURE_MODULE, rulespec_roots=FIXTURE_RULESPEC_ROOTS + ).input_inventory() + + def test_invalid_authority_is_rejected_before_fingerprint_reads( + self, tmp_path, monkeypatch + ): + def forbidden_read(self): + raise AssertionError("must validate authority before fingerprinting") + + monkeypatch.setattr(AxiomEngine, "_inventory_fingerprints", forbidden_read) + with pytest.raises(ValueError): + AxiomEngine(FIXTURE_MODULE, rulespec_roots=(tmp_path,)).input_inventory() + + class TestLazyImport: def test_adapter_constructs_without_the_engine(self) -> None: adapter = AxiomEngine(FIXTURE_MODULE, rulespec_roots=FIXTURE_RULESPEC_ROOTS) diff --git a/tools/inventory_axiom_concepts.py b/tools/inventory_axiom_concepts.py new file mode 100644 index 000000000..6d66adc9e --- /dev/null +++ b/tools/inventory_axiom_concepts.py @@ -0,0 +1,29 @@ +#!/usr/bin/env python3 +"""Print a real Axiom input-coverage diagnostic without reading any microdata.""" + +import argparse +import json +from pathlib import Path + +from microcosm.build.concept_coverage import build_concept_coverage +from microcosm.frame.adapters.axiom import AxiomEngine +from microcosm.frame.schema import EntitySchema + + +def main() -> None: + """Compile the explicit module/root/entity surface and emit closed JSON.""" + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--module", type=Path, required=True) + parser.add_argument("--rulespec-root", type=Path, action="append", required=True) + parser.add_argument("--person-entity", default="person") + parser.add_argument("--group-entity", action="append", required=True) + args = parser.parse_args() + schema = EntitySchema( + person_entity=args.person_entity, group_entities=tuple(args.group_entity) + ) + engine = AxiomEngine(args.module, schema, rulespec_roots=args.rulespec_root) + print(json.dumps(build_concept_coverage(engine), indent=2, sort_keys=True)) + + +if __name__ == "__main__": + main()