Implement State Integrity Report CLI and validation

[mdheller]

Context

sourceos-syncd is now the canonical home for the SourceOS State Integrity Report and Index Lane model.

Specs and fixtures:

• docs/specs/sourceos-state-integrity-report.md
• docs/specs/sourceos-index-lanes.md
• examples/health/healthy.snapshot.json
• examples/health/degraded.snapshot.json
• examples/health/repair-plan.json

Work

Implement the first non-destructive runtime surface:

• sourceos-syncd health snapshot
• sourceos-syncd health explain
• sourceos-syncd health verify
• sourceos-syncd repair plan

Do not implement destructive repair apply until the plan format, policy checks, checkpoint logic, and Lampstand evidence path are validated.

Acceptance criteria

• CLI emits JSON matching the example shape.
• health explain translates raw counters into diagnosis.
• health verify checks journal, heartbeat, shadow/checkpoint, schema, tombstone, disk pressure, and policy invariants.
• repair plan produces a non-destructive preview with preconditions, steps, blocked destructive steps, rollback, and postconditions.
• Golden examples are used as fixtures in tests.
• The implementation preserves health-of-health fields: partial collection, redaction, permission denied, timed out, unavailable.

[mdheller]

Turn 1/7 integration pass completed.

Implemented the first CLI/runtime foundation:

• Python package metadata and console script: sourceos-syncd
• State Integrity report engine in src/sourceos_syncd/reports.py
• CLI in src/sourceos_syncd/cli.py
• Tests for runtime snapshot, golden healthy/degraded examples, and preview-only repair plans
• GitHub Actions CI for pytest plus CLI smoke checks

Notes:

• repair plan is intentionally preview-only.
• Destructive repair remains blocked until policy grant, checkpoint, rollback, and evidence plumbing are implemented.
• GitHub did not yet show workflow runs or combined status for the latest commit when checked immediately after commit.

Next turns:

2/7: add JSON schema files and stronger validation.
3/7: add real store/journal fixture model and filesystem-backed local state prototype.
4/7: add PolicyFabric decision hook stub.
5/7: add Lampstand evidence writer stub.
6/7: add AgentPlane trust-gate contract artifact and examples.
7/7: wire Delivery Excellence scorecard fixture and close the first stable integration loop.

[mdheller]

Turn 2/7 integration pass completed.

Implemented schema hardening:

• Added canonical JSON Schemas under schemas/:
  • sourceos.state-integrity-report.v1alpha1.schema.json
  • sourceos.repair-plan.v1alpha1.schema.json
• Added standard-library contract validation in src/sourceos_syncd/reports.py:
  • identity, collection, runtime timestamp, store, lane, resource, policy, invariant, and repair-plan checks
  • duplicate store/lane detection
  • decoded flag consistency checks, including flags_hex matching flags_raw
  • pressure/status/severity enum enforcement
• Updated sourceos-syncd health verify to emit contract_errors from the stronger validator.
• Expanded tests to parse schemas, validate runtime snapshots and examples, catch bad flags, catch duplicate lanes, and validate repair preview plans.
• Updated CI to parse schema files, run tests, and smoke-test the CLI against healthy and degraded examples.
• Updated README to expose the JSON Schema contracts.

Verification note: the latest commit was fetched back and the CI workflow file reflects the new checks. GitHub did not yet show workflow runs for the newest commit when checked immediately after commit.

Countdown remaining:

3/7: add real store/journal fixture model and filesystem-backed local state prototype.
4/7: add PolicyFabric decision hook stub.
5/7: add Lampstand evidence writer stub.
6/7: add AgentPlane trust-gate contract artifact and examples.
7/7: wire Delivery Excellence scorecard fixture and close the first stable integration loop.

[mdheller]

Turn 3/7 integration pass completed.

Implemented the filesystem-backed local state prototype:

• Added src/sourceos_syncd/local_store.py:
  • explicit local store initialization
  • manifest and JSONL journal
  • durable/rebuildable/disposable directory classes
  • lane summaries, object counters, tombstone counters, registry counts, producer attribution, generation markers
  • read paths do not silently initialize state
• Added src/sourceos_syncd/store_reports.py:
  • projects local store summaries into State Integrity Reports
  • keeps local attribution under diagnosis to preserve the closed top-level report contract
• Extended CLI:
  • sourceos-syncd store init --root <path>
  • sourceos-syncd store record --root <path> --event-type <type> --lane <lane> ...
  • sourceos-syncd health snapshot --store-root <path>
• Added tests/test_local_store.py:
  • initialized store behavior
  • uninitialized store snapshot behavior
  • event recording and summarization
  • store-backed report validation
• Updated CI:
  • initializes a temp store
  • records normal and priority events
  • generates a store-backed snapshot
  • verifies the generated report
• Added docs/local-state-store.md explaining the prototype, layout, CLI, reporting behavior, and current limits.

Verification note: latest committed files were fetched back. GitHub did not yet show workflow runs for the newest commit when checked immediately after commit.

Countdown remaining:

4/7: add PolicyFabric decision hook stub.
5/7: add Lampstand evidence writer stub.
6/7: add AgentPlane trust-gate contract artifact and examples.
7/7: wire Delivery Excellence scorecard fixture and close the first stable integration loop.

[mdheller]

Turn 4/7 integration pass completed.

Implemented the PolicyFabric-compatible hook:

• Added src/sourceos_syncd/policy.py with a local deterministic policy evaluator and stable sourceos.policy-decision/v1alpha1 decision shape.
• Wired store-backed reports through the local evaluator in src/sourceos_syncd/store_reports.py:
  • policy.policy_engine = policy-fabric-local-stub
  • policy.policy_version = v0.1.0-local-stub
  • policy.policy_decisions now carries numeric status counts
  • representative decision samples live under diagnosis.policy
• Added schemas/sourceos.policy-decision.v1alpha1.schema.json.
• Added tests/test_policy_hook.py for secure lane handling, unknown actions, status counting, and report integration.
• Added docs/policy-fabric-hook.md with the contract, local defaults, report mapping, and replacement path for the real PolicyFabric service.
• Added examples/policy/secure-agent-access.decision.json as a canonical decision fixture.

Verification note: latest committed files were fetched back. GitHub did not yet show workflow runs for the newest commit when checked immediately after commit.

Countdown remaining:

5/7: add Lampstand evidence writer stub.
6/7: add AgentPlane trust-gate contract artifact and examples.
7/7: wire Delivery Excellence scorecard fixture and close the first stable integration loop.

[mdheller]

Turn 5/7 integration pass completed.

Implemented the Lampstand-compatible local evidence path:

• Added src/sourceos_syncd/evidence.py:
  • canonical JSON artifact digesting
  • deterministic evidence IDs
  • sourceos.lampstand-evidence/v1alpha1 envelopes
  • local unsigned attestation metadata
  • evidence validation and tamper detection
  • atomic local JSON writes
• Added schemas/sourceos.lampstand-evidence.v1alpha1.schema.json.
• Extended CLI:
  • sourceos-syncd evidence wrap
  • sourceos-syncd evidence write
  • sourceos-syncd evidence validate
• Added tests/test_evidence.py for digest integrity, tamper detection, and local file writing.
• Added examples/evidence/secure-agent-access.evidence.json, wrapping the policy-decision fixture as evidence.
• Updated CI to exercise local evidence wrapping, validation, and writing.
• Added docs/lampstand-evidence.md documenting the envelope, CLI, local behavior, security posture, and replacement path for real Lampstand signing/storage.

Verification note: latest committed files were fetched back. GitHub did not yet show workflow runs for the newest commit when checked immediately after commit.

Countdown remaining:

6/7: add AgentPlane trust-gate contract artifact and examples.
7/7: wire Delivery Excellence scorecard fixture and close the first stable integration loop.

[mdheller]

Turn 6/7 integration pass completed.

Implemented the AgentPlane-compatible trust gate:

• Added src/sourceos_syncd/trust.py:
  • sourceos.agent-trust-decision/v1alpha1 decision shape
  • report contract validation before trust
  • lane availability checks
  • degraded-mode handling
  • signed-attestation requirement handling
  • policy-count restriction signals
  • trust decision validation
• Added schemas/sourceos.agent-trust-decision.v1alpha1.schema.json.
• Extended CLI:
  • sourceos-syncd trust evaluate
  • sourceos-syncd trust validate
• Added tests/test_trust_gate.py.
• Added examples/trust/normal-read.allowed.json.
• Added docs/agentplane-trust-gate.md.
• Updated CI to evaluate allowed trust, check an expected blocked attestation path, validate the trust fixture, wrap a trust decision as evidence, and validate the resulting evidence envelope.

Verification note: latest committed files were fetched back. GitHub did not yet show workflow runs for the newest commit when checked immediately after commit.

Countdown remaining:

7/7: wire Delivery Excellence scorecard fixture and close the first stable integration loop.

[mdheller]

Wrap-up / remiss-items pass completed.

Closed the remaining Delivery Excellence gap and added fixture validation:

• Added src/sourceos_syncd/scorecard.py for Delivery Excellence-compatible local scorecards.
• Added schemas/sourceos.delivery-scorecard.v1alpha1.schema.json.
• Extended CLI with sourceos-syncd scorecard evaluate and sourceos-syncd scorecard validate.
• Added tests/test_scorecard.py.
• Added examples/scorecards/healthy.scorecard.json.
• Extended tests/test_fixtures.py to validate the committed scorecard fixture, in addition to policy, evidence, and trust fixtures.
• Updated CI so the full loop is exercised: health snapshot -> policy/trust/evidence -> scorecard -> evidence.
• Added docs/delivery-excellence-scorecard.md documenting the status model, gates, dimensions, CLI, and Delivery Excellence replacement path.

Final stable local loop now exists:

State Integrity Report -> PolicyFabric-compatible decisions -> AgentPlane-compatible trust decision -> Lampstand-compatible evidence envelope -> Delivery Excellence scorecard.

Verification note: committed files were fetched back. GitHub still did not surface workflow runs immediately after the newest commit when checked.

[mdheller]

Closed: the State Integrity Report CLI and validation landed across PRs #36 (sync-daemon, check-health), #38/#39 (sync status command + coverage), and #49 (integrate-sourceos-syncd-daemon). The health snapshot, explain, verify, and repair plan surface is implemented.