Skip to content

feat(evidence): add exact-SHA validation evidence#120

Merged
Nobodyworld merged 15 commits into
mainfrom
feat/exact-sha-validation-evidence
Jul 23, 2026
Merged

feat(evidence): add exact-SHA validation evidence#120
Nobodyworld merged 15 commits into
mainfrom
feat/exact-sha-validation-evidence

Conversation

@Nobodyworld

@Nobodyworld Nobodyworld commented Jul 21, 2026

Copy link
Copy Markdown
Owner

Summary

Delivers issue #114 from the exact merged-worker base and adds the first complete deterministic exact-SHA validation workflow.

base SHA: f549ab7bb2efc274dc2d79e12502d5653ddc8886
head SHA: e938d3fdc3937babb4f13d3a7980a6c704b8cebd
branch: feat/exact-sha-validation-evidence
ahead of main: 15 commits
behind main: 0
changed files: 34
merged: no

Trusted manifest:

name/version: validate-switchboard@1
digest: 10e99418e4e6f0e9f4a6e95fb5b9a267dab4eeac4671cf58533c8b9afe1fed98

Implemented

  • Immutable reviewed fixed-argv validation steps whose complete executable definition contributes to the manifest digest.
  • Metadata-only manifest APIs that omit argv, executable paths, shell strings, and scripts.
  • Exact-SHA disposable read-only worktrees with unchanged canonical repositories.
  • Worker-owned evidence directories separated from source and canonical repository roots.
  • Strict relative artifact paths, containment checks, regular-file and symlink/reparse rejection, count/size limits, SHA-256 hashing, and marker-verified configurable retention.
  • Structured test, coverage, security, and dependency-audit parsing.
  • Deterministic evidence fingerprinting bound to repository, exact SHA, manifest, worker environment, dependency locks, normalized step outcomes, and artifact hashes.
  • Strict completion persistence and GET /api/execution/runs/{run_id}/evidence.
  • Truthful timeout, cancellation, ownership-loss, artifact, cleanup, dependency-lock, and local-record failure handling.
  • Two harmless server-backed exact-SHA executions with retained evidence and released leases.
  • Operator, API, architecture, and living ExecPlan documentation.

Server output safety

The final connector-review follow-up adds a server-owned absolute-local-path policy independent of worker redaction:

  • recursively rejects Windows drive paths, UNC paths, file: URIs, and rooted POSIX paths in compact evidence;
  • rejects those shapes in completion and run terminal text before persistence;
  • preserves safe relative artifact/log references and common non-local URIs;
  • makes run and evidence APIs fail closed if malformed legacy/persisted data contains a local path;
  • retains worker-side redaction as defense in depth.

Direct malformed-worker schema and API tests prove rejected payloads are not persisted or returned.

Validation

Final SHA workflows:

  • Commitlint 29901539272success;
  • CI 29901539178success.

The complete hosted matrix passed:

  • lint/pre-commit and TODO validation;
  • Mypy/typecheck;
  • full pytest;
  • Bandit and dependency/security audit;
  • full-history Gitleaks;
  • link validation;
  • strict browser UI tests;
  • coverage and all configured module thresholds.

Recorded local evidence:

  • focused server: 45 passed;
  • focused worker/server-backed: 23 passed, 1 expected platform skip;
  • cancellation/ownership: 26 passed;
  • full pytest: 384 passed, 4 skipped;
  • aggregate coverage: 93%; all 16 module thresholds passed;
  • strict browser: 2 passed, zero skips;
  • pip-audit: no known vulnerabilities;
  • Gitleaks: 203 commits, no leaks;
  • exact-SHA server-backed proofs remain green for 3a8422b70e15133be064094e46358c22a661a929 and 1bb9bc24daadd3d46064323a76a25ff353b6ea65.

Local-only limitation: the shared interpreter retains its unrelated pre-existing opencv-python / NumPy pip check conflict. Hosted Python 3.11 validation is fully green.

Boundaries

This PR does not add artifact-byte upload/download, GitHub integration, MCP, cost routing, evidence reuse, a known-baseline registry, RPA, arbitrary commands, or repository writes.

Full logs and artifact bytes remain local. Normal APIs expose only strict compact metadata, safe relative references, and hashes.

Closes #114.

Copy link
Copy Markdown
Owner Author

Connector handoff checkpoint

Repository coordination is complete through current head deedf93f0eea43561eb08a7d5dfd8ee8d3b14ee3.

Completed through the connector:

Current blocker:

  • CI run 29809281686 fails only the pinned pre-commit step because a hook rewrites the working tree. The connector log view truncates the hook-generated diff. Browser and coverage jobs are skipped only because they depend on lint.

Local continuation requirements:

  1. Start exactly from feat/exact-sha-validation-evidence at deedf93f0eea43561eb08a7d5dfd8ee8d3b14ee3.
  2. Require a clean working tree and matching origin head.
  3. Run python -m pre_commit run --all-files --show-diff-on-failure using the repository environment.
  4. Inspect and retain the exact hook mutation; do not weaken, skip, repin, or reconfigure any hook.
  5. Re-run pre-commit until clean, then continue the complete feat(evidence): add exact-SHA validation manifest and structured local evidence #114 implementation and living ExecPlan.
  6. Push only this branch; no rebase, force push, new branch, new PR, merge, or work on release: execute Linux symlink and final local validation #104.
  7. Keep PR feat(evidence): add exact-SHA validation evidence #120 draft and unmerged for final connector review.

The issue #114 contract and .agent/execplans/008_exact_sha_validation_evidence.md are authoritative for product behavior, security boundaries, tests, and validation.

Copy link
Copy Markdown
Owner Author

Local handoff blocker: stale pytest temp directory

The Codex starting checks matched the canonical branch and SHAs, but stopped because the repository-root path worker-pytest-tmp/ is unreadable and unignored.

This path is not product source. The completed worker ExecPlan already records the full suite as passing while explicitly excluding the unreadable permitted worker-pytest-tmp/ collection path, and separately states that generated temporary worktrees/test data must not be committed.

Disposition:

  • treat worker-pytest-tmp/ as a stale local pytest temp directory;
  • verify it is the exact repository-root path and is not a symlink, junction, or tracked path;
  • recover local permissions if necessary, then remove only that directory;
  • do not change product files, reset, clean the repository broadly, or delete any other path;
  • rerun the mandatory starting checks against unchanged head deedf93f0eea43561eb08a7d5dfd8ee8d3b14ee3.

No GitHub branch, PR, issue, or code state changed as a result of this local-only blocker. PR #120 remains open, draft, unmerged, and based on f549ab7bb2efc274dc2d79e12502d5653ddc8886.

Copy link
Copy Markdown
Owner Author

Local continuation remains blocked before any repository edit. The mandatory safety checks passed, but Windows denied ownership transfer for the stale repository-root worker-pytest-tmp/ directory:

ERROR: Access is denied.
takeown failed.

The cleanup script stopped before icacls or deletion. No repository files, commits, branches, or PR state changed.

Required operator action: open PowerShell as Administrator, remove only the exact non-reparse, untracked worker-pytest-tmp/ directory using the previously supplied guarded cleanup block, close the elevated shell, then resume the Codex task from unchanged head deedf93f0eea43561eb08a7d5dfd8ee8d3b14ee3. Do not add a Git ignore rule merely to conceal the unreadable directory.

Copy link
Copy Markdown
Owner Author

Local implementation checkpoint — quota-blocked, nothing new pushed

Codex stopped before committing or pushing incomplete issue #114 implementation because its execution quota was exhausted.

Verified remote state

  • PR remains open, draft, and unmerged.
  • Remote head remains deedf93f0eea43561eb08a7d5dfd8ee8d3b14ee3.
  • Remote branch remains 8 commits ahead of f549ab7bb2efc274dc2d79e12502d5653ddc8886 and 0 behind.
  • The remote diff is still limited to the ExecPlan, cancellation-precedence worker fix, and its regression tests.

Local-only state reported by Codex

  • One local commit exists but is not pushed: 892a3ef fix(worker): retain pinned formatter output.
  • Additional issue feat(evidence): add exact-SHA validation manifest and structured local evidence #114 product implementation is present but uncommitted and remains unverified.
  • Pinned pre-commit passed.
  • Cancellation suites passed: 3 + 12 + 6 tests.
  • Schema/storage tests reached 19 passed and one platform skip after correcting one assertion mismatch.
  • Two server-backed executions passed once at exact SHAs:
    • 3a8422b70e15133be064094e46358c22a661a929
    • 1bb9bc24daadd3d46064323a76a25ff353b6ea65
  • Later API-malformation and absolute-path-redaction changes remain unverified.
  • The focused test retry was blocked by Windows CreateProcessAsUserW 1312; the required escalated retry was then blocked by Codex quota exhaustion until July 28, 2026.

Required continuation

Preserve the existing dirty worktree exactly. Do not reset, clean, stash, switch branches, rebase, or discard files. On resume, verify the expected local/remote relationship, rerun the blocked focused tests with the necessary Windows execution permissions, complete validation, update the living ExecPlan, commit the remaining coherent implementation, push only the existing branch, and keep this PR draft and unmerged for connector review.

@Nobodyworld Nobodyworld left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Connector review at effa3271fbe5afe2ba2ca5f53ea42089cddb7f8e confirms the implementation scope, exact-SHA identity binding, fixed trusted argv, evidence containment/hashing/retention, compact endpoint, two server-backed executions, and fully green hosted matrix.

One acceptance boundary remains before this PR should be marked ready:

The server must independently reject absolute local paths in worker-submitted compact evidence and terminal result text.

The trusted worker currently redacts Windows/POSIX paths before building summaries, and the end-to-end test confirms the normal worker-produced API response contains no local paths. However, the server-side ExecutionEvidence/ExecutionCompletionIn contracts still accept arbitrary bounded strings in fields such as step summary, terminal_reason, and result_summary. An authenticated malformed or buggy worker can therefore submit a local absolute path that the server persists and returns through /api/execution/runs, or submit it inside evidence returned by /api/execution/runs/{run_id}/evidence.

Required narrow correction:

  1. Put the absolute-local-path detection policy in a server-owned shared helper.
  2. Reject Windows and POSIX absolute-local-path shapes recursively across compact evidence free-text fields.
  3. Apply the same rejection to completion result_summary and other terminal free-text returned by run APIs.
  4. Keep relative artifact/log references valid.
  5. Add direct schema/API regressions that submit malformed worker payloads; do not only assert that the normal worker happened to redact them.
  6. Preserve current worker redaction as defense in depth.
  7. Rerun focused evidence/API tests and the complete hosted matrix.

Keep the PR draft and unmerged until this server-side output invariant is proven.

@Nobodyworld Nobodyworld left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Final connector review completed at e938d3fdc3937babb4f13d3a7980a6c704b8cebd.

The prior server-output blocker is resolved:

  • absolute local path detection is server-owned and recursive;
  • Windows drive, UNC, file: URI, and rooted POSIX path forms are rejected;
  • safe relative artifact/log paths and common non-local URIs remain valid;
  • malformed worker completion/evidence payloads fail before persistence;
  • corrupt persisted run/evidence text fails closed without echoing the path;
  • worker redaction remains defense in depth;
  • deterministic fingerprint and exact-SHA evidence behavior remain intact.

The follow-up is limited to two commits and six files beyond the previously reviewed head. The branch remains 15 commits ahead and zero behind its exact base. Final Commitlint 29901539272 and CI 29901539178 are successful, including lint, typecheck, tests, security, secrets audit, links, strict browser UI, and coverage.

No connector-review blockers remain for issue #114. The PR is suitable to leave draft status and enter normal review/merge gating. Do not merge without owner authorization.

Copy link
Copy Markdown
Owner Author

Additional independent evidence from documentation PR #123 confirms the ownership-loss race fixed in this PR is real and remains present on current main.

PR #123 final-head CI run 29952225701 reproduced:

client/python/tests/test_execution_worker_monitor.py::
test_control_plane_cancellation_stops_parent_and_child_without_stale_success[ownership]

expected no completion; observed stale failed completion:
worker_error:RuntimeError

The documentation changes do not touch worker code. Gitleaks, links, lint/pre-commit, Mypy, and security passed on that run; only the known nondeterministic cancellation regression failed. PR #123 has therefore been returned to draft and explicitly blocked behind this PR rather than duplicating the runtime fix.

PR #120 itself remains unchanged at e938d3fdc3937babb4f13d3a7980a6c704b8cebd, ready, mergeable, unmerged, and fully green in Commitlint 29901539272 and CI 29901539178.

@Nobodyworld
Nobodyworld merged commit dcb8e28 into main Jul 23, 2026
9 checks passed
@Nobodyworld
Nobodyworld deleted the feat/exact-sha-validation-evidence branch July 23, 2026 07:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(evidence): add exact-SHA validation manifest and structured local evidence

1 participant