Implement the September 6 CTO outcome audit and release 4.3.0 - #106
Merged
Conversation
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…e outcome audit Kernel: direction criterion when the result waits on the owner, one point-of-use obligation for delegation, act-shaped playbook triggers, routing by reasoning demand, consequence, and cost of checking, owned temporary state and unachieved effects in reconciliation, refusal and permission wording, disclosure scoped to the authorized audience. Playbooks: dated host mechanics in delegation, independent expectation in verification. Documents: Codex sandbox and spawn_agent facts, project_doc_max_bytes boundary, README delivery claim, audit disposition in decisions and evidence, 4.3.0 changelog, ledgers rebound to 4.3.0 with the 4.2.0 host receipts kept at their immutable source. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…atches A direct `pytest tests` run imported a shipped script and wrote `plugins/skiphow/skills/skiphow/scripts/__pycache__/`, which the package payload hash counts as a package change, so the identity checks failed until that directory was removed by hand. `tests/conftest.py` now sets `sys.dont_write_bytecode` before any test module is imported, and exports `PYTHONDONTWRITEBYTECODE` for the Python processes the tests spawn. The environment variable alone does not help, because the interpreter reads it at startup. When a receipt or an installed payload does not match the candidate, the error now names the extra, missing, and changed package-relative paths, so a polluted checkout is diagnosed instead of silently cleaned. `committed_package_payload` carries the committed per-file payload for that diff; the receipt schema is unchanged. The full pytest run in `scripts/check.py` asks for `--durations=10` and prints the slowest tests to stderr on success too, since the run is bounded at 120 seconds and a passing run otherwise shows nothing. The bound and `-p no:cacheprovider` are unchanged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Per-run eligibility discarded what retained records do establish. A capture that shows the kernel body entering a session settles loading and nothing else, and a destination record plus an independent grader settles delivery whatever terminal state the session reached. Eligibility is now decided per claim, with the rules fixed before any further run. scripts/claim_eligibility.py reads a receipt directory's claims.json, which names per claim the retained records carrying its evidence, and prints one row per claim: claim, package version, host, status, and the reason or the missing item. Observed, FAIL and UNVERIFIED do not substitute for one another: a confirmed failure stays a failure, a loading record can never reach the delivery claim, absence of a loading event counts only against a comparable positive on the same instrument, and unknown cost blocks nothing. The validator starts no model, edits no ledger, and upgrades nothing. The rules were validated against the retained captures rather than fitted to them, and no new run was made. The 4.2.0 and 4.1.0 isolated Codex directories and the 4.1.0 Claude pilot now carry claims.json files whose declared status a test compares with the derived one, so the ledger cannot drift from the evidence. docs/evidence.md states the per-claim rules above the full run record, which comparative and canonical-suite claims still need, and replaces "None" with the per-claim table the validator produces. Two rows are confirmed failures. evals/README.md documents the file and requires the host's stderr stream to be retained beside the JSON event stream, because the 4.2.0 sandbox refusal reached only stderr and is absent from the trace. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Worktrees share the protected Git directory on Codex; Claude effort can also come from skill frontmatter; the reviewer obligation is an independent expectation, not a reading order; the delegation obligation is named as the one playbook a duty depends on; stalled work stays a diagnosis trigger; public pages attribute the 4.2.0 receipts to 4.2.0 and link the immutable ledger; nine playbooks, not eight. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…bility Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The test asserted `sys.dont_write_bytecode` in its own process, but
`scripts/check.py` already sets `PYTHONDONTWRITEBYTECODE=1` for the pytest
subprocess it starts, so the assertion held with `tests/conftest.py` deleted and
the gate could not catch its removal. The test now spawns a small pytest
selection that imports the shipped activation script, with
`PYTHONDONTWRITEBYTECODE` and `PYTHONPYCACHEPREFIX` stripped from the child
environment, and asserts that nothing appeared under `plugins/skiphow/`, naming
any directory it finds before removing it. A companion test runs the same
selection with `--noconftest` and requires the bytecode to appear, so the
conftest is proven to be the mechanism rather than an ambient setting. Renaming
the conftest away now fails the first test, which was verified.
The durations test asserted `kwargs.get("timeout", 120) == 120`, which held
whatever the bound was because `offline_checks` passes no timeout. It now
asserts that no timeout is passed and that `checked`'s own default is still 120
seconds.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Two independent reviews found the validator accepting evidence that does not establish what it claimed, and hiding failures behind the first unusable declaration. A loading confirmation now has to be content the host returned: a recognized result event that completed with exit code zero, whose returned text carries the kernel body. The kernel sentence inside a command the session asked for, or inside the output of a failed read, no longer counts. The final report is what the owner was shown and never private reasoning. A control arm must be declared package-free and its transcript must agree, so a control that read the kernel is rejected. A manifest without a mode and a 64-character content hash for a path no longer compares equal to another manifest missing the same fields. Two ordering defects let one unusable declaration hide an established failure: completion honesty and foreign-work preservation now judge every declared part, effect, and path, raise the contradiction first, and name what went unchecked. A missing declared status is drift rather than agreement, so --check-declared catches an entry that states none. Malformed nested records become missing evidence instead of an exception that aborts the directory, and a grader timeout is missing evidence too. The row prints the host and the host version as separate fields, and the loading reason claims only what the rule establishes rather than an ordering nothing compares. The retained receipts keep every status they had. docs/evidence.md follows the changed rules, drops the ordering clause from the loading rows, and says that only a receipt directory carrying behavioral claims has a claims.json. Ten further tests cover each repaired defect. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
User-visible result
SkipHow 4.3.0 implements the September 6 CTO outcome audit (
skiphow-audit-2026-09-06.md, second revision) end to end where no model run is required, and records the rest as designed experiments.delegationin context before dispatching, act-shaped playbook triggers, no re-read of an unchanged playbook (F5); routing by reasoning demand, consequence, and cost of checking, set through the host's control, replacing "demonstrated adequate" (F4); owned clones and scratch checkouts and unachieved material effects in the completion reconciliation, no bypass of a refusal, name the exact blocker and permission (F6, F9); disclosure scoped to the authorized audience (F9).delegation.md, with the Codex read-and-verify fallback stated as the current limitation (F3); the reviewer's independent source of expectation and the candidate-before-integration split inverification.md(F11).sandbox_modedocumented but not applied in 0.153.0 source,spawn_agenthas no directory or sandbox parameter,project_doc_max_bytescaps project files only, ClaudepermissionModeis not a read-only boundary for plugin subagents, the README delivery prompt did not name SkipHow but named the destination and verification.pytestrun writes no bytecode under the package; identity mismatches name extra, missing, and changed paths; the gate prints its slowest tests and keeps its 120 s limit.scripts/claim_eligibility.py,claims.jsonper receipt directory) defined before any new run and validated against the retained 4.1.0 and 4.2.0 captures; the ledger's observed-behavior section lists each claim's status, including two confirmedFAILobservations.Scope
No hook or import ships (F1 needs the staged comparison, which needs isolated Claude authentication and a grant). No paid model session was run. Version 4.3.0 (minor): the kernel widens what a run does within existing authority; no authority boundary, skill name, or record format changes.
Tests run
python scripts/check.pypasses in the worktree (Python 3.14.7, pinned deps).git diff --checkclean.python scripts/check_hosts.py --skip-install: Claude schema validation PASS; Codex validator runs in CI.scripts/claim_eligibility.py <dir> --check-declaredexits 0 for all three receipt directories.Package evidence
evals/host-smoke.jsonis rebound to 4.3.0 with every rowUNVERIFIED; the 4.2.0 receipts stay at the immutablef684eb2ledger. Fresh 4.3.0 host receipts follow in a separate PR after merge, as for 4.2.0.Blockers and unverified limits
Every behavioral consequence is
UNVERIFIED. Designed and not run: the three-mechanism Claude activation comparison, the Codex refused-command capture test, the direction-rule drift fixture, the fixed known-bad reviewer candidate, a Codex writer lane. Each is listed indocs/evidence.mdwith what it needs.🤖 Generated with Claude Code