diff --git a/CHANGELOG.md b/CHANGELOG.md index 325c81b..a8f915f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -31,7 +31,7 @@ The always-loaded kernel regains the duties an independent audit found weakened This is a minor release. The kernel widens what a run does within existing authority: it stops and hands over one batch when the result waits on the owner, reads `delegation` before dispatching, and names the permission it lacks. The delegation playbook adds host mechanics. No authority boundary, public skill name, or record format changes, and existing grants and restrictions survive the upgrade. -Every behavioral consequence is `UNVERIFIED`; acceptance is static. The experiments the audit designed, a three-mechanism Claude activation comparison, a refused-command capture test on Codex, a fixture at the drift point for the direction rule, a fixed known-bad candidate for the revised reviewer, and a Codex writer lane, need the owner's grant and are recorded with their designs in [docs/evidence.md](docs/evidence.md). Host receipts for the exact 4.3.0 package are recorded in `evals/host-smoke.json` when they exist; until then every row there is `UNVERIFIED` and the 4.2.0 receipts remain at their immutable source. +Every behavioral consequence is `UNVERIFIED`; acceptance is static. The experiments the audit designed, a three-mechanism Claude activation comparison, a refused-command capture test on Codex, a fixture at the drift point for the direction rule, a fixed known-bad candidate for the revised reviewer, and a Codex writer lane, need the owner's grant and are recorded with their designs in [docs/evidence.md](docs/evidence.md). On the exact 4.3.0 package, Claude Code 2.1.263 installed and uninstalled the seventeen files in an empty configuration directory, and Codex CLI 0.153.0 installed them byte-identically from the approved Git source into an isolated home and removed them; those four cells of `evals/host-smoke.json` are `PASS` with receipts in `evals/receipts/host-validation-430-20260906/`, every other cell is `UNVERIFIED`, and the 4.2.0 receipts remain at their immutable source. ## 4.2.0 (2026-09-06) diff --git a/SECURITY.md b/SECURITY.md index c371571..49d47da 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -15,7 +15,7 @@ services keep their own security policies. ## Package validation, 2026-09-06 -Version 4.3.0 is validated per capability in [`evals/host-smoke.json`](evals/host-smoke.json); every row there is `UNVERIFIED` until a receipt made on the exact 4.3.0 package is recorded, and the 4.2.0 receipts remain at their [immutable source](https://github.com/mzored/SkipHow/blob/f684eb2f3d2e7baf8b8488e8efb5ac4703d67cff/evals/host-smoke.json): Claude Code 2.1.261 clean install and uninstall, and Codex CLI 0.153.0 clean install from the approved Git source, uninstall, persistent setup, explicit fallback, and playbook load each carried one for 4.2.0. The [dated support summary](docs/evidence.md#support-summary-as-of-2026-09-06) states what each host has shown for each package it names. The previous 4.1.1 receipts remain in `evals/receipts/host-validation-411-20260905/`; the September 5 isolated Codex diagnostics remain 4.1.0 observations, and the [September 6 diagnostics](evals/receipts/isolated-host-420-20260906/README.md) are separate 4.2.0 observations. +Version 4.3.0 is validated per capability in [`evals/host-smoke.json`](evals/host-smoke.json); Claude Code 2.1.263 clean install and uninstall, and Codex CLI 0.153.0 clean install from the approved Git source and uninstall, each carry one receipt on the exact 4.3.0 package in [`evals/receipts/host-validation-430-20260906/`](evals/receipts/host-validation-430-20260906/); every other row is `UNVERIFIED` until a receipt made on 4.3.0 is recorded, and the 4.2.0 receipts, including persistent setup, explicit fallback, and playbook load, remain at their [immutable source](https://github.com/mzored/SkipHow/blob/f684eb2f3d2e7baf8b8488e8efb5ac4703d67cff/evals/host-smoke.json). The [dated support summary](docs/evidence.md#support-summary-as-of-2026-09-06) states what each host has shown for each package it names. The previous 4.1.1 receipts remain in `evals/receipts/host-validation-411-20260905/`; the September 5 isolated Codex diagnostics remain 4.1.0 observations, and the [September 6 diagnostics](evals/receipts/isolated-host-420-20260906/README.md) are separate 4.2.0 observations. The historical 4.1.0 candidate passed both host schema validators. Claude Code 2.1.261 installed all fifteen regular files byte for byte and uninstalled them in a @@ -74,7 +74,7 @@ page under `learn.chatgpt.com`; the redirect target is the page actually read. | Per-agent read-only controls | Subagent frontmatter takes a `tools` allowlist, `disallowedTools`, and `permissionMode`, whose values include `plan` for read-only exploration. `permissionMode` is ignored for plugin subagents and overridden by a parent in bypass, accept-edits, or auto mode, so only a `tools` allowlist checked to have applied is a read-only boundary. | [Subagents](https://code.claude.com/docs/en/sub-agents) | 2026-09-06 | none | `UNVERIFIED` (documented) | | Worktree isolation | `isolation: worktree` runs a subagent in a temporary git worktree. | [Subagents](https://code.claude.com/docs/en/sub-agents) | 2026-09-04 | none | `UNVERIFIED` (documented) | | Plugin validation | Manifest `.claude-plugin/plugin.json`; `claude plugin validate ` validates it and `--strict` treats warnings as errors. | [Plugins](https://code.claude.com/docs/en/plugins) | 2026-09-04 | 2.1.259 | `PASS` (`scripts/check_hosts.py`, 2026-09-04) | -| Clean installation | `claude plugin marketplace add`, `claude plugin install --scope user`, `claude plugin uninstall --scope user`; `CLAUDE_CONFIG_DIR` points the host at a scratch home. | [Discover plugins](https://code.claude.com/docs/en/discover-plugins), [Skills](https://code.claude.com/docs/en/skills) | 2026-09-06 | 2.1.261 | `PASS` (`scripts/check_hosts.py --smoke`: clean home, install, 17 regular files matching exact 4.2.0 payload `5bcd09d1…`, uninstall verified; [4.2.0 ledger](https://github.com/mzored/SkipHow/blob/f684eb2f3d2e7baf8b8488e8efb5ac4703d67cff/evals/host-smoke.json)) | +| Clean installation | `claude plugin marketplace add`, `claude plugin install --scope user`, `claude plugin uninstall --scope user`; `CLAUDE_CONFIG_DIR` points the host at a scratch home. | [Discover plugins](https://code.claude.com/docs/en/discover-plugins), [Skills](https://code.claude.com/docs/en/skills) | 2026-09-06 | 2.1.263 | `PASS` (`scripts/check_hosts.py --smoke`: clean home, install, 17 regular files matching exact 4.3.0 payload `a0901a39…`, uninstall verified; [ledger](evals/host-smoke.json)) | ### Codex CLI @@ -85,7 +85,7 @@ page under `learn.chatgpt.com`; the redirect target is the page actually read. | Per-agent read-only controls | Custom agents are TOML files in the Codex home `agents/` directory or the project `.codex/agents/` and the page says they may set `sandbox_mode` per agent, naming a read-only agent as the example, and that subagents otherwise inherit the parent's sandbox policy and permission mode. In the `rust-v0.153.0` source, `core/src/agent/role.rs` applies developer instructions, model, reasoning effort and summary, verbosity, personality, service tier, features, and skills, and not `sandbox_mode`; `role_tests.rs` asserts a role cannot expand the parent's permissions. | [Subagents](https://developers.openai.com/codex/subagents), [openai/codex `role.rs`](https://github.com/openai/codex/blob/rust-v0.153.0/codex-rs/core/src/agent/role.rs) | 2026-09-06 | none | `UNVERIFIED` (documented; not applied in source as of 2026-09-06, so a subagent inherits the parent's sandbox) | | Worktree isolation | The subagents page documents no worktree or separate-checkout option for a subagent, and `spawn_agent` takes an agent type, the message, and where exposed `model` and `reasoning_effort`, with no working directory, worktree, or sandbox parameter. | [Subagents](https://developers.openai.com/codex/subagents), [openai/codex `multi_agents_spec.rs`](https://github.com/openai/codex/blob/rust-v0.153.0/codex-rs/core/src/tools/handlers/multi_agents_spec.rs) | 2026-09-06 | none | `UNVERIFIED` (no per-delegate directory or sandbox exists in this version; a separate-checkout writer lane is untried) | | Plugin validation | Manifest `.codex-plugin/plugin.json`. There is no `codex plugin validate` subcommand; validation runs the `validate_plugin.py` script shipped with the plugin-creator system skill in the Codex repository, which CI checks out at a pinned commit. | [openai/codex plugin-creator scripts](https://github.com/openai/codex/tree/333beecd41281b1350688b417a2f20c66e2a743e/codex-rs/skills/src/assets/samples/plugin-creator/scripts) | 2026-09-04 | none locally | `UNVERIFIED` locally (validator not on this machine); required to `PASS` in CI | -| Clean installation | `codex plugin marketplace add`, `codex plugin add`, `codex plugin list --json`, `codex plugin remove` exist in `codex plugin --help`; `CODEX_HOME` relocates the host home. The plugins page documents the plugin browser and uninstall but none of these commands. | [Plugins](https://developers.openai.com/codex/plugins), `codex plugin --help` 0.153.0 | 2026-09-06 | 0.153.0 | `PASS` for exact 4.2.0 from the approved Git source in an isolated home: 17 regular files byte-identical to the committed package, then removed ([4.2.0 ledger](https://github.com/mzored/SkipHow/blob/f684eb2f3d2e7baf8b8488e8efb5ac4703d67cff/evals/host-smoke.json)); the release runner's local marketplace is still refused by the managed `/etc/codex/requirements.toml` source policy | +| Clean installation | `codex plugin marketplace add`, `codex plugin add`, `codex plugin list --json`, `codex plugin remove` exist in `codex plugin --help`; `CODEX_HOME` relocates the host home. The plugins page documents the plugin browser and uninstall but none of these commands. | [Plugins](https://developers.openai.com/codex/plugins), `codex plugin --help` 0.153.0 | 2026-09-06 | 0.153.0 | `PASS` for exact 4.3.0 from the approved Git source in an isolated home: 17 regular files byte-identical to the committed package, then removed ([ledger](evals/host-smoke.json)); the release runner's local marketplace is still refused by the managed `/etc/codex/requirements.toml` source policy | ### Codex surfaces diff --git a/docs/evidence.md b/docs/evidence.md index 069568c..954b42c 100644 --- a/docs/evidence.md +++ b/docs/evidence.md @@ -92,12 +92,12 @@ marketplace-source policy before installation, so Codex clean install remains ## Support summary as of 2026-09-06 -This is the one dated statement of what each host has shown. Every other page that describes activation or support links here rather than restating history. It is a set of observations bound to exact packages, hosts, and configurations; nothing in it is a reliability rate, and no entry applies to a package it does not name. The current package is 4.3.0; no receipt exists on it yet, and the newest receipts below are on 4.2.0, retained at the [immutable 4.2.0 ledger](https://github.com/mzored/SkipHow/blob/f684eb2f3d2e7baf8b8488e8efb5ac4703d67cff/evals/host-smoke.json). +This is the one dated statement of what each host has shown. Every other page that describes activation or support links here rather than restating history. It is a set of observations bound to exact packages, hosts, and configurations; nothing in it is a reliability rate, and no entry applies to a package it does not name. The current package is 4.3.0; its only receipts are the clean install and uninstall checks in the package-validation column, and every behavioral row below is on 4.2.0 or older, retained at the [immutable 4.2.0 ledger](https://github.com/mzored/SkipHow/blob/f684eb2f3d2e7baf8b8488e8efb5ac4703d67cff/evals/host-smoke.json). | Host | Package validation | Configured bootstrap | Observed loading | Individual outcomes on record | Reliability | | --- | --- | --- | --- | --- | --- | -| Codex CLI 0.153.0 | Exact 4.2.0 installed from the approved Git source into an isolated home, all seventeen files byte-identical to the committed package, and removed afterwards; recorded in the [4.2.0 ledger](https://github.com/mzored/SkipHow/blob/f684eb2f3d2e7baf8b8488e8efb5ac4703d67cff/evals/host-smoke.json). Schema validation runs in CI. | Deterministic, and observed once: asked by name to enable itself, the agent ran the packaged helper, which resolved a non-empty `AGENTS.override.md` over `AGENTS.md`, previewed, asked once, and wrote one owned block there. Asked to disable, it removed exactly that block. | From the persistent block, twice: once on 4.1.0 from an `AGENTS.md` block in a home whose fixture lived beside operator artifacts, and once on 4.2.0 from an `AGENTS.override.md` block on a clean fixture, the kernel loading before any edit. Explicit `$skiphow` invocation loaded it in the 4.2.0 enable and disable sessions, and on 4.1.0 the kernel was in context again after a native resume and after a native compaction; the [per-claim table](#3x-and-4x-observed-behavior) lists each loading row and the two unrelated requests where it correctly did not load. | 4.1.0: four correct repairs delivered to a synthetic remote, read-only analysis, unrelated requests unloaded, native resume and compaction. 4.2.0: four correct repairs delivered to `fix/catalog` on a synthetic remote and verified from a fresh clone, foreign work preserved, no publication; the [4.2.0 receipts](../evals/receipts/isolated-host-420-20260906/README.md) list the deviations. | Unknown. | -| Claude Code 2.1.261 | Schema validation passed and the exact 4.2.0 package installed all seventeen files byte for byte and uninstalled cleanly in an empty configuration directory; recorded in the [4.2.0 ledger](https://github.com/mzored/SkipHow/blob/f684eb2f3d2e7baf8b8488e8efb5ac4703d67cff/evals/host-smoke.json). | Deterministic: the helper resolves `CLAUDE.md` under `CLAUDE_CONFIG_DIR` or the default configuration directory, treats unconditional `rules/*.md` files as read, and consolidates duplicate copies it can edit. | Never with persistent configuration. Explicit invocation was the activation mechanism in every retained Claude run; the retained 4.1.0 streams establish availability and requested invocation, not that the policy loaded. One bare-prompt pilot did not select it. Isolated authentication is unavailable, so this stays `UNVERIFIED`. | 4.1.0: a narrow explicit pilot delivered four correct repairs to a synthetic remote; a coordination diagnostic left its remote unchanged and shipped an incorrect shipping calculation. | Unknown. | +| Codex CLI 0.153.0 | Exact 4.3.0 installed from the approved Git source into an isolated home, all seventeen files byte-identical to the committed package, and removed afterwards; recorded in the [ledger](../evals/host-smoke.json), as 4.2.0 was in the [4.2.0 ledger](https://github.com/mzored/SkipHow/blob/f684eb2f3d2e7baf8b8488e8efb5ac4703d67cff/evals/host-smoke.json). Schema validation runs in CI. | Deterministic, and observed once: asked by name to enable itself, the agent ran the packaged helper, which resolved a non-empty `AGENTS.override.md` over `AGENTS.md`, previewed, asked once, and wrote one owned block there. Asked to disable, it removed exactly that block. | From the persistent block, twice: once on 4.1.0 from an `AGENTS.md` block in a home whose fixture lived beside operator artifacts, and once on 4.2.0 from an `AGENTS.override.md` block on a clean fixture, the kernel loading before any edit. Explicit `$skiphow` invocation loaded it in the 4.2.0 enable and disable sessions, and on 4.1.0 the kernel was in context again after a native resume and after a native compaction; the [per-claim table](#3x-and-4x-observed-behavior) lists each loading row and the two unrelated requests where it correctly did not load. | 4.1.0: four correct repairs delivered to a synthetic remote, read-only analysis, unrelated requests unloaded, native resume and compaction. 4.2.0: four correct repairs delivered to `fix/catalog` on a synthetic remote and verified from a fresh clone, foreign work preserved, no publication; the [4.2.0 receipts](../evals/receipts/isolated-host-420-20260906/README.md) list the deviations. | Unknown. | +| Claude Code 2.1.263 | Schema validation passed and the exact 4.3.0 package installed all seventeen files byte for byte and uninstalled cleanly in an empty configuration directory; recorded in the [ledger](../evals/host-smoke.json), as 4.2.0 was on 2.1.261 in the [4.2.0 ledger](https://github.com/mzored/SkipHow/blob/f684eb2f3d2e7baf8b8488e8efb5ac4703d67cff/evals/host-smoke.json). | Deterministic: the helper resolves `CLAUDE.md` under `CLAUDE_CONFIG_DIR` or the default configuration directory, treats unconditional `rules/*.md` files as read, and consolidates duplicate copies it can edit. | Never with persistent configuration. Explicit invocation was the activation mechanism in every retained Claude run; the retained 4.1.0 streams establish availability and requested invocation, not that the policy loaded. One bare-prompt pilot did not select it. Isolated authentication is unavailable, so this stays `UNVERIFIED`. | 4.1.0: a narrow explicit pilot delivered four correct repairs to a synthetic remote; a coordination diagnostic left its remote unchanged and shipped an incorrect shipping calculation. | Unknown. | Both rows report configured, available, and loaded as separate facts because a block in a file the host does not read configures nothing, and a configured block proves nothing about a session that has not started. Managed policy files on this machine restrict Codex marketplace sources; the helper reports such files and does not evaluate or bypass them. diff --git a/evals/host-smoke.json b/evals/host-smoke.json index 2cb942b..d53c234 100644 --- a/evals/host-smoke.json +++ b/evals/host-smoke.json @@ -60,8 +60,23 @@ "claude-code": { "results": { "clean_install": { - "status": "UNVERIFIED", - "receipt": null + "receipt": { + "check": "clean_install", + "cleanup_result": "plugin absent before scratch host home cleanup", + "command_or_session": "host marketplace add, install, list, inspect, uninstall, list", + "configuration": "empty CLAUDE_CONFIG_DIR; exact local marketplace snapshot", + "date": "2026-09-06", + "host": "claude-code", + "host_version": "2.1.263 (Claude Code)", + "observable_evidence": "installed regular files exactly matched the candidate payload", + "outcome": "PASS", + "package_commit": "335912b6de00fa1411a954183f1a4c9b82a4c148", + "package_payload_sha256": "a0901a3927c4a33b74e28da6677e42329156b9bbca6b9ba388350866aaf9dc89", + "package_tree": "c96108f395ac278823b772d9c678cd52b1aeb6e0", + "package_version": "4.3.0", + "source": "scripts/check_hosts.py --smoke" + }, + "status": "PASS" }, "persistent_setup": { "status": "UNVERIFIED", @@ -92,16 +107,46 @@ "receipt": null }, "uninstall": { - "status": "UNVERIFIED", - "receipt": null + "receipt": { + "check": "uninstall", + "cleanup_result": "plugin absent before scratch host home cleanup", + "command_or_session": "host marketplace add, install, list, inspect, uninstall, list", + "configuration": "empty CLAUDE_CONFIG_DIR; exact local marketplace snapshot", + "date": "2026-09-06", + "host": "claude-code", + "host_version": "2.1.263 (Claude Code)", + "observable_evidence": "host inventory no longer listed SkipHow", + "outcome": "PASS", + "package_commit": "335912b6de00fa1411a954183f1a4c9b82a4c148", + "package_payload_sha256": "a0901a3927c4a33b74e28da6677e42329156b9bbca6b9ba388350866aaf9dc89", + "package_tree": "c96108f395ac278823b772d9c678cd52b1aeb6e0", + "package_version": "4.3.0", + "source": "scripts/check_hosts.py --smoke" + }, + "status": "PASS" } } }, "codex": { "results": { "clean_install": { - "status": "UNVERIFIED", - "receipt": null + "status": "PASS", + "receipt": { + "package_version": "4.3.0", + "package_commit": "da1154c730efd6e7efa037841a6e5bad07a74578", + "package_tree": "c96108f395ac278823b772d9c678cd52b1aeb6e0", + "package_payload_sha256": "a0901a3927c4a33b74e28da6677e42329156b9bbca6b9ba388350866aaf9dc89", + "host": "codex", + "host_version": "codex-cli 0.153.0", + "date": "2026-09-06", + "source": "operator procedure recorded in evals/receipts/host-validation-430-20260906.md", + "check": "clean_install", + "outcome": "PASS", + "configuration": "isolated CODEX_HOME with native login only; approved Git marketplace source https://github.com/mzored/SkipHow.git at main (da1154c)", + "command_or_session": "codex plugin marketplace add, codex plugin add skiphow@skiphow --json, codex plugin list --json, operator byte comparison of the plugin cache against the committed plugins/skiphow tree with scripts/check_hosts.py _payload", + "observable_evidence": "all 17 installed regular files matched the committed package; payload sha256 equal; no symlink or extra file; host listed the plugin as installed and disabled", + "cleanup_result": "plugin removed immediately after inspection; isolated home restored to auth.json and config.toml" + } }, "persistent_setup": { "status": "UNVERIFIED", @@ -132,8 +177,23 @@ "receipt": null }, "uninstall": { - "status": "UNVERIFIED", - "receipt": null + "status": "PASS", + "receipt": { + "package_version": "4.3.0", + "package_commit": "da1154c730efd6e7efa037841a6e5bad07a74578", + "package_tree": "c96108f395ac278823b772d9c678cd52b1aeb6e0", + "package_payload_sha256": "a0901a3927c4a33b74e28da6677e42329156b9bbca6b9ba388350866aaf9dc89", + "host": "codex", + "host_version": "codex-cli 0.153.0", + "date": "2026-09-06", + "source": "operator procedure recorded in evals/receipts/host-validation-430-20260906.md", + "check": "uninstall", + "outcome": "PASS", + "configuration": "same isolated CODEX_HOME directly after the install check; no model session ran", + "command_or_session": "codex plugin remove skiphow@skiphow, codex plugin list --json", + "observable_evidence": "host inventory no longer listed skiphow@skiphow; the marketplace entry itself is configured by the machine's managed /etc/codex/config.toml and stayed", + "cleanup_result": "plugin cache and temporary marketplace snapshot removed from the isolated home" + } } } } diff --git a/evals/receipts/host-validation-430-20260906.md b/evals/receipts/host-validation-430-20260906.md new file mode 100644 index 0000000..0758062 --- /dev/null +++ b/evals/receipts/host-validation-430-20260906.md @@ -0,0 +1,34 @@ +Scope: this release runner. External candidate and model-session receipts are recorded separately. + +| Capability | Status | Detail | +| --- | --- | --- | +| Deterministic package gate | UNVERIFIED | not run; pass --package-gate | +| Codex schema validation | UNVERIFIED | Codex plugin validator unavailable | +| Claude schema validation | PASS | claude plugin validate --strict | +| Clean Codex install | UNVERIFIED | managed source policy refused the local marketplace | +| Clean Claude install | PASS | receipt claude-clean-install-2026-09-06.json | +| Explicit invocation | UNVERIFIED | not run by the release runner; external model-session evidence is recorded separately | +| Implicit activation | UNVERIFIED | not run by the release runner; external model-session evidence is recorded separately | +| Continuity/bootstrap | UNVERIFIED | not run by the release runner; external model-session evidence is recorded separately | +| Behavioral contract suite | UNVERIFIED | never run or implied by CI; the versioned summary is docs/evidence.md | + +## External candidate checks + +The local deterministic gate passed on the 4.3.0 candidate. Claude Code 2.1.263 +installed all seventeen regular files byte for byte in an empty configuration +directory and uninstalled them; the receipt is in this directory. The Codex row +above records the refusal of a local marketplace by the machine's managed source +policy. Separately, after the change set merged as `da1154c`, the exact 4.3.0 +package was installed into an isolated Codex home (native login only, `CODEX_HOME` +and the operating-system home pointed at scratch locations) from the approved Git +marketplace source `https://github.com/mzored/SkipHow.git`: `codex plugin +marketplace add`, `codex plugin add skiphow@skiphow --json`, `codex plugin list +--json`, then an operator byte comparison of the plugin cache against the committed +`plugins/skiphow` tree, which found all seventeen regular files identical, payload +`a0901a39…`, no symlink and no extra file. `codex plugin remove skiphow@skiphow` +then emptied the host inventory of the plugin, and the cache and temporary +marketplace snapshot were removed from the isolated home. No model session ran on +4.3.0; the ledger cells in [`../host-smoke.json`](../host-smoke.json) record only +these install and uninstall facts, and every other Codex and Claude cell stays +`UNVERIFIED`. The Codex plugin validator was unavailable on this machine and runs +in CI. diff --git a/evals/receipts/host-validation-430-20260906/claude-clean-install-2026-09-06.json b/evals/receipts/host-validation-430-20260906/claude-clean-install-2026-09-06.json new file mode 100644 index 0000000..093c2a1 --- /dev/null +++ b/evals/receipts/host-validation-430-20260906/claude-clean-install-2026-09-06.json @@ -0,0 +1,129 @@ +{ + "date": "2026-09-06", + "detail": "exact candidate installed, inspected, and uninstalled from a clean host home", + "host": "claude-code", + "host_version": "2.1.263 (Claude Code)", + "installed_files": { + ".claude-plugin/plugin.json": "fcb51cf29f3486604ef86502d7608df902c0de21142dacd2cdc741631218f5cc", + ".codex-plugin/plugin.json": "bc87bf214bac3bb60c42f45d469003acf57ad0607dbe542ccb4af370a30ed51f", + "LICENSE": "f6c21980331ae32f64afe9585dabbb60098b5b26b09e7873505fe23042cb99df", + "SOURCES.json": "d74f6ccef12488b27e195bdfc677781c130bc81b514a1bca6218eaf82b93093b", + "THIRD_PARTY_NOTICES.md": "aa6eb608121360a0f46f4c153ce6c29cf7e52d2952b2d9c85938a3ec43444263", + "skills/skiphow/SKILL.md": "f5ca31cba54be983af1d4b259787ace07865562650cca57e675289fc7cd42acd", + "skills/skiphow/agents/openai.yaml": "c7ba7daa2c66aef3a99c7e5dd001905baa0976913b9e903f94726d4bc2904f51", + "skills/skiphow/references/delegation.md": "bcc41f3ad33f9499d12d82bcc00613a55da32a1e4d814c475a12d3620096b02c", + "skills/skiphow/references/diagnosis.md": "8d0f337437d9ee3982e6df27ed1a58b89a01269e500e81b4dfce553c3976deb4", + "skills/skiphow/references/integration.md": "dee2c76f254c2083108451e65207516d9da045dfb2cda3af0f1d64cf8e523671", + "skills/skiphow/references/operations.md": "def02aa656056dd0a35fcd1c3ce72a67698b95d71098612b365b6f8490c166d0", + "skills/skiphow/references/product.md": "b4e098938fe93db53cdf026de09c54c1f6fb1839eaa42b057ab10d883c377ac6", + "skills/skiphow/references/setup.md": "cdb63b0820307414a850027dbf1032f99e2d02bc8a8a8c45ff1deefef4915eee", + "skills/skiphow/references/technical-design.md": "24e94087743cf14747e59920d3fb12d9ac6a0ae51bff52fb43b7a491acbf87b8", + "skills/skiphow/references/tracked-work.md": "df6a5a00caa2a1bc30a5a518185658e4cc10f3f18fb18a71179554c95a8927f8", + "skills/skiphow/references/verification.md": "971e051fbc171c2fa75fc4a597904216e4af650a4b0e9560eb8015dd8cd9afbb", + "skills/skiphow/scripts/activation.py": "9c70f335a9fc34f212553f4810e176195f00dbf6bd622713e852880b04595f7b" + }, + "result": "PASS", + "results": { + "clean_install": { + "receipt": { + "check": "clean_install", + "cleanup_result": "plugin absent before scratch host home cleanup", + "command_or_session": "host marketplace add, install, list, inspect, uninstall, list", + "configuration": "empty CLAUDE_CONFIG_DIR; exact local marketplace snapshot", + "date": "2026-09-06", + "host": "claude-code", + "host_version": "2.1.263 (Claude Code)", + "observable_evidence": "installed regular files exactly matched the candidate payload", + "outcome": "PASS", + "package_commit": "335912b6de00fa1411a954183f1a4c9b82a4c148", + "package_payload_sha256": "a0901a3927c4a33b74e28da6677e42329156b9bbca6b9ba388350866aaf9dc89", + "package_tree": "c96108f395ac278823b772d9c678cd52b1aeb6e0", + "package_version": "4.3.0", + "source": "scripts/check_hosts.py --smoke" + }, + "status": "PASS" + }, + "uninstall": { + "receipt": { + "check": "uninstall", + "cleanup_result": "plugin absent before scratch host home cleanup", + "command_or_session": "host marketplace add, install, list, inspect, uninstall, list", + "configuration": "empty CLAUDE_CONFIG_DIR; exact local marketplace snapshot", + "date": "2026-09-06", + "host": "claude-code", + "host_version": "2.1.263 (Claude Code)", + "observable_evidence": "host inventory no longer listed SkipHow", + "outcome": "PASS", + "package_commit": "335912b6de00fa1411a954183f1a4c9b82a4c148", + "package_payload_sha256": "a0901a3927c4a33b74e28da6677e42329156b9bbca6b9ba388350866aaf9dc89", + "package_tree": "c96108f395ac278823b772d9c678cd52b1aeb6e0", + "package_version": "4.3.0", + "source": "scripts/check_hosts.py --smoke" + }, + "status": "PASS" + } + }, + "schema": "skiphow-host-smoke-bundle/1", + "steps": [ + { + "detail": "", + "status": "PASS", + "step": "plain marketplace" + }, + { + "detail": "", + "status": "PASS", + "step": "clean host home" + }, + { + "detail": "", + "status": "PASS", + "step": "marketplace" + }, + { + "detail": "", + "status": "PASS", + "step": "install" + }, + { + "detail": "", + "status": "PASS", + "step": "list" + }, + { + "detail": "17 regular files, exact candidate bytes, no symlinks", + "status": "PASS", + "step": "inspect installed files" + }, + { + "detail": "package ships no hook", + "status": "UNVERIFIED", + "step": "inspect hook trust/state" + }, + { + "detail": "", + "status": "PASS", + "step": "uninstall" + }, + { + "detail": "", + "status": "PASS", + "step": "list after uninstall" + }, + { + "detail": "", + "status": "PASS", + "step": "verify uninstall" + }, + { + "detail": "9.5 step 5: starts a model; not run by this script", + "status": "UNVERIFIED", + "step": "start a clean session" + }, + { + "detail": "9.5 step 6: starts a model; not run by this script", + "status": "UNVERIFIED", + "step": "verify explicit invocation" + } + ] +} diff --git a/evals/receipts/host-validation-430-20260906/codex-clean-install-2026-09-06.json b/evals/receipts/host-validation-430-20260906/codex-clean-install-2026-09-06.json new file mode 100644 index 0000000..595c9f1 --- /dev/null +++ b/evals/receipts/host-validation-430-20260906/codex-clean-install-2026-09-06.json @@ -0,0 +1,61 @@ +{ + "date": "2026-09-06", + "detail": "Error: marketplace source `/private/marketplace` is not allowed by requirements from /etc/codex/requirements.toml", + "host": "codex", + "host_version": "codex-cli 0.153.0", + "installed_files": {}, + "result": "FAIL", + "results": { + "clean_install": { + "receipt": { + "check": "clean_install", + "cleanup_result": "scratch host home removed; uninstall was not verified", + "command_or_session": "host marketplace add, install, list, inspect, uninstall, list", + "configuration": "empty CODEX_HOME; exact local marketplace snapshot", + "date": "2026-09-06", + "host": "codex", + "host_version": "codex-cli 0.153.0", + "observable_evidence": "Error: marketplace source `/private/marketplace` is not allowed by requirements from /etc/codex/requirements.toml", + "outcome": "FAIL", + "package_commit": "335912b6de00fa1411a954183f1a4c9b82a4c148", + "package_payload_sha256": "a0901a3927c4a33b74e28da6677e42329156b9bbca6b9ba388350866aaf9dc89", + "package_tree": "c96108f395ac278823b772d9c678cd52b1aeb6e0", + "package_version": "4.3.0", + "source": "scripts/check_hosts.py --smoke" + }, + "status": "FAIL" + }, + "uninstall": { + "receipt": null, + "status": "UNVERIFIED" + } + }, + "schema": "skiphow-host-smoke-bundle/1", + "steps": [ + { + "detail": "", + "status": "PASS", + "step": "plain marketplace" + }, + { + "detail": "", + "status": "PASS", + "step": "clean host home" + }, + { + "detail": "Error: marketplace source `/private/marketplace` is not allowed by requirements from /etc/codex/requirements.toml", + "status": "FAIL", + "step": "marketplace" + }, + { + "detail": "9.5 step 5: starts a model; not run by this script", + "status": "UNVERIFIED", + "step": "start a clean session" + }, + { + "detail": "9.5 step 6: starts a model; not run by this script", + "status": "UNVERIFIED", + "step": "verify explicit invocation" + } + ] +}