Skip to content

refactor(hooks): inject the permission verdict so rewrite_cmd tests don't read ~/.claude/settings.json - #2636

Open
pidefrem wants to merge 1 commit into
rtk-ai:developfrom
pidefrem:test/rewrite-cmd-permission-isolation
Open

refactor(hooks): inject the permission verdict so rewrite_cmd tests don't read ~/.claude/settings.json#2636
pidefrem wants to merge 1 commit into
rtk-ai:developfrom
pidefrem:test/rewrite-cmd-permission-isolation

Conversation

@pidefrem

@pidefrem pidefrem commented Jun 26, 2026

Copy link
Copy Markdown

Problem

rewrite_cmd.rs's unattestable_passthrough tests call evaluate(cmd, …), which calls check_command(cmd)permissions::load_permission_rules() — reading the host's real Claude permission rules at test time. So 3 of 6 tests fail on any machine with ordinary deny/allow rules (a deny for rm -rf pre-empts the expected Passthrough; an allow for git status turns the expected Ask into Allow) while passing on default-config CI — "green on CI, red on my box."

Root cause + fix

evaluate mixes the permission verdict (I/O — reads host settings) with the rewrite/passthrough decision (pure logic over the command string), with no seam. This extracts evaluate_with_verdict(cmd, verdict, excluded, transparent) holding the existing body; evaluate becomes check_command(cmd) + delegation. Production behavior is byte-identical; tests inject an explicit PermissionVerdict (Ask neutral for the passthrough/rewrite branch; Deny/Allow to pin the verdict-dependent branches that were previously untestable without real rules). This mirrors the permission layer's own check_command_with_rules injection pattern.

Files

  • src/hooks/rewrite_cmd.rs — split out evaluate_with_verdict; test module injects the verdict.

Test plan

  • cargo fmt --all --check && cargo clippy --all-targets && cargo test
  • cargo test --bin rtk rewrite_cmd::tests::unattestable_passthrough — 8 pass on any machine regardless of ~/.claude/settings.json (was 3 failing locally).
  • +2 tests pin the verdict→outcome mapping: DenyRewriteOutcome::Deny (short-circuit), AllowRewriteOutcome::Allow(_).

Context

The exact bug (rewrite_cmd permission tests reading host settings) is unreported, but it is the same class of test-isolation defect upstream is already fixing for telemetry: #2106 (telemetry tests write a real .device_salt) and #2603 (adds telemetry test-isolation helpers — the same injection pattern). Filed as a sibling to that work; see also the related issue. Test-only, no production behavior change.


Closes #2640.

@iliaal

iliaal commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Independent reproduction on develop at f9d8c77.

Hit this today on a machine with "Bash(git status:*)" in ~/.claude/settings.json. check_command returns Allow, so evaluate yields Allow(_) where test_plain_command_still_rewrites and test_fd_dup_redirect_still_rewrites assert Ask(_).

Isolating HOME is enough to flip it, which pins the cause to the settings read rather than anything in the rewrite path. Same invocation, same commit, only the environment differs:

$ cargo test --bin rtk unattestable_passthrough
running 6 tests
test result: FAILED. 4 passed; 2 failed

$ HOME=$(mktemp -d) cargo test --bin rtk unattestable_passthrough
running 6 tests
test result: ok. 6 passed; 0 failed

I see 2 of the 3 failures you describe, not 3: none of my deny rules match rm -rf, so the substitution tests keep passing here. The Allow-flips-Ask half reproduces exactly as written.

I wrote this fix independently before finding your PR and converged on the same shape: evaluate_with_verdict(cmd, verdict, excluded, transparent_prefixes), Ask as the injected neutral verdict, plus a Deny short-circuit test. Applying it takes my full suite from 2 failed to 2497 passed, 0 failed. Dropping mine in favour of this one.

…ttings

The unattestable_passthrough tests called evaluate() → check_command(), which
reads the host's real Claude/rtk permission rules. On a machine with a deny rule
for 'rm -rf' or an allow rule for 'git status', the outcomes flipped (Deny/Allow
instead of the expected Passthrough/Ask), failing 3 tests locally while passing on
default-config CI. Extract evaluate_with_verdict() so tests inject the verdict
(the same seam check_command_with_rules already provides), making them
machine-independent. Adds explicit Deny/Allow verdict coverage now that it's testable.

Production behavior is unchanged: evaluate() still calls check_command(cmd) and
delegates. The split is a refactor, not a behavior change.
@pidefrem
pidefrem force-pushed the test/rewrite-cmd-permission-isolation branch from 42a1c18 to f8838fe Compare July 22, 2026 21:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(tests): rewrite_cmd permission tests read ~/.claude/settings.json — 3/6 fail on a normal dev machine

2 participants