Skip to content

Add S001 integration tests; fix -n flag collisions breaking analyze/benchmark - #1484

Merged
Gbangbolaoluwagbemiga merged 2 commits into
HyperSafeD:mainfrom
presidoclintonbased-alt:fix/s001-integration-tests
Aug 27, 2026
Merged

Add S001 integration tests; fix -n flag collisions breaking analyze/benchmark#1484
Gbangbolaoluwagbemiga merged 2 commits into
HyperSafeD:mainfrom
presidoclintonbased-alt:fix/s001-integration-tests

Conversation

@presidoclintonbased-alt

@presidoclintonbased-alt presidoclintonbased-alt commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Summary

#1455 — integration tests for S001/auth_gap
The issue's file pointer (tooling/sanctifier-core/src/rules/s001.rs) doesn't exist — S001 is
the AUTH_GAP finding code (finding_codes.rs), implemented by rules/auth_gap.rs (rule name
"auth_gap"). Existing SARIF/JSON integration tests in cli_tests.rs already assert "some
finding fired" against the shared vulnerable_contract.rs fixture, but none specifically target
S001/auth_gap the way the issue asks ("write tests that run the CLI against a mock workspace and
assert on the SARIF/JSON output").

Added two tests, matching every existing test in the file's conventions
(assert_cmd::Command::cargo_bin, tempdir-based fixtures):

  • test_s001_auth_gap_appears_in_json_output — runs the CLI against a new, minimal fixture (a
    contract missing require_auth before a storage write, with no other issues so the assertion
    is unambiguous), asserts the JSON output's rule_violations contains a rule_name == "auth_gap" entry with a location, and that error_codes includes S001.
  • test_s001_auth_gap_appears_in_sarif_output — same fixture, asserts the SARIF output's
    results array contains a ruleId == "auth_gap" entry with a physical location URI.

A real, severe pre-existing bug found by actually running these tests

Compiling passed, but running the new tests failed — sanctifier analyze panics on every
non-release invocation: main.rs's global -n/--network flag collides with analyze.rs's own
-n for --no-cache. Clap only catches short-flag collisions via a debug_assert!, so this
never shows up in a release build, but panics immediately under cargo test/cargo run — and
for anyone building this crate locally without --release. Confirmed this broke every existing
integration test in cli_tests.rs that invokes analyze
, not just mine (ran
test_analyze_valid_contract before my fix; same panic, nothing to do with this PR).

Found and fixed a second instance of the identical bug: benchmark.rs's iterations field also
used -n, breaking benchmark and (via completions, which validates the whole command tree) 3
further tests. Fixed both by dropping the short flag (long form --no-cache/--iterations
unaffected) rather than reassigning --network's -n, since network is the global flag used
across every subcommand.

Test plan

  • cargo test --test cli_tests: 61 passed, 0 failed, 2 ignored (the 2 ignored are
    pre-existing, unrelated — a documented flaky-HTTP-mock test). Includes both new S001 tests
    and every previously-broken analyze/benchmark/completions test, now passing.

Closes #1455
Closes #1459
Closes #1457
Closes #1454

The issue's file pointer (tooling/sanctifier-core/src/rules/s001.rs)
doesn't exist -- S001 is the AUTH_GAP finding code (finding_codes.rs),
implemented by rules/auth_gap.rs (rule name "auth_gap"). Existing
SARIF/JSON integration tests in cli_tests.rs already assert "some
finding fired" against the shared vulnerable_contract.rs fixture, but
none specifically target S001/auth_gap the way the issue asks.

Added two tests: test_s001_auth_gap_appears_in_json_output and
test_s001_auth_gap_appears_in_sarif_output. Both run the CLI end-to-
end (assert_cmd::Command::cargo_bin, matching every other test in
this file) against a new, minimal single-purpose fixture -- a
contract missing require_auth before a storage write, with no other
issues, so the assertion is unambiguous about which finding it's
checking -- and assert the JSON output's rule_violations contains a
rule_name == "auth_gap" entry with a location, that error_codes
includes S001, and that the SARIF output's results array contains a
ruleId == "auth_gap" entry with a physical location URI.

Closes HyperSafeD#1455
@vercel

vercel Bot commented Aug 27, 2026

Copy link
Copy Markdown

@presidoclintonbased-alt is attempting to deploy a commit to the gbangbolaoluwagbemiga's projects Team on Vercel.

A member of the Team first needs to authorize it.

@drips-wave

drips-wave Bot commented Aug 27, 2026

Copy link
Copy Markdown

@presidoclintonbased-alt Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

Actually running the new S001 tests (rather than just compiling them)
revealed that `sanctifier analyze` currently panics on every
non-release invocation: main.rs's global `-n`/`--network` flag
collides with analyze.rs's own `-n` for `--no-cache`, which clap only
catches via a debug_assert! -- so it doesn't show up in a release
build, but panics immediately under `cargo test`/`cargo run` (and
would panic for anyone building this crate locally in a normal `cargo
build` without --release). This broke every existing integration
test in cli_tests.rs that invokes `analyze` -- confirmed by running
test_analyze_valid_contract before this fix; it panicked with the
exact same clap error, unrelated to anything in this PR.

Same bug, second instance: benchmark.rs's `iterations` field also
used `-n`, breaking `benchmark` and (via `completions`, which
validates the full command tree) 3 further tests.

Fixed both by dropping the short flag (long form --no-cache/
--iterations unaffected) rather than reassigning --network's `-n`,
since network is the global flag used across every subcommand.

Full suite verified: `cargo test --test cli_tests` -- 61 passed, 0
failed, 2 ignored (pre-existing, unrelated flaky-HTTP ignore).
@presidoclintonbased-alt presidoclintonbased-alt changed the title Add S001 (auth_gap) integration tests Add S001 integration tests; fix -n flag collisions breaking analyze/benchmark Aug 27, 2026
@Gbangbolaoluwagbemiga
Gbangbolaoluwagbemiga merged commit 768412a into HyperSafeD:main Aug 27, 2026
14 of 25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants