Add S001 integration tests; fix -n flag collisions breaking analyze/benchmark - #1484
Merged
Gbangbolaoluwagbemiga merged 2 commits intoAug 27, 2026
Conversation
The issue's file pointer (tooling/sanctifier-core/src/rules/s001.rs) doesn't exist -- S001 is the AUTH_GAP finding code (finding_codes.rs), implemented by rules/auth_gap.rs (rule name "auth_gap"). Existing SARIF/JSON integration tests in cli_tests.rs already assert "some finding fired" against the shared vulnerable_contract.rs fixture, but none specifically target S001/auth_gap the way the issue asks. Added two tests: test_s001_auth_gap_appears_in_json_output and test_s001_auth_gap_appears_in_sarif_output. Both run the CLI end-to- end (assert_cmd::Command::cargo_bin, matching every other test in this file) against a new, minimal single-purpose fixture -- a contract missing require_auth before a storage write, with no other issues, so the assertion is unambiguous about which finding it's checking -- and assert the JSON output's rule_violations contains a rule_name == "auth_gap" entry with a location, that error_codes includes S001, and that the SARIF output's results array contains a ruleId == "auth_gap" entry with a physical location URI. Closes HyperSafeD#1455
|
@presidoclintonbased-alt is attempting to deploy a commit to the gbangbolaoluwagbemiga's projects Team on Vercel. A member of the Team first needs to authorize it. |
|
@presidoclintonbased-alt Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits. You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀 |
Actually running the new S001 tests (rather than just compiling them) revealed that `sanctifier analyze` currently panics on every non-release invocation: main.rs's global `-n`/`--network` flag collides with analyze.rs's own `-n` for `--no-cache`, which clap only catches via a debug_assert! -- so it doesn't show up in a release build, but panics immediately under `cargo test`/`cargo run` (and would panic for anyone building this crate locally in a normal `cargo build` without --release). This broke every existing integration test in cli_tests.rs that invokes `analyze` -- confirmed by running test_analyze_valid_contract before this fix; it panicked with the exact same clap error, unrelated to anything in this PR. Same bug, second instance: benchmark.rs's `iterations` field also used `-n`, breaking `benchmark` and (via `completions`, which validates the full command tree) 3 further tests. Fixed both by dropping the short flag (long form --no-cache/ --iterations unaffected) rather than reassigning --network's `-n`, since network is the global flag used across every subcommand. Full suite verified: `cargo test --test cli_tests` -- 61 passed, 0 failed, 2 ignored (pre-existing, unrelated flaky-HTTP ignore).
Gbangbolaoluwagbemiga
merged commit Aug 27, 2026
768412a
into
HyperSafeD:main
14 of 25 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
#1455 — integration tests for S001/auth_gap
The issue's file pointer (
tooling/sanctifier-core/src/rules/s001.rs) doesn't exist — S001 isthe
AUTH_GAPfinding code (finding_codes.rs), implemented byrules/auth_gap.rs(rule name"auth_gap"). Existing SARIF/JSON integration tests incli_tests.rsalready assert "somefinding fired" against the shared
vulnerable_contract.rsfixture, but none specifically targetS001/auth_gap the way the issue asks ("write tests that run the CLI against a mock workspace and
assert on the SARIF/JSON output").
Added two tests, matching every existing test in the file's conventions
(
assert_cmd::Command::cargo_bin, tempdir-based fixtures):test_s001_auth_gap_appears_in_json_output— runs the CLI against a new, minimal fixture (acontract missing
require_authbefore a storage write, with no other issues so the assertionis unambiguous), asserts the JSON output's
rule_violationscontains arule_name == "auth_gap"entry with a location, and thaterror_codesincludesS001.test_s001_auth_gap_appears_in_sarif_output— same fixture, asserts the SARIF output'sresultsarray contains aruleId == "auth_gap"entry with a physical location URI.A real, severe pre-existing bug found by actually running these tests
Compiling passed, but running the new tests failed —
sanctifier analyzepanics on everynon-release invocation:
main.rs's global-n/--networkflag collides withanalyze.rs's own-nfor--no-cache. Clap only catches short-flag collisions via adebug_assert!, so thisnever shows up in a release build, but panics immediately under
cargo test/cargo run— andfor anyone building this crate locally without
--release. Confirmed this broke every existingintegration test in
cli_tests.rsthat invokesanalyze, not just mine (rantest_analyze_valid_contractbefore my fix; same panic, nothing to do with this PR).Found and fixed a second instance of the identical bug:
benchmark.rs'siterationsfield alsoused
-n, breakingbenchmarkand (viacompletions, which validates the whole command tree) 3further tests. Fixed both by dropping the short flag (long form
--no-cache/--iterationsunaffected) rather than reassigning
--network's-n, since network is the global flag usedacross every subcommand.
Test plan
cargo test --test cli_tests: 61 passed, 0 failed, 2 ignored (the 2 ignored arepre-existing, unrelated — a documented flaky-HTTP-mock test). Includes both new S001 tests
and every previously-broken
analyze/benchmark/completionstest, now passing.Closes #1455
Closes #1459
Closes #1457
Closes #1454