Skip to content

Add #[mutants::exclude_re("pattern")] attribute#607

Open
sandersaares wants to merge 14 commits into
sourcefrog:mainfrom
sandersaares:exclude-re-attr
Open

Add #[mutants::exclude_re("pattern")] attribute#607
sandersaares wants to merge 14 commits into
sourcefrog:mainfrom
sandersaares:exclude-re-attr

Conversation

@sandersaares

@sandersaares sandersaares commented Apr 14, 2026

Copy link
Copy Markdown
Collaborator

Add a new attribute that excludes specific mutations by regex, without disabling all mutations on the item like #[mutants::skip] does.

Closes #551.

Behavior

  • Regex matches against the full mutant name (the same string shown by cargo mutants --list), using the same syntax as --exclude-re on the command line.
  • Scopes supported: functions, impl blocks, trait blocks, modules, files (as #![mutants::exclude_re(...)] inner attribute), and expressions that can carry an attribute (match, struct literals, calls, method calls, unary).
  • Patterns from outer scopes are inherited: if an impl block excludes a pattern, every method inside also excludes it, in addition to any patterns on the method itself. Patterns combine — a mutant matching any active pattern is excluded.
  • Multiple #[mutants::exclude_re(...)] attributes on the same item are all honoured.
  • Recognized inside cfg_attr, e.g. #[cfg_attr(test, mutants::exclude_re("..."))]. As with mutants::skip, the cfg condition is not evaluated; the inner attribute is always honoured.
  • A malformed attribute (missing/non-string/multiple arguments) or invalid regex is a hard error that names the offending source location and explains the failure. Only the first such error per run is surfaced, so the user gets a single actionable message.

Changes

  • mutants_attrs/src/lib.rs: new exclude_re proc-macro attribute (no-op, like skip); mutants crate bumped 0.0.4 → 0.0.5.
  • src/visit.rs: exclude_re_stack on DiscoveryVisitor with scoped push/pop helpers (in_exclude_re_scope), attrs_exclude_re_patterns for direct and cfg_attr-wrapped attributes, filtering in collect_mutant, and self.error.get_or_insert_with(...) to preserve the first error.
  • src/visit/test/: unit tests for every supported scope (function, impl, trait, mod, file, cfg_attr, outer-scope inheritance, malformed/invalid attribute reporting), and dedicated submodules for each expression-attribute kind (exclude_re_expr_call, _match, _method_call, _struct, _unary, and a _common module covering outer-scope inheritance into expressions and the skip > exclude_re precedence).
  • tests/main.rs + testdata/exclude_re_attr_two_invalid_regexes/: integration test exclude_re_attr_two_invalid_regexes_reports_only_the_first verifying end-to-end CLI behavior — the first malformed regex is reported with source location and the regex parser's explanation, and any subsequent malformed regex is silently suppressed.
  • book/src/attrs.md: documentation, including the mutants crate version requirement.
  • NEWS.md: changelog entry.

@sandersaares sandersaares marked this pull request as ready for review April 14, 2026 02:43
@sourcefrog sourcefrog self-assigned this Apr 16, 2026
@sourcefrog sourcefrog requested a review from Copilot April 16, 2026 14:34

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new #[mutants::exclude_re("...")] attribute to allow excluding only specific mutants (by regex) while still generating other mutations, including inheritance from outer scopes and support inside cfg_attr.

Changes:

  • Introduces the mutants::exclude_re proc-macro attribute (no-op at compile time) and documents it.
  • Implements exclude-by-regex behavior in the discovery visitor via an inherited scope stack, plus regex parsing from attributes (including cfg_attr).
  • Adds integration + fixture coverage (new testdata tree, new snapshot, and a new integration test).

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
mutants_attrs/src/lib.rs Adds the exclude_re proc-macro attribute entry point and docs.
src/visit.rs Implements exclude-re parsing, scope inheritance, and filtering during mutant collection; adds unit tests.
testdata/exclude_re_attr/src/lib.rs New fixture crate source exercising supported scopes.
testdata/exclude_re_attr/Cargo_test.toml New fixture crate manifest for the exclude-re attribute tests.
tests/main.rs Adds an integration test to snapshot --list output for the new fixture.
tests/util/snapshots/main__util__list_mutants_in_exclude_re_attr.snap Snapshot for the new integration test output.
book/src/attrs.md Documents the new attribute, usage, and scope inheritance rules.
NEWS.md Adds an “Unreleased” changelog entry for the new attribute.
Comments suppressed due to low confidence (1)

src/visit.rs:636

  • visit_item_mod pushes an exclude_re scope, but the early-return path when find_path_attribute reports an invalid (absolute) #[path] returns without popping. That leaves the exclude_re stack unbalanced and can incorrectly apply the module’s exclude patterns to the rest of the file. Pop the scope before returning (or use an RAII guard to ensure pop on all exits).
        if !self.push_exclude_re(&node.attrs) {
            return;
        }

        let source_location = Span::from(node.span());

        // Extract path attribute value, if any (e.g. `#[path="..."]`)
        let path_attribute = match find_path_attribute(&node.attrs) {
            Ok(path) => path,
            Err(path_attribute) => {
                let definition_site = self
                    .source_file
                    .format_source_location(source_location.start);
                error!(?path_attribute, ?definition_site, %mod_name, "invalid filesystem traversal in mod path attribute");
                return;
            }

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/visit.rs Outdated
Comment thread src/visit.rs Outdated
Comment thread src/visit.rs
Comment thread book/src/attrs.md Outdated
Comment thread testdata/exclude_re_attr/src/lib.rs Outdated
Comment thread mutants_attrs/src/lib.rs
Comment thread src/visit.rs Outdated
Comment thread src/visit.rs Outdated
Comment thread book/src/attrs.md Outdated
sandersaares and others added 6 commits May 11, 2026 14:35
Add a new attribute that excludes specific mutations by regex, without
disabling all mutations on the function like #[mutants::skip] does.

The attribute can be placed on functions, impl blocks, trait blocks,
modules, and files (as an inner attribute). Patterns from outer scopes
are inherited. Also supported within cfg_attr.

Closes sourcefrog#551
add_numbers used 'replace .* with ()' where () was a regex capture
group matching empty string, causing it to exclude ALL mutations
instead of just 'with ()'. Use r"with \(\)" instead.

subtract used 'replace .* with' which excluded everything. Use
'with 0' to demonstrate cfg_attr while keeping other mutations.
- mutants_attrs: bump version 0.0.4 -> 0.0.5 for the new exclude_re
  attribute.
- visit.rs: replace TokenStream::to_string() + substring scan in
  attr_mutants_exclude_re_pattern with AST-based parsing using
  Attribute::parse_args_with and Attribute::parse_nested_meta, mirroring
  attr_is_mutants_skip. Handles multiple exclude_re attrs in one
  cfg_attr, arbitrary whitespace, and other token shapes correctly.
- visit.rs: malformed #[mutants::exclude_re] (missing arg, multiple
  args, non-string arg) is now a hard error stored on visitor.error,
  matching the existing invalid-regex behaviour. Previously these were
  silently no-ops, giving users a false sense of safety.
- visit.rs: invalid-regex error now includes the source file/line of
  the offending attribute and the offending pattern, e.g.
  "src/main.rs:3:1: invalid regex in #[mutants::exclude_re(\"(unclosed\")]: ...".
- visit.rs: introduce in_exclude_re_scope closure helper that
  guarantees pop_exclude_re runs even if the visited body returns
  early. visit_item_mod previously left the exclude_re stack
  unbalanced on its invalid-#[path] early return; this also covers
  future early-return paths in visit_item_impl and friends.
- Tests: rewrite exclude_re_attr_filters_specific_mutants to actually
  exercise filtering on an i32 function (exclude "with 0", assert
  "with 1"/"with -1"/binop survive) - previously the regex with \(\)
  was a no-op on i32, so the test passed even if filtering was broken.
- Tests: strengthen exclude_re_attr_keeps_all_when_no_match to compare
  position-independent mutant names, not just lengths.
- Tests: add regression tests for malformed attribute forms and for
  the source-location in the error message.
- testdata/exclude_re_attr & book/src/attrs.md: switch the misleading
  with \(\) example to with 0 so it actually demonstrates the
  attribute working on an i32 function. Regenerate the integration
  snapshot, which now visibly shows with 0 filtered out of
  add_numbers.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@sandersaares

sandersaares commented May 11, 2026

Copy link
Copy Markdown
Collaborator Author

Addressed all feedback. Robot summary:

Owner feedback (@sourcefrog):

  • mutants_attrs version bumped 0.0.4 → 0.0.5.
  • Invalid-regex (and malformed-attribute) errors now include the source file/line and the bad pattern.
  • cfg_attr parsing rewritten from TokenStream::to_string() substring scan to proper AST traversal via Attribute::parse_nested_meta + syn::LitStr.
  • Misleading i32 + with () example in book/src/attrs.md replaced with "with 0".

Copilot reviewer feedback:

  • Fragile string scan replaced with AST parse (above).
  • Malformed #[mutants::exclude_re] (no arg / multiple args / non-string arg) is now a hard error instead of silent no-op, with 4 new unit tests.
  • Unit test that previously couldn't have detected a broken regex filter has been rewritten to exclude with 0 against -> i32 and now asserts that with 0 is gone while with 1, with -1, and the binop mutants survive.
  • Same misleading example fixed in book/src/attrs.md and testdata/exclude_re_attr/src/lib.rs; the snapshot main__util__list_mutants_in_exclude_re_attr.snap was regenerated and now visibly omits replace add_numbers -> i32 with 0.

Also addressed (from the suppressed low-confidence review comment about visit_item_mod): the early-return path on an invalid #[path] attribute could leave exclude_re_stack unbalanced. Fixed by introducing an in_exclude_re_scope(attrs, |v| { ... }) closure helper that does the push/pop around the visit body, and converting every visitor method (visit_file, visit_item_fn, visit_impl_item_fn, visit_trait_item_fn, visit_item_impl, visit_item_trait, visit_item_mod) to use it, so a future early return can't desync the stack.

Also added exclude_re_attr_keeps_all_when_no_match, strengthened to compare full mutant names — proves that a pattern matching nothing produces the exact same mutant set as no attribute at all, matching --exclude-re CLI behaviour.

@sourcefrog sourcefrog self-requested a review May 11, 2026 15:03
sandersaares and others added 3 commits May 20, 2026 16:10
The feature was already supported (via DiscoveryVisitor::visit_file
picking up the file's inner attributes) and had a unit test in
visit.rs, but the integration testdata tree didn't exercise it.

Add a sub-module file_scoped.rs that uses an inner attribute
#![mutants::exclude_re("replace .* -> bool")] and refresh the
list_mutants_in_exclude_re_attr snapshot to confirm the bool-returning
function's mutations are filtered out while the i32-returning function's
mutations remain.

Also fix a cargo fmt deviation introduced earlier on this branch in the
exclude_re_attr_keeps_all_when_no_match test.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Wrap the body of visit_expr_call, visit_expr_method_call, visit_expr_unary,
visit_expr_match, and visit_expr_struct in in_exclude_re_scope so that an
exclude_re attribute on such an expression suppresses mutants generated by
that expression and any expression nested within it. Patterns from enclosing
scopes are still inherited via the existing exclude_re stack.

Add a comment to visit_expr_binary explaining why no equivalent wrapping is
applied there: a #[mutants::exclude_re("...")] placed before a bare binary
expression like a + b is absorbed by syn into the leftmost operand's attrs
(e.g., the path expression) rather than into ExprBinary.attrs, so the field
is unreachable from current Rust source.

Tests are organised one file per expression type under src/visit/test/ and
declared as submodules of the existing inline mod test in src/visit.rs.

The exclude_re_expr_common.rs file also locks in the precedence rule that
mutants::skip wins over mutants::exclude_re on the same expression (even
when the regex is malformed) in both attribute orders, to guard against
regressions in future unification refactors.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@sandersaares sandersaares requested a review from Copilot May 27, 2026 03:39
@sandersaares

sandersaares commented May 27, 2026

Copy link
Copy Markdown
Collaborator Author

I noticed that #[mutants::skip] is actually parsed at many different scopes not just functions (as the book suggests) and that #[mutants::exclude_re] was missing support for expression-level parsing, differing from #[mutants::skip] in this regard. Added expression-level support now. Will make a 2nd PR to align the book with the implementation, as well.

sandersaares and others added 2 commits May 27, 2026 07:39
The previous wording (looks for ... within other attributes such as cfg_attr, without evaluating the outer attribute) was imprecise on two points:

- It implied that any wrapper attribute can contain mutants::exclude_re, when only cfg_attr is actually recognised.

- It described cfg_attr as 	he outer attribute and said it was not �valuated, leaving ambiguous whether the cfg *condition* was evaluated.

Reword to make both points unambiguous.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Resolve conflicts in NEWS.md, book/src/attrs.md, and src/visit.rs by keeping both the new mutants::exclude_re documentation/tests and the newly-added mutants::skip scope documentation/tests from main.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Comment thread tests/main.rs
.assert_insta("list_mutants_in_cfg_attr_test_skip_json");
}

#[test]

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Like in #618, it seems the integration test and test tree is mostly(?) duplicating things covered by the unit tests. I think I would rather keep the test collection relatively smaller and non-duplicative, both for the sake of making it easier to navigate and for test speed.

However, one thing I would think about adding at this level is a check that the error from a malformed regex comes out of the CLI.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adjusted as requested - unit test duplicating integration tests removed, added invalid-regex CLII integration test.

Comment thread book/src/attrs.md
(`x.foo(...)`), and unary expressions (`!x`, `-x`) — applies to the
expression and everything nested inside it.

## Excluding specific mutations with an attribute

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we caution here that you need mutants_attrs 0.5.0?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added version number notice.

Personally, I am of the opinion that "docs apply to latest version" is a given and such notes are mostly noise but I defer to your judgement.

Comment thread NEWS.md Outdated
@@ -4,6 +4,8 @@

- New: mutate `NonZero<T>` into `1`, and also `-1` when `T` is or may be signed.

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we also mention a new mutants_attrs 0.5.0 is needed for this feature?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added version number notice.

sandersaares and others added 3 commits June 7, 2026 11:11
…ail-fast test

The exclude_re_attr testdata tree and its list_mutants_in_exclude_re_attr integration test only exercised behavior that is already covered by the exclude_re_attr_* unit tests in src/visit.rs (function/impl/trait/mod/file scopes, cfg_attr, outer-scope inheritance), so they added no signal.

Keep one integration test that can only be verified end-to-end: exclude_re_attr_two_invalid_regexes_reports_only_the_first asserts the CLI surfaces the first malformed regex with source location and the regex parser's explanation, and silently drops any subsequent malformed regex, locking in the `preserve the first error` semantics of DiscoveryVisitor::push_exclude_re.

Also note in book/src/attrs.md and NEWS.md that `mutants::exclude_re` requires the `mutants` crate version 0.0.5 or later.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The merge with upstream/main brought in the released 27.1.0 NEWS section that already contained three of the four bullets that were previously under our local `## Unreleased` (NonZero<T>, skip docs clarification, reflinks mtime). Git auto-merged the remaining `exclude_re` bullet under 27.1.0, but that release does not actually include the exclude_re attribute. Reinstate `## Unreleased` above 27.1.0 holding only the exclude_re entry.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature request: define regex excludes via attributes in code

3 participants