Summary
#405 was one instance of a systemic problem: Python's \s matches newlines, so every \s+/\s* in a static pattern can silently bridge a paragraph boundary (blank line) and stitch together two unrelated sentences. #417 fixed the one reported regex. The disease is everywhere else:
- 1,485 occurrences of
\s+/\s* across all 15 static_patterns_*.py files (counted on current main).
- All of them run via
re.finditer(pattern, content, re.IGNORECASE | re.MULTILINE) over the whole file content, so nothing stops a match from spanning paragraphs.
Reproducible example, live on main today
The pattern directly below the one #417 fixed, in the same EA1 list:
(r"(?:allow|grant|enable)\s+(?:access\s+to\s+)?(?:all|any|every)\s+tools?", 0.8)
against this benign markdown:
lists the features you enable
Any tools mentioned below require review.
matches enable\n\nAny tools → EA1/0.8 false positive bridging two unrelated paragraphs. Every prose-style pattern in all 15 files has the same exposure.
Why per-pattern fixes are the wrong tool
Replacing \s+ with [ \t]+ per pattern is wrong for prose: a sentence soft-wrapped mid-phrase (allow access to\nall tools) is one sentence and should match. The bug is only crossing blank lines (paragraph boundaries). Hand-tuning 1,485 quantifiers case-by-case is unmaintainable whack-a-mole — #405/#417 was one mole.
Proposed direction (one fix, all 15 files)
Fix it once in the shared runner instead of in every pattern. Options, in rough order of preference:
- Split content into paragraphs (
re.split(r"\n[ \t]*\n", content)) in the shared matching loop and run patterns per paragraph, offsetting line numbers. No pattern changes at all. Single-line config patterns (like EA1's wildcard) are unaffected.
- Rewrite quantifiers at compile time: a small helper that transforms
\s+/\s* into (?:[^\S\n]|\n(?![ \t]*\n))+ (whitespace, but a newline only when not followed by a blank line) before re.compile. Keeps whole-content matching for anything that legitimately needs it.
- Per-pattern audits — only as a last resort, for patterns that intentionally span paragraphs (none found so far).
Option 1 is the smallest diff and the easiest to reason about; it also makes matched text/line attribution cleaner (no more 'enable\n\nAny tools' spans in reports).
Acceptance criteria
- The example above no longer fires.
- Existing single-line and same-paragraph positive tests still pass unchanged.
- A regression test asserts that no static pattern can match across a blank line.
Summary
#405 was one instance of a systemic problem: Python's
\smatches newlines, so every\s+/\s*in a static pattern can silently bridge a paragraph boundary (blank line) and stitch together two unrelated sentences. #417 fixed the one reported regex. The disease is everywhere else:\s+/\s*across all 15static_patterns_*.pyfiles (counted on currentmain).re.finditer(pattern, content, re.IGNORECASE | re.MULTILINE)over the whole file content, so nothing stops a match from spanning paragraphs.Reproducible example, live on main today
The pattern directly below the one #417 fixed, in the same EA1 list:
against this benign markdown:
matches
enable\n\nAny tools→ EA1/0.8 false positive bridging two unrelated paragraphs. Every prose-style pattern in all 15 files has the same exposure.Why per-pattern fixes are the wrong tool
Replacing
\s+with[ \t]+per pattern is wrong for prose: a sentence soft-wrapped mid-phrase (allow access to\nall tools) is one sentence and should match. The bug is only crossing blank lines (paragraph boundaries). Hand-tuning 1,485 quantifiers case-by-case is unmaintainable whack-a-mole — #405/#417 was one mole.Proposed direction (one fix, all 15 files)
Fix it once in the shared runner instead of in every pattern. Options, in rough order of preference:
re.split(r"\n[ \t]*\n", content)) in the shared matching loop and run patterns per paragraph, offsetting line numbers. No pattern changes at all. Single-line config patterns (like EA1's wildcard) are unaffected.\s+/\s*into(?:[^\S\n]|\n(?![ \t]*\n))+(whitespace, but a newline only when not followed by a blank line) beforere.compile. Keeps whole-content matching for anything that legitimately needs it.Option 1 is the smallest diff and the easiest to reason about; it also makes matched text/line attribution cleaner (no more
'enable\n\nAny tools'spans in reports).Acceptance criteria