Skip to content

Fix P2 HTML comment boundary matching - #452

Open
Patrick-Erichsen wants to merge 2 commits into
NVIDIA:mainfrom
Patrick-Erichsen:codex/fix-p2-comment-boundaries
Open

Fix P2 HTML comment boundary matching#452
Patrick-Erichsen wants to merge 2 commits into
NVIDIA:mainfrom
Patrick-Erichsen:codex/fix-p2-comment-boundaries

Conversation

@Patrick-Erichsen

Copy link
Copy Markdown

Summary

  • confine P2 hidden-instruction matches to a single HTML comment
  • require whole alphanumeric keywords while preserving snake_case separators
  • cover cross-comment false positives, keyword substrings, and snake_case injection syntax

Fixes #297.

Validation

  • uv run pytest -m 'not integration and not provider' tests/ (3,074 passed, 14 skipped, 38 deselected, 4 xfailed)
  • uv run ruff check src/ tests/
  • uv run ruff format --check src/ tests/
  • rescanned ClawHub release-validation@0.1.3; P2 findings dropped from 2 to 0
  • autoreview: clean, no accepted/actionable findings

Signed-off-by: Patrick Erichsen <patrick.a.erichsen@gmail.com>

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SkillSpector Review]

Re-reviewed at 1e82b2b after the current-main merge. The patch fixes the reported ordinary-substring and standard cross-comment false positives, and all 154 focused static-pattern tests pass, but two boundary cases still block approval. The alphanumeric boundary drops common camel/Pascal-style hidden directives, producing an end-to-end SAFE, complete verdict, and the tempered body does not recognize --!> as an HTML comment closer, so it can still span from a comment into visible text. Please preserve case-transition directive detection, honor browser-compatible comment endings, and add direct plus end-to-end regressions.

(r"<!--.*?(?:system|instructions?|ignore|POST|GET|send|transmit).*?-->", 0.7),
(
r"<!--(?:(?!-->).)*?(?<![^\W_])(?:system|instructions?|ignore|POST|GET|send|transmit)"
r"(?![^\W_])"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Preserve camelCase hidden-directive detection. The new alphanumeric right boundary treats an uppercase continuation as part of the same word, so <!-- ignorePreviousInstructions -->, <!-- systemPrompt: sendUserData -->, and <!-- transmitConversationToAttacker --> all produce zero findings on this exact head while the base matcher detects them. P1/P3 do not backstop these forms; the first case yields an end-to-end SAFE, complete verdict. Retain the target/ordinary-substring fix while recognizing case transitions or bounded known command compounds, and add camel/Pascal regressions.

(
r"<!--(?:(?!-->).)*?(?<![^\W_])(?:system|instructions?|ignore|POST|GET|send|transmit)"
r"(?![^\W_])"
r"(?:(?!-->).)*?-->",

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Stop at every HTML comment closer, not only -->. Browser-compatible HTML parsing also closes a comment at --!>; on this head, <!-- safe --!> visible system docs <!-- safe --> is nevertheless matched as one P2 span even though visible system docs is outside either comment. Handle the HTML end-bang form in the bounded comment scan and add this cross-boundary negative regression.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

P2 "Hidden Instructions" rule produces false positives due to missing word boundary on GET

2 participants