fix(reviewer): gate auth-prompt detection on absence of review structure - #333
Conversation
…ructure Adopt agy's own adversarial review finding (flagged BLOCKING on the sync PRs): the initial is_auth_prompt() matched the OAuth-login signature anywhere in agy's output, so a GENUINE review that quotes those strings -- e.g. one reviewing this very script (whose diff contains them) or a PR implementing OAuth -- would be wrongly discarded in the retry loop or blocked from posting. Require BOTH conditions now: the auth signature present AND the mandated review structure absent. The reviewer prompt requires a "### Blocking issues" section in every review (agy writes "None found." when clean), which agy's raw interactive login flow never contains. A real review thus keeps its section and posts; only the raw login flow (signature present, structure absent) is rejected. Verified against six cases incl. the exact false-positive agy flagged (a review quoting the auth strings WITH a Blocking-issues section now posts). bash -n clean; byte-identical across the template and all three Rusty* repos. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Warning Review limit reached
Next review available in: 2 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
Comment |
Adopt agy's second-round BLOCKING finding: the review-structure check
used an unanchored, case-insensitive `grep -qi 'Blocking issues'`, so the
mere phrase appearing in prose or an error line -- alongside an OAuth URL
-- would disarm is_auth_prompt() and let the login flow post (a false
NEGATIVE = a leak).
Anchor to the Markdown section header at line start (`^#{2,4} Blocking
issues`, case-sensitive) instead. This fails SAFE: text without a genuine
header is treated as a possible login flow and blocked, while a real
review's mandated "### Blocking issues" header always matches, so a
review that quotes the auth strings still posts.
Verified against seven cases, incl. the exact bypass agy described (a raw
login flow containing the phrase "blocking issues" in prose + an OAuth
URL is now correctly blocked) and a lowercase-phrase-plus-URL log line.
bash -n clean; byte-identical across template + all three Rusty* repos.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Antigravity review (Gemini via Ultra)This PR updates Blocking issues
Suggestions
Nitpicks
Automated first-pass review by |
…tructure
Redesign converging several rounds of agy's own adversarial review, which
oscillated between the guard being too broad (false positive: suppressing
a genuine review that discusses auth or quotes this script) and too narrow
(false negative: an OAuth URL leaking when a section header appeared in the
same output). Both failure modes came from trying to answer "is this a
review?" heuristically.
Key on the one artifact that unambiguously marks agy's login flow instead:
a live Google OAuth authorization URL (the https scheme + host + /o/oauth2/
endpoint path).
* oauth_url_present() -- Layer 3, now UNCONDITIONAL: refuse to post any
body carrying such a URL, with no header/structure exemption an
interleaved URL could disarm (fixes the round-3 leak finding; also
immune to header case and trailing punctuation).
* is_auth_prompt() -- Layer 1: a lapsed session is detected by the live
URL (always present in the flow) plus agy's terminal failure line; it
gates only the retry / clean-exit UX, never the security guarantee.
A code review quotes this script's BARE regex (host + path, no scheme) or
discusses auth in prose -- neither is a live URL -- so reviewing the
reviewer or an OAuth-related PR no longer false-positives.
Verified against seven cases covering every scenario agy raised across
three rounds: raw login flow (blocked), a review quoting the bare regex
(posts), a title-case header with auth prose (posts), a header+colon with
an interleaved live URL (blocked), a failure-line-only lapse, empty, and a
real review body. bash -n clean; byte-identical across template + all
three Rusty* repos.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Review adjudication (all rounds) — resolved via redesignThanks to Redesign — key the guard on the live OAuth URL itself, not on review structure:
A genuine review quotes this script's bare regex (host + path, no scheme) or discusses auth in prose — neither is a live URL — so reviewing the reviewer or an OAuth-related PR no longer false-positives (fixes round 1). Confirmed the script does not self-match. Verification — 7 cases, each mapping to a scenario raised across the three rounds:
Not adopted, with rationale:
|
…llback
Adopt agy's round-4 findings on the OAuth-URL leak guard:
* OAUTH_URL_RE no longer requires a trailing slash after `/o/oauth2`. A
URL emitted as `.../o/oauth2?client_id=...` or bare `.../o/oauth2` would
otherwise slip past oauth_url_present and leak. The scheme (`https?://`)
is still required, so a review quoting this script's bare host/path
pattern still does not match -- no safety lost. Matching is also made
case-insensitive (URL scheme/host are case-insensitive; still self-safe).
* Removed the secondary `authentication failed or timed out` substring
fallback and collapsed both layers onto the single oauth_url_present
check. agy flagged the fallback (correctly) as broad enough to abort a
genuine review that merely discusses auth-timeout handling. The live URL
is always present in a real login flow, so it is both necessary and
sufficient on its own; the same unconditional check now serves Layer 1
(retry / clean-exit) and Layer 3 (block the post).
Verified: no-slash and query-only URLs are now blocked; a review quoting
the bare pattern or discussing auth timeouts posts normally; the script
does not self-match. bash -n clean; byte-identical across template + all
three Rusty* repos.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Round-4 findings adoptedBoth round-4 blocking items on the OAuth-URL guard were valid and are fixed:
The guard is now one case-insensitive grep for a live OAuth authorization URL, applied unconditionally at both the retry-loop capture and the assembled body. Re-verified (no-slash / query-only URLs blocked; bare-pattern quote and auth discussion post; no self-match). The |
Convergence note (RustyNES) — why this PR's own agy run fails, and why that's expectedThe reviewer track has converged: the sibling sync PRs ( This PR's own agy run fails, and that is a bootstrap artifact, not a defect. #333's reviewer executes the script on RustyNES Stale/adjudicated items in the comment above:
Recommended merge order: #333 first (fixes RustyNES |
Summary
Follow-up to #332 (the OAuth-login-prompt guard). Adopts a blocking finding from the
agyreviewer's own adversarial review of the sibling sync PRs (doublegate/RustySNES#259,doublegate/RustyN64#207): the initialis_auth_prompt()matched the auth signature anywhere in agy's output, so a genuine review that quotes those strings — one reviewing this very script (whose diff contains them), or a PR implementing OAuth — would be wrongly discarded in the retry loop (Layer 1) or blocked from posting (Layer 3).Fix
is_auth_prompt()now requires both:### Blocking issuessection in every review (agy writes "None found." when clean), which agy's raw interactive login flow never contains.So a real review keeps its
Blocking issuessection and posts; only the raw login flow (signature present, structure absent) is rejected. On a healthy runner, behavior is unchanged.Verification
Six cases, all pass:
### Blocking issueshave_text) ✓bash -nclean. Byte-identical (same sha256) with the template and the RustySNES/RustyN64 sync PRs.🤖 Generated with Claude Code