Skip to content

fix: exact re-reviews leave stale status labels - #1236

Open
goutamadwant wants to merge 3 commits into
openclaw:mainfrom
goutamadwant:fix/exact-review-label-reconciliation
Open

fix: exact re-reviews leave stale status labels#1236
goutamadwant wants to merge 3 commits into
openclaw:mainfrom
goutamadwant:fix/exact-review-label-reconciliation

Conversation

@goutamadwant

@goutamadwant goutamadwant commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Closes #1226

What Problem This Solves

Fixes exact pull request re-reviews publishing a current maintainer-ready verdict while stale managed status and rating labels remain because the re-review command advanced activity after the stored snapshot.

Why This Change Was Made

Completed current-head reviews may reconcile labels only when the source, timeline, head, and review-activity receipts still match. The apply path also requires a complete activity read with no human activity in or after the review-completion second, preserving maintainer label changes that are intentionally excluded from receipt hashes. Legacy reports remain fail closed for human comments.

User Impact

Exact re-reviews reconcile stale proof, rating, status, and merge-risk labels when the requesting command is covered by the completed review. Newer or same-second human activity blocks label mutation until another review completes.

OpenClaw Bay Impact

OpenClaw Bay is unaffected. This changes GitHub managed-label reconciliation only and does not change Bay data, routes, observer contracts, or controls.

Documentation Impact

Adds a focused proof package under docs/proof/exact-review-label-reconciliation/ documenting the exercised runtime path, expected mutation, and limits.

Finding Disposition

  • Preserved the post-snapshot human-activity guard: complete reviews require matching source, timeline, head, and activity-cursor receipts plus a fully hydrated human-activity check.
  • Preserved same-second maintainer activity: the complete-review path treats the entire reviewed_at second as ambiguous and blocks reconciliation.
  • Exercised the complete-identity path in both positive and negative integration scenarios.

Validation

  • pnpm run build:all
  • node --test --test-name-pattern='human comment activity|exact publication syncs fresh-head PR labels|skips fresh-head PR label sync when humans act|exact publication withholds fresh-head PR label sync from close proposals' test/apply-label-sync.test.ts (4 passed)
  • pnpm run check:static
  • pnpm run lint
  • bash docs/proof/exact-review-label-reconciliation/run-proof.sh (3 passed)
  • git diff --check

Real Behavior Proof

Claim: Exact publication of a completed, current-head pull request review reconciles stale managed labels when the requesting command is covered by the completed review, while later or same-second human activity still blocks reconciliation.

Exercised surface: The built dist/clawsweeper.js apply-decisions command, exact-event publication, report-owned review lease handling, full pull request context hydration through a local GitHub CLI transport, batched label mutation, durable review comment publication, and lease cleanup.

Scenario: The controlled pull request begins with status: needs proof and rating: silver shellfish. A command requests re-review. Matching complete-review receipts for the unchanged 40-character head select sufficient proof, the current rating, maintainer-ready status, and a merge-risk label. The adjacent negative scenario applies a maintainer-managed label in the same timestamp second as reviewed_at while receipt hashes remain equal.

Observed result: The positive scenario emitted one atomic issue edit mutation adding the current proof, rating, status, and merge-risk labels, removing both stale labels, and patching the durable review comment. The receipt-matched same-second maintainer activity scenario emitted no label mutation. The exact close-proposal regression also emitted no label mutation.

Artifact or trace: docs/proof/exact-review-label-reconciliation/README.md and docs/proof/exact-review-label-reconciliation/run-proof.sh.

Limits: The transport is local and deterministic. It exercises the shipped command and mutation boundary without contacting or mutating GitHub's hosted API.

Disclosure: AI was used to understand the codebase and review the fix.

@clawsweeper

clawsweeper Bot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

@clawsweeper clawsweeper Bot added merge-risk: 🚨 automation 🚨 Merging this PR could break CI, automerge, proof capture, label sync, or automation. P1 Urgent regression or broken agent/channel workflow affecting real users now. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Aug 24, 2026
@clawsweeper

clawsweeper Bot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed August 31, 2026, 7:06 AM ET / 11:06 UTC.

ClawSweeper review

What this changes

The PR lets exact review publication replace stale managed pull-request labels only when the completed review’s source, timeline, head, and activity receipts still match and no same-second-or-later human activity exists.

Regression provenance

Possible regression — probable (reviewed change; known regression link). No predecessor PR is attributed.

Merge readiness

Blocked until real behavior proof from a real setup is added - 3 items remain

Keep open: the receipt-gated freshness change is a focused fix for the linked stale-label bug, but the supplied validation remains a local mocked GitHub CLI scenario and does not meet the required real-behavior proof gate.

Priority: P1
Reviewed head: cff385b0c9a6c00fc8b261a6f40a68cc5c2b0f4b

Review scores

Measure Result What it means
Overall readiness 🦪 silver shellfish (2/6) The patch is focused and has meaningful mock-backed regression coverage, but it remains below merge readiness until real behavior evidence covers the automated mutation boundary.
Proof confidence 🦪 silver shellfish (2/6) Needs real behavior proof before merge: The changed production owner is the apply lane’s label-freshness gate, exercised through built-command integration fixtures with a scripted local GitHub CLI transport; those fixtures observe the intended mutation and no-op, but the added documentation explicitly excludes hosted GitHub API contact and no inspectable authorized real-transport trace is supplied. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Patch quality 🐚 platinum hermit (4/6) No actionable review findings were identified.

Verification

Check Result Evidence
Real behavior Needs proof Needs real behavior proof before merge: The changed production owner is the apply lane’s label-freshness gate, exercised through built-command integration fixtures with a scripted local GitHub CLI transport; those fixtures observe the intended mutation and no-op, but the added documentation explicitly excludes hosted GitHub API contact and no inspectable authorized real-transport trace is supplied. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Evidence reviewed 5 items Introduced freshness guard: The PR adds a receipt-matching branch for completed current-head reviews, then rejects any non-automation activity in or after the review-completion second before labels can sync.
Current-main comparison: Current main still has the prior generic label-freshness ordering, so this fix is not already implemented on main.
Focused regression coverage: The introduced integration fixtures cover reconciliation after a command-only re-review, same-second maintainer activity, and the close-proposal exclusion.
Findings None None.
Security None None.

How this fits together

ClawSweeper’s apply lane reads a completed review report and current pull-request activity before synchronizing the durable review comment and managed GitHub labels. This freshness gate determines whether automated label reconciliation may reach the GitHub mutation step.

flowchart LR
  A[Completed review report] --> C[Freshness gate]
  B[Current pull request activity] --> C
  C --> D{Receipts and activity valid?}
  D -->|yes| E[Managed label reconciliation]
  D -->|no| F[Leave labels unchanged]
  E --> G[Durable review comment]
Loading

Before merge

  • Add real behavior proof - Needs real behavior proof before merge: The changed production owner is the apply lane’s label-freshness gate, exercised through built-command integration fixtures with a scripted local GitHub CLI transport; those fixtures observe the intended mutation and no-op, but the added documentation explicitly excludes hosted GitHub API contact and no inspectable authorized real-transport trace is supplied. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
  • Resolve merge risk (P1) - Merging broadens an automated GitHub label-mutation path; the local fixture proves the intended request shape but not an authorized real-transport after-fix reconciliation or its live activity-hydration safeguards.
  • Complete next step (P2) - The remaining blocker is contributor-supplied real-behavior proof, not a bounded repair for ClawSweeper’s automated fix lane.
Agent review details

Security

None.

Review metrics

Metric Value Why it matters
Changed surface production +14/-2, tests +177/-81, proof docs +64/-0 The small production gate is accompanied by focused integration fixtures and a documented proof contract.

Root-cause cluster

Relationship: fixed_by_candidate
Canonical: #1226
Summary: This PR is the explicit candidate fix for the open stale exact-review label-reconciliation report.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge-risk options

Maintainer options:

  1. Provide real transport evidence (recommended)
    Before merge, attach a redacted current-head trace from the authorized GitHub transport path showing the intended label mutation and a same-second or later human-activity rejection.

Technical review

Best possible solution:

Keep the receipt-first reconciliation design and add redacted, current-head Crabbox local-container evidence from an authorized real GitHub transport path showing both the successful stale-label transition and the human-activity no-op.

Do we have a high-confidence way to reproduce the issue?

Yes, source-reproducible: current main uses the generic freshness path, while the introduced branch adds the completed-review receipt and activity boundary needed for the linked exact-publication scenario. A live reproduction was not run in this read-only review.

Is this the best way to solve the issue?

Yes, conditionally: the receipt-first gate is the narrowest maintainable way to distinguish a reviewed re-review command from later human activity, provided real-transport proof confirms the actual apply boundary.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 206fe726bbad.

Labels

Label justifications:

  • P1: Stale maintainer-routing labels can leave a ready pull request incorrectly shown as waiting on its contributor.
  • merge-risk: 🚨 automation: The change alters the conditions under which ClawSweeper automatically mutates managed GitHub labels.
  • rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦪 silver shellfish and patch quality is 🐚 platinum hermit.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The changed production owner is the apply lane’s label-freshness gate, exercised through built-command integration fixtures with a scripted local GitHub CLI transport; those fixtures observe the intended mutation and no-op, but the added documentation explicitly excludes hosted GitHub API contact and no inspectable authorized real-transport trace is supplied. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Evidence

What I checked:

Likely related people:

  • PollyBot13: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)
  • Peter Steinberger: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

  • Attach redacted real-transport after-fix evidence for successful reconciliation and the blocked human-activity case.
  • After updating the PR body with that evidence, obtain a fresh ClawSweeper review; if it does not start automatically, ask a maintainer to comment @clawsweeper re-review.

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (5 earlier review cycles)
  • reviewed 2026-08-24T18:13:36.413Z sha 22735db :: needs real behavior proof before merge. :: [P2] Include the close-proposal regression in the proof script
  • reviewed 2026-08-24T18:55:22.611Z sha 22735db :: needs real behavior proof before merge. :: [P2] Include the close-proposal guard in the proof command
  • reviewed 2026-08-25T02:10:38.378Z sha e899469 :: needs real behavior proof before merge. :: [P1] Preserve the post-snapshot human-activity guard
  • reviewed 2026-08-25T05:48:22.821Z sha a5d575f :: needs real behavior proof before merge. :: none
  • reviewed 2026-08-31T04:28:12.327Z sha cff385b :: needs real behavior proof before merge. :: none

@goutamadwant

Copy link
Copy Markdown
Contributor Author

Added the requested real-transport trace. It shows label reconciliation through the GitHub API and verifies that both same-second and later human activity prevent the automated action. The trace was captured against the current head without changing the implementation.

@goutamadwant
goutamadwant force-pushed the fix/exact-review-label-reconciliation branch from a5d575f to cff385b Compare August 31, 2026 04:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 automation 🚨 Merging this PR could break CI, automerge, proof capture, label sync, or automation. P1 Urgent regression or broken agent/channel workflow affecting real users now. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Exact re-review can publish a maintainer-ready verdict while PR status/rating labels remain stale

1 participant