feat(proof): collect selective evidence within the original review

[roboclaw-bot]

What Problem This Solves

Let ClawSweeper request relevant behavioral checks during its original review and use their observations before making its decision. This does not restore the unconditional post-review step retired by ClawSweeper #1280.

Requested by @brokemac79 following @obviyus's feedback. Paired producer: OpenClaw #138953. Original documentation: merged OpenClaw #138210.

Consumer candidate: d405f24703ad7a7fc1539d06304faefdd98ebe8d; base: bb8bd3eb709b093a10ff1336227e89986dbcbb0c; tree: 30141944285f2b5dbfaa9032a6406266aee4ac4c. Exact-head local proof and review are complete; they do not establish deployed availability or current GitHub readiness.

Why This Change Was Made

Eligible OpenClaw PR reviews gain two optional tools:

• Telegram: a bounded, data-only PR-specific plan, with trusted producer execution and bounded observations for the reviewer to judge.
• Web UI: the existing fixed browser chat/send/reply recipe against a mocked Gateway, not arbitrary browser automation or provider/authentication proof.

The reviewer invokes checks when relevant. Maintainers can also comment @clawsweeper proof, or restrict the review to web-ui-chat-proof, telegram-bot-e2e-proof, or both as a comma-separated list. The router captures the current head automatically; an optional SHA must match it. Commands queue the normal proof-focused review, not the old secondary batch consumer.

The queue enforces the exact live owner, candidate and scenario allowlist independently of model instructions. Up to three distinct plans share one proof lifetime. Prepared dispatch and completed evidence require matching durable acknowledgements; expired or terminal updates cannot masquerade as accepted transitions. Cached evidence also requires a final read-only ownership confirmation after awaited GitHub checks, without rewriting the cache. The trusted Worker binds producer identity before dispatch, validates Actions OIDC and receipt/artifact provenance, and returns observations only to the owning review. Unknown dispatch outcomes are not blindly retried.

No second review is enqueued by a proof tool. Normal publication, proof assessment and readiness owners retain their authority. Legacy consumer/recipe code remains compatibility and reconciliation tooling; the legacy Markdown recipe is not a third inline tool.

User Impact

Useful behavioral evidence can inform the original review. Reviews that do not need checks skip them. Explicit commands need no manually typed head SHA. A completed process, artifact or video alone does not clear unrelated review, security or CI blockers.

Real Behavior Proof

Claim: evidence reaches the same original app-server turn; ownership, scenario scope, deadline and artifact boundaries fail closed.

Environment: Docker-backed Crabbox local-container, Docker 29.5.2, image node:24-bookworm, observed image ID sha256:be23f54a88d34e8824c741b19b91064094f92c1c97b194144bfc8b50d67258e2, Node 24.20.0, pnpm 11.10.0. The disposable container verified the exact head/tree above before testing. Lease cbx_e58f16958013, run run_654e2a4f0901.

Executed commands:

pnpm run build:all
node --experimental-strip-types --test test/dashboard-review-proof-execution.test.ts test/review-proof-client.test.ts test/review-proof-app-server-boundary.test.ts test/repair/proof-command.test.ts test/repair/direct-re-review-admission.test.ts test/dashboard-review-proof-requests.test.ts test/dashboard-review-proof-producer-auth.test.ts
node --experimental-strip-types --test --test-name-pattern='inline proof returns real HTTP' test/codex-process.test.ts
CI=1 NODE_OPTIONS=--max-old-space-size=6144 pnpm run check

Observed: 63 focused tests and the real-HTTP original-turn test passed. The compiled router/HTTP intake fixture exercised five selections: automatic, Web UI only, Telegram only, both, and unsupported-only. Each preserved the current SHA and corresponding backend scope and used dispatch_clawsweeper, without a legacy proof dispatch.

Current review finding fixed: cached delivery now has the same final owner fence as fresh delivery. The controlled proof pauses the run-metadata HTTP response after initial queue admission, changes the durable owner/scope/deadline, then resumes verification. Each rejected case asserts inconclusive, no returned observations, and unchanged cache; the active owner receives the expected observations. Before repair the four changed-owner cases returned completed observations. Executed trace:

actual_source_tree=30141944285f2b5dbfaa9032a6406266aee4ac4c
PASS cached proof HTTP delivery: active
PASS cached proof HTTP delivery: revoked
PASS cached proof HTTP delivery: reassigned
PASS cached proof HTTP delivery: scope_revoked
PASS cached proof HTTP delivery: expired
PASS cached proof HTTP delivery: max_result
PASS proof HTTP response beyond bounded envelope headroom is rejected
focused: 63 passed, 0 failed
original-turn real HTTP: 1 passed, 0 failed

The maximum-size case stores exactly 256 KiB through the actual durable queue and delivers it through the executor and real HTTP to the compiled client. This failed before the envelope correction. The 192 KiB observation and 256 KiB result limits remain unchanged; only 1 KiB transport headroom is reserved for the completion wrapper, including its authoritative expiry timestamp. A valid JSON response one byte beyond the wire cap is rejected.

Producer authentication ordering: the actual HTTP Worker/queue fixture proves missing, malformed, forged or wrong-scope OIDC tokens cause zero durable-object calls. This failed before repair because unauthenticated requests reached the queue lookup. Valid signed claims must then match the durable repository/workflow/run pins; a final redemption still fences an owner revoked during verification. No queue index, schema or cache was added.

missing / malformed / forged / wrong repository / wrong run / unsupported workflow => 403, DO calls=[]
valid signed identity + matching durable pins => 200, producer-record then redeem
valid signed identity + wrong durable workflow SHA => 403, producer-record only
owner revoked after producer-record => 409 at final redeem

Authorized 20-minute ceiling: controlled clocks exercise the actual queue/client boundaries without waiting 20 real minutes. Producer-record/redemption succeeds at minute 16 and rejects at minute 20; a later plan retains the first deadline. Pending proof continues after minute 15 and returns immediately on completion. A second call beginning at minute 16 cannot accept a response arriving at minute 20. Missing/invalid expiry fails closed; an earlier queue expiry cannot be extended by a later call. The existing stuck-HTTP app-server proof still returns inconclusive with time left for the original final decision. The shared-clock regression failed before repair; it and all new deadline checks pass in the container.

The expiry regressions failed before repair for the intended reason: 10 failed because rejected/expired acknowledgements permitted dispatch or evidence delivery. After repair, 27 owner tests passed, including durable expiry persistence across queue recreation.

Full check: exact candidate pnpm run check passed: 5,292 tests, 5,274 passed, zero failed, 18 skipped. Coverage: 86.50% lines, 76.78% branches, 89.95% functions. All static/build/lint gates and 13 changed-coverage tests passed. The full-check phase took 5m16.620s; wrapper and command exited zero. Crabbox reported leaseStopped=true, and Docker absence was independently verified.

Artifacts: retained local reproduction script csw152-consumer-final-preauth-check.sh, exact source bundle csw152-consumer-final-preauth.bundle, stdout/stderr captures csw152-consumer-final-preauth.stdout.log and csw152-consumer-final-preauth.stderr.log, and validation record csw152-consumer-twenty-minute-proof.md. Inspectable excerpts are included above rather than relying on access to local filenames.

Limits: controlled GitHub/model fixtures, actual HTTP/Worker/queue and archive verification prove the consumer contract—not live Telegram, deployed producer availability, semantic model-selection quality, or complete hosted latency. An earlier full run had one unchanged legacy tmux assertion failure; its same-container targeted rerun and this final complete run passed.

Codex Review

Mandatory dirty reviews were scoped-clean before both the cached-owner and envelope commits. Accepted findings were fixed and focused proof rerun. The final complete committed branch review uses:

python <autoreview-skill>/scripts/autoreview --mode branch --base bb8bd3eb709b093a10ff1336227e89986dbcbb0c --max-priority P2 --prompt <approved inline-review requirements> --output <outside-repository.md> --json-output <outside-repository.json> --stream-engine-output

Result: full committed branch scoped-clean at P2, both review partitions with zero accepted/actionable findings, exit zero. Dirty review first found the per-call clock reset; it was fixed and repeated clean before commit. Branch review then found producer authentication after queue lookup; that was fixed, regression-proved and dirty-reviewed clean before the final commit. A later report's outdated return_run_details recommendation was rejected against the pinned 2026-03-10 official GitHub API contract; the repeated full review independently confirmed that contract and finished clean. Earlier reports remain preserved, not relabeled clean. Final report: csw152-consumer-final-preauth-branch-review-r2.md / .json.

A recommendation to restore dispatch_proof was rejected because it would restore the superseded secondary-review design. Its underlying stale documentation was corrected, and the actual intended command route was proved above.

Risks and Rollout

• Martin authorized a 20-minute shared proof ceiling, not 20 minutes per check or a mandatory wait. Effective proof time is the lesser of that shared ceiling and the remaining original review budget minus its final-decision reserve. The unchanged default 20-minute review permits at most 18m30s if proof starts immediately, less after analysis. Completion returns immediately; missing, late or failed evidence stays inconclusive.
• Measured cold trusted-plus-candidate preparation plus the successful Gateway proof took 13m17.758s in the separate producer test, excluding hosted queue/checkout/upload/finalizer. Complete hosted timing remains unproved; both checks are not guaranteed to finish. Only the shared proof ceiling changed: enclosing review timeout, producer job limit, credential lifetime and Crabbox limits are unchanged.
• Client timeout/cancellation aborts its HTTP wait and returns inconclusive; it does not itself cancel a dispatched workflow or physically delete its box. Existing producer authority renewal (10 seconds, maximum 30-second grant), expiry/owner checks and cleanup govern that work. The new ceiling tests prove client return and authority fencing, not new physical cleanup behavior.
• Protected main can advance between producer pinning and dispatch. A mismatch intentionally becomes inconclusive; pins are not loosened and dispatch is not blindly retried.
• Deploy the paired reviewed producer workflows and consumer Worker/runtime, then verify exact-head CI, fresh review and hosted behavior. Workflow/OAuth scope remains a rollout prerequisite.
• The inline path reuses existing queue storage and existing Convex lease APIs: no Convex schema/API deployment or new broker credential is required. Telegram still needs the existing credential service and disposable sandbox prerequisites.
• OpenClaw Bay remains observer-only. Private proof plans, observations and lease capabilities are excluded from its/public queue projections; no Bay action or schema change is needed.
• No production gates or merge authority were changed by this local work. Old remote labels are not evidence for this rewritten head.

Release-note context: selective Telegram and fixed Web UI checks inform the original review, with a maintainer command retained as an override.

[clawsweeper]

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

[clawsweeper]

Codex review: blocked before merge. Reviewed September 6, 2026, 4:19 PM ET / 20:19 UTC.

ClawSweeper review

What this changes

Adds optional, review-selected Telegram and fixed Web UI checks that return verified observations to the original review, plus maintainer proof commands and compatibility reconciliation.

Merge readiness

⛔ Blocked before merge - 3 items remain

This remains distinct, sponsored work absent from current main. The cached-owner finding is repaired, and the documented 20-minute ceiling addresses the previous timing disposition; no new blocking correctness defect was established.

Priority: P2
Reviewed head: d405f24703ad7a7fc1539d06304faefdd98ebe8d

Review scores

Measure	Result	What it means
Overall readiness	🐚 platinum hermit (4/6)	Useful, bounded implementation with concrete consumer-boundary proof and no established blocking finding; hosted rollout remains separately qualified.
Proof confidence	🐚 platinum hermit (4/6)	Sufficient (terminal): The captured current-head Crabbox local-container traces exercise the production client, Worker, executor and durable queue through real HTTP, including same-turn delivery, forbidden producer identities, revoked/reassigned owners and expiry rejection. They sufficiently establish the consumer contract; hosted producer availability and latency remain explicitly outside that proof.
Patch quality	🐚 platinum hermit (4/6)	No actionable review findings were identified.

Verification

Check	Result	Evidence
Real behavior	Verified	Sufficient (terminal): The captured current-head Crabbox local-container traces exercise the production client, Worker, executor and durable queue through real HTTP, including same-turn delivery, forbidden producer identities, revoked/reassigned owners and expiry rejection. They sufficiently establish the consumer contract; hosted producer availability and latency remain explicitly outside that proof.
Evidence reviewed	10 items	Applicable repository policy: Read the complete root AGENTS.md. No nested AGENTS.md or maintainer-notes directory was found. Applied the proof, review-continuity, observer-only Bay, and read-only review boundaries. Still necessary on current main: The pinned main documentation describes automatic proof as retired and offers no replacement execution lane. The new inline client and executor are absent from the supplied latest-release tree. The merged retirement PR, #1280, addresses the older post-review design rather than implementing this request. Prior ownership finding resolved: Cached and fresh delivery require a matching durable acknowledgement, result, request, plan and unexpired deadline. The queue revalidates the live owner and scope for confirm_completed without rewriting cached evidence.
Findings	None	None.
Security	None	None.

How this fits together

ClawSweeper reviews GitHub changes through a leased review queue. The new proof bridge sends bounded requests to trusted OpenClaw workflows and returns authenticated observations to the same reviewer before publication.

flowchart TD
  A[GitHub change or maintainer command] --> B[Leased review]
  B --> C[Select relevant check]
  C --> D[Validate owner and scope]
  D --> E[Trusted proof workflow]
  E --> F[Verify receipt and observations]
  F --> B
  B --> G[Normal review publication]

Loading

Before merge

• Resolve merge risk (P1) - Hosted queue, checkout, upload and finalization overhead remain unmeasured; optional checks can consume substantial review time and still return inconclusive.
• Resolve merge risk (P1) - Useful hosted execution depends on coordinated deployment of the paired producer and its existing credential and sandbox services; consumer proof does not establish that deployment.
• Resolve review confidence - ClawSweeper must reach high confidence before merge readiness is known.

Agent review details

Security

None.

Review metrics

Metric	Value	Why it matters
Production and validation growth	Production +5,041 net lines; tests and runtime harnesses +6,762 net lines	The stated scope includes the inline bridge, authority checks and retained compatibility tooling, making this a substantial automation addition.
Shared proof budget	3 distinct plans; 20-minute shared ceiling; at most 18m30s under the default review budget	Checks compete with analysis time and must leave time for the original final decision.

Merge-risk options

Maintainer options:

1. Retain bounded rollout expectations (recommended)
   Accept inconclusive outcomes during coordinated deployment and assess hosted completion latency without weakening ownership pins or extending each check's budget.

Technical review

Best possible solution:

Keep proof optional and bounded within the original review, preserve inconclusive fallback, and coordinate consumer deployment with the reviewed producer contract.

Do we have a high-confidence way to reproduce the issue?

Not applicable as a feature request; the supplied controlled HTTP traces demonstrate consumer behavior and repaired ownership/deadline cases, but were not rerun during this review.

Is this the best way to solve the issue?

Yes for the stated consumer boundary: selective same-turn observations address the retired design's inability to inform the original decision while preserving ordinary publication authority.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against bb8bd3eb709b.

Labels

Label justifications:

• P2: This is a bounded improvement to review evidence collection without an established urgent user-facing regression.
• merge-risk: 🚨 automation: The change adds optional workflow dispatch and waiting inside active reviews, with hosted completion latency and coordinated deployment still operational concerns.
• rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
• status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The captured current-head Crabbox local-container traces exercise the production client, Worker, executor and durable queue through real HTTP, including same-turn delivery, forbidden producer identities, revoked/reassigned owners and expiry rejection. They sufficiently establish the consumer contract; hosted producer availability and latency remain explicitly outside that proof.
• proof: sufficient: Contributor real behavior proof is sufficient. The captured current-head Crabbox local-container traces exercise the production client, Worker, executor and durable queue through real HTTP, including same-turn delivery, forbidden producer identities, revoked/reassigned owners and expiry rejection. They sufficiently establish the consumer contract; hosted producer availability and latency remain explicitly outside that proof.

Evidence

What I checked:

• Applicable repository policy: Read the complete root AGENTS.md. No nested AGENTS.md or maintainer-notes directory was found. Applied the proof, review-continuity, observer-only Bay, and read-only review boundaries. (AGENTS.md:1, d405f24703ad)
• Still necessary on current main: The pinned main documentation describes automatic proof as retired and offers no replacement execution lane. The new inline client and executor are absent from the supplied latest-release tree. The merged retirement PR, fix: retire automatic live proof #1280, addresses the older post-review design rather than implementing this request. (docs/live-proof.md:12, bb8bd3eb709b)
• Prior ownership finding resolved: Cached and fresh delivery require a matching durable acknowledgement, result, request, plan and unexpired deadline. The queue revalidates the live owner and scope for confirm_completed without rewriting cached evidence. (dashboard/review-proof-execution.ts:68, d405f24703ad)
• Producer authorization boundary: The Worker authenticates the signed producer identity before queue lookup, matches durable workflow/run pins, and performs a final redemption check. The inspected HTTP fixture covers forged and wrong-scope identities, mismatched workflow pins, and revocation between lookup and redemption. (dashboard/worker.ts:5367, d405f24703ad)
• Current-head controlled behavior proof: The captured PR body identifies head d405f24 and tree 30141944285f2b5dbfaa9032a6406266aee4ac4c. Docker-backed Crabbox local-container lease cbx_e58f16958013, run run_654e2a4f0901, records real HTTP delivery through the production client/executor/queue, rejected revoked/reassigned/scope-revoked/expired owners, unchanged caches, bounded maximum-size delivery, and same-turn observations. The source fixture supports these scenarios; tests were not executed during this read-only review. (test/dashboard-review-proof-requests.test.ts:277, d405f24703ad)
• Timing disposition and retained limits: The captured body records Martin's authorization of a shared 20-minute ceiling. Source retains the earliest deadline across calls and reserves final-decision time; the default review budget permits at most 18m30s of proof. Reported cold producer preparation and execution consumed 13m17.758s before hosted overhead. (src/review-proof-limits.ts:2, d405f24703ad)

Likely related people:

• Vincent Koc: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)
• Peter Steinberger: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)

Rating scale

Score	Internal tier	Crab rank	Meaning
6/6	S	🦀 challenger crab	Exceptional readiness
5/6	A	🦞 diamond lobster	Very strong readiness
4/6	B	🐚 platinum hermit	Good normal PR; ordinary maintainer review
3/6	C	🦐 gold shrimp	Useful, but confidence is limited
2/6	D	🦪 silver shellfish	Proof or implementation needs work
1/6	F	🧂 unranked krab	Not merge-ready
N/A	NA	🌊 off-meta tidepool	Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

• ClawSweeper keeps one durable marker-backed review comment per issue or PR.
• Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
• A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
• PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
• Maintainers can also comment @clawsweeper review to request a fresh review only.
• Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
• Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
• Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (16 earlier review cycles; latest 8 shown)

• reviewed 2026-09-05T17:53:31.535Z sha 6117648 :: needs real behavior proof before merge. :: [P2] Generate failure evidence in the paired QA failure case
• reviewed 2026-09-05T18:43:56.020Z sha b319315 :: blocked before merge. :: none
• reviewed 2026-09-05T18:56:23.209Z sha b319315 :: blocked before merge. :: none
• reviewed 2026-09-05T22:31:34.308Z sha 0234803 :: needs real behavior proof before merge. :: none
• reviewed 2026-09-05T22:49:19.254Z sha 0234803 :: blocked before merge. :: none
• reviewed 2026-09-05T23:05:42.977Z sha 0234803 :: blocked before merge. :: none
• reviewed 2026-09-06T18:50:21.701Z sha d5aa763 :: needs real behavior proof before merge. :: [P2] Recheck live ownership before returning cached proof
• reviewed 2026-09-06T19:25:19.301Z sha 9407f9c :: blocked before merge. :: none

[roboclaw-bot]

@clawsweeper re-review

[clawsweeper]

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event exact_review_queue).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

[brokemac79]

@clawsweeper re-review

[clawsweeper]

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event exact_review_queue).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

• State: Waiting
• Detail: A newer exact-review queue owner superseded this run. This run completed as a no-op.
• Run: https://github.com/openclaw/clawsweeper/actions/runs/34052659619
• Updated: 2026-09-06T18:46:01.976Z

[brokemac79]

@clawsweeper re-review

[clawsweeper]

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event exact_review_queue).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

[brokemac79]

@clawsweeper re-review

[clawsweeper]

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event exact_review_queue).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

• State: Complete
• Detail: The durable review result and its route handoff completed.
• Run: https://github.com/openclaw/clawsweeper/actions/runs/34057680898
• Updated: 2026-09-06T20:20:42.409Z