Skip to content

docs(mantis): clarify inline proof commands and activation - #140452

Open
brokemac79 wants to merge 1 commit into
mainfrom
openclaw/mantis-inline-proof-docs-alignment
Open

docs(mantis): clarify inline proof commands and activation#140452
brokemac79 wants to merge 1 commit into
mainfrom
openclaw/mantis-inline-proof-docs-alignment

Conversation

@brokemac79

@brokemac79 brokemac79 commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

What Problem This Solves

The Mantis page still calls command integration future work and lists automatic selection as an open question. That conflicts with the paired request-bound implementation in ClawSweeper #1425 and OpenClaw #138953.

Related: merged #138210. This follows @brokemac79's request and @obviyus's feedback to make useful proof available inside the original review.

Why This Change Was Made

Correct the operator-facing instructions without adding more changes to the producer PR. The producer already adds a detailed later section about isolation and observations; this separate, non-overlapping patch supplies command, activation and timing guidance.

User Impact

  • Distinguishes legacy manual Actions entrypoints from selective inline proof; keeps the warning that @clawsweeper mantis is not a typed dispatch command.
  • Shows the bare @clawsweeper proof override and either/both supported scenario selections, with automatic current-head resolution.
  • Explains the shared 20-minute ceiling and smaller remaining-review bound, rather than promising a fresh 20 minutes per check.
  • Makes coordinated producer/Worker deployment explicit, retains the existing Convex service without requiring its deployment, and does not claim the feature is already active.
  • Removes the obsolete automatic-selection open question. No automatic post-review recording is restored.

Evidence

Scope is one English Markdown page only. No runtime, tests, workflow, database/schema, navigation, generated/localized docs or CHANGELOG.md changes.

Documentation proof: compared live merged/current-main page blob 38b736cccb9cdb41d6cc79927888d87b802ef5ad with consumer d405f24703ad7a7fc1539d06304faefdd98ebe8d command parser/prompt/budget and producer 72ace9d87d5002fed4828b811ecb4f01ca6284b1, plus both live PR bodies. An independent combined-patch audit found no contradictory claims or overlapping edits with the producer's later detailed section.

Executed on Node 24.19.0 using the dependency-ready checkout's repository tooling against the exact edited page:

node scripts/check-docs-mdx.mjs <docs-worktree>/docs/concepts/mantis.md
  Docs MDX check passed (1 files).
pnpm exec oxfmt --check <docs-worktree>/docs/concepts/mantis.md
  All matched files use the correct format.
git diff --check
  passed in the docs worktree.

No new headings, internal links, config fences or navigation routes are introduced. Both external PR links were inspected live. English remains the source of truth; translations are generated in the separate docs publish repository.

Codex reviews are scoped-clean through P2, zero actionable findings: dirty review confidence 0.97; committed-branch review confidence 0.98 at 7f5e0d0b84e6ce5442b43b848acc67b0aff1351a, base 1fde67b08a4e8426474a46acd9ea968e300f8872. Commands: python .agents/skills/autoreview/scripts/autoreview --mode local --max-priority P2 --prompt <paired source context> before commit, then --mode branch --base 1fde67b08a4e8426474a46acd9ea968e300f8872 --max-priority P2 after commit. Full scanning and final source verification stayed enabled. Source/mirror trees and complete binary branch diff matched; no findings required edits. git merge-tree --write-tree <docs-head> 72ace9d87d5002fed4828b811ecb4f01ca6284b1 also passed, proving the docs changes combine without conflict with the producer head.

Risks and Rollout

Pinned consumer source verification

The first ClawSweeper review could not retrieve the companion repository's blobs. To make the cross-repository evidence inspectable here, the following excerpts were checked against the GitHub Contents API at exact consumer revision d405f24703ad7a7fc1539d06304faefdd98ebe8d; every returned blob SHA matched the local Git object, not a stale web-search result.

Command parser (blob 50a85f20dc72172815594384d4a2a4686250e01e):

export const PROOF_COMMAND_USAGE =
  "@clawsweeper proof [scenario-id[,scenario-id...]] [40-character-head-sha]";

Its supported list is ["web-ui-chat-proof", "telegram-bot-e2e-proof"]; selection === "auto" returns both as available options. Admission sets const scenarioId = match[1] ?? "auto" and const headSha = match[2] ?? input.currentHeadSha, rejecting a supplied SHA that differs from the current PR head. The review prompt says to select useful supported checks in auto mode; explicit mode says to attempt each requested supported check and explain unavailable/inconclusive selections. Both use observations in that review's final decision and explicitly forbid enqueueing another review.

Shared ceiling (blob 161fd310c29f66220c8d4c40078fde0855736df3):

export const REVIEW_PROOF_LIFETIME_MS = 20 * 60_000;

Original review bound (blob 733f4a2f899aa5754d2c293baa0a62b3cf157b13):

const remainingProofMs = Math.max(
  0,
  Math.floor(reviewDeadline - Date.now() - Math.min(90_000, options.timeoutMs / 10)),
);

The worker defines reviewDeadline = Date.now() + options.timeoutMs once and uses AbortSignal.timeout(remainingProofMs) for proof. Default review timeout (blob 84278c4602d9fdaabfca98a9b162f11337264aa0) is DEFAULT_REVIEW_CODEX_TIMEOUT_MS = 1_200_000. Thus immediate proof has at most 1_200_000 - 90_000 = 1_110_000 ms, or 18m30s, less elapsed analysis. These exact-source checks resolve the review's source-access question; they do not claim deployment.

This validates prose/source accuracy and MDX/formatting, not hosted activation, live Telegram or end-to-end latency. It must remain clear that the paired workflows/runtime need deployment. The existing producer PR retains the detailed execution boundaries; neither docs PR is a deployment or merge authorization.

Release-note context: clarify how selective inline proof is requested, bounded and activated.

@clawsweeper

clawsweeper Bot commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

@brokemac79

Copy link
Copy Markdown
Contributor Author

@clawsweeper review

@clawsweeper

clawsweeper Bot commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event exact_review_queue).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

@openclaw-barnacle openclaw-barnacle Bot added docs Improvements or additions to documentation size: XS maintainer Maintainer-authored PR labels Sep 6, 2026
@clawsweeper clawsweeper Bot added P3 Low-priority cleanup, docs, polish, ergonomics, or speculative work. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels Sep 6, 2026
@clawsweeper

clawsweeper Bot commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Codex review: blocked before merge. Reviewed September 6, 2026, 5:27 PM ET / 21:27 UTC.

ClawSweeper review

What this changes

Updates the Mantis guide with inline proof commands, deployment prerequisites, supported scenarios, and shared review time limits.

Merge readiness

Blocked before merge - 1 item remains

The clarification remains useful and is absent from current main. No actionable defect was found; the expanded source excerpts address the earlier review concern without claiming hosted activation.

Priority: P3
Reviewed head: 7f5e0d0b84e6ce5442b43b848acc67b0aff1351a

Review scores

Measure Result What it means
Overall readiness 🐚 platinum hermit (4/6) A focused documentation update with appropriate validation, clearer source evidence, and no actionable findings.
Proof confidence 🌊 off-meta tidepool Not applicable: Real behavior proof is not required because this PR only changes files under docs/.
Patch quality 🐚 platinum hermit (4/6) No actionable review findings were identified.

Verification

Check Result Evidence
Real behavior Not applicable Not applicable: Real behavior proof is not required because this PR only changes files under docs/.
Evidence reviewed 7 items Verified documentation-only introduction: The pinned merge-base-to-head diff changes only the Mantis page. The full page and introduced hunks were read; deployment remains conditional, and successful execution explicitly does not establish proof sufficiency or merge permission.
Current-main necessity: Fetched main still describes command integration as future work and directs readers to manual Actions dispatch. Its page blob is 38b736cccb9cdb41d6cc79927888d87b802ef5ad. The latest release page blob differs, but its contents could not be retrieved; no shipped-version claim is made.
Pinned consumer evidence and re-review continuity: The captured PR body supplies command-parser and timing excerpts at d405f24703ad7a7fc1539d06304faefdd98ebe8d. They support automatic head resolution, both named scenarios, a shared 20-minute ceiling, and the remaining-review reserve. Local Git tree entries independently match all four supplied blob IDs. Repository ownership was verified from its origin remote. Full parser and worker blobs remained unavailable, so this is excerpt-based verification, not a full consumer source audit.
Findings None None.
Security None None.

How this fits together

Mantis supplies behavioral observations for OpenClaw reviews. ClawSweeper selects and assesses those observations, while separate workflows run the checks and manage cleanup.

flowchart LR
  A[Maintainer proof request] --> B[ClawSweeper review]
  B --> C[Select supported checks]
  C --> D[Deployed proof workflows]
  D --> E[Bounded observations]
  E --> F[Review decision]
  D --> G[Producer cleanup]
Loading

Before merge

  • Resolve merge risk (P1) - Full pinned consumer parser and worker blobs could not be retrieved; their detailed behavior was checked against the newly supplied excerpts, with blob identities independently corroborated.
Agent review details

Security

None.

PR surface

Docs +30. Total +30 across 1 file.

View PR surface stats
Area Files Added Removed Net
Source 0 0 0 0
Tests 0 0 0 0
Docs 1 37 7 +30
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 1 37 7 +30

Review metrics

None.

Merge-risk options

Maintainer options:

  1. Decide the mitigation before merge
    Keep the command guidance conditional on coordinated deployment and preserve the distinction between observations, proof sufficiency, and merge readiness.
  2. Pause or close
    Do not merge this PR until maintainers decide whether the risk is worth taking.

Technical review

Best possible solution:

Keep the command guidance conditional on coordinated deployment and preserve the distinction between observations, proof sufficiency, and merge readiness.

Do we have a high-confidence way to reproduce the issue?

Not applicable: this PR changes documentation, and the outdated guidance is directly visible on fetched main.

Is this the best way to solve the issue?

Yes: updating the existing guide is a focused solution, and the conditional wording accurately separates the paired implementation from deployment.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against 270762e51d65.

Labels

Label justifications:

  • P3: This is a bounded operator-documentation clarification with no runtime or deployment changes.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🌊 off-meta tidepool and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: Real behavior proof is not required because this PR only changes files under docs/.

Evidence

What I checked:

  • Verified documentation-only introduction: The pinned merge-base-to-head diff changes only the Mantis page. The full page and introduced hunks were read; deployment remains conditional, and successful execution explicitly does not establish proof sufficiency or merge permission. Public docs: docs/concepts/mantis.md. (docs/concepts/mantis.md:321, 7f5e0d0b84e6)
  • Current-main necessity: Fetched main still describes command integration as future work and directs readers to manual Actions dispatch. Its page blob is 38b736cccb9cdb41d6cc79927888d87b802ef5ad. The latest release page blob differs, but its contents could not be retrieved; no shipped-version claim is made. Public docs: docs/concepts/mantis.md. (docs/concepts/mantis.md:315, 270762e51d65)
  • Pinned consumer evidence and re-review continuity: The captured PR body supplies command-parser and timing excerpts at d405f24703ad7a7fc1539d06304faefdd98ebe8d. They support automatic head resolution, both named scenarios, a shared 20-minute ceiling, and the remaining-review reserve. Local Git tree entries independently match all four supplied blob IDs. Repository ownership was verified from its origin remote. Full parser and worker blobs remained unavailable, so this is excerpt-based verification, not a full consumer source audit. (src/repair/proof-command.ts, d405f24703ad)
  • Review timeout corroboration: The pinned policy source returned DEFAULT_REVIEW_CODEX_TIMEOUT_MS = 1_200_000 at line 84. Combined with the supplied 90-second reserve expression, the documented immediate upper bound is correctly 18m30s. (src/clawsweeper-policy.ts:84, d405f24703ad)
  • Related work remains complementary: The supplied GitHub context identifies docs(mantis): correct proof publishing and QA dispatch guidance #138210 as merged and feat(qa): run isolated behavioral proof for inline reviews #138953 as an open producer implementation explicitly paired with this documentation follow-up. Neither supersedes its command and timing clarification. The unrelated OpenClaw pull request numbered 1425 is not the linked ClawSweeper consumer.
  • Feature-history routing: Current-main page history includes repeated Mantis work by Ayaan Zaidi, including Telegram proof leasing and serialized proofs, followed by the merged publishing/dispatch documentation correction. This supports area routing without asserting causal introduction. Public docs: docs/concepts/mantis.md. (docs/concepts/mantis.md, cff996e41305)

Likely related people:

  • Ayaan Zaidi: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)
  • brokemac79: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (1 earlier review cycle)
  • reviewed 2026-09-06T21:19:07.977Z sha 7f5e0d0 :: blocked before merge. :: none

@brokemac79

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event exact_review_queue).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Improvements or additions to documentation maintainer Maintainer-authored PR P3 Low-priority cleanup, docs, polish, ergonomics, or speculative work. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. size: XS status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant