Skip to content

fix(live-proof): only publish recordings that demonstrate a change - #1199

Merged
steipete merged 1 commit into
mainfrom
steipete/live-proof-show-something
Aug 18, 2026
Merged

fix(live-proof): only publish recordings that demonstrate a change#1199
steipete merged 1 commit into
mainfrom
steipete/live-proof-show-something

Conversation

@steipete

Copy link
Copy Markdown
Contributor

Summary

Every live proof published so far was useless, and I only caught it by watching the videos rather than checking that the URLs resolved.

  • Terminal clips were 0.9s: an empty prompt, the typed command, end — before any output rendered. expect_output captured the pane exactly once with no polling immediately after Enter, so it threw instantly; it also grepped the whole pane including the echoed command, so other plans could match the command the model had just typed.
  • Browser clips ran 16s with every frame identical, because plans asserted static text already rendered at load (feat(bay): focus the crustacean finder with the / shortcut #1182 asserted "press /", the finder's own hint) and nothing scrolled the changed region into frame.
  • Nothing gated attachment: drive_status was recorded but unused, so failed drives were uploaded and posted to public review comments.

Fixes: terminal run waits for real output and expect_output polls with a timeout against post-command content only; both drivers settle between steps, hold the end state, and honour a minimum length; browser steps scroll their target into view.

The substance gate is semantic, not pixel-based. Attachment now requires an expectation that was absent at plan start and satisfied afterwards — a demonstration that verifies nothing publishes nothing. Pixel-difference gating was implemented, measured against real artifacts, and rejected: the junk frozen-page recording scores SSIM 0.914 between first and last frame while a genuine blade-opening demo scores 0.980, because ambient page animation swamps any threshold. The review prompt now requires a state-changing action plus an assertion on content that is absent beforehand, and real selectors rather than prose.

Validation

Run locally against the two real plans:

  • the organic feat(bay): focus the crustacean finder with the / shortcut #1182 plan that produced the frozen video now logs skip: plan verified nothing that changed and writes no manifest — nothing would be attached;
  • a plan asserting "Found openclaw/clawsweeper#1178" (absent at load) attaches, and its extracted frames show the input filling, the status flipping, and the toast appearing.

pnpm build clean; live-proof suite 33/33; full unit suite 2,326 passed / 0 failed (parallel with review, exit 0). Autoreview (Codex, gpt-5.6-sol, high): clean, "patch is correct (0.98)".

Follow-up

The five PRs that received junk videos need cleanup once this lands: re-dispatching the same heads overwrites the head-scoped R2 objects and record block, and any plan that legitimately has nothing to show needs its stale Live Proof block stripped from the record.

Every published proof was useless: terminal clips ended before the
command produced output (expect_output captured the pane once, with no
polling, and matched the echoed command), and browser clips showed a page
that never changed because plans asserted static text already rendered at
load. Nothing gated attachment, so failed drives shipped anyway.

Terminal steps now wait for real output and match only post-command
content; both drivers settle between steps, hold the end state, and honour
a minimum length. Attachment requires an expectation that was absent at
plan start and satisfied afterwards, so a demonstration that proves
nothing publishes nothing. Pixel-difference gating was evaluated and
rejected: ambient page animation scores higher (SSIM 0.914) than a real
blade-opening demo (0.980), so no threshold separates them. The review
prompt now requires a state-changing action plus an assertion on content
absent beforehand, with real selectors.
@clawsweeper

clawsweeper Bot commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

@clawsweeper

clawsweeper Bot commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

ClawSweeper status: review started.

I am starting a fresh review of this pull request: fix(live-proof): only publish recordings that demonstrate a change This is item 1/1 in the current shard. Shard 0/1.

This placeholder means the worker is alive and reading the current context. I will edit this same comment with the actual review when the claws are done clicking.

Crustacean status: shell secured, claws on keyboard, evidence pebbles being sorted.

@steipete
steipete merged commit 03f8a74 into main Aug 18, 2026
8 checks passed
@steipete
steipete deleted the steipete/live-proof-show-something branch August 18, 2026 00:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant