Skip to content

feat(todos): wire caller-approved validation_command into todo completion - #3142

Open
NIU-123370 wants to merge 1 commit into
huangruiteng:mainfrom
NIU-123370:feat/todo-completion-validation
Open

feat(todos): wire caller-approved validation_command into todo completion#3142
NIU-123370 wants to merge 1 commit into
huangruiteng:mainfrom
NIU-123370:feat/todo-completion-validation

Conversation

@NIU-123370

Copy link
Copy Markdown

Implements #3082 along the shape @huangruiteng sketched: reuse the existing acceptance-loop validation handler (lifted into a shared module) rather than adding a new validator, and gate durable writeback on a typed receipt.

What this does

  • A todo may declare a caller-approved validation_command (+ optional validation_label) at loopx todo add time, stored on the todo metadata.
  • complete_goal_todo runs that command before the durable writeback commits. Pass → completion commits as usual; fail → returns ok=False with a typed validation receipt and leaves the state unchanged. Because complete_task keys quota spend off the completion ok marker, a blocked completion also blocks the spend for free.
  • Todos without a declared command keep the current fast path unchanged (no global forcing).

How it maps to the agreed shape

  • Handler reuse, not a new validator: the _run_caller_validation / _require_passed pair from capabilities/issue_fix/acceptance_loop.py is lifted verbatim into control_plane/runtime/validation_command.py — privacy invariant preserved (stdout/stderr/local_path_captured stay False; schema stays issue_fix_validation_command_v0). acceptance_loop.py now imports them from there (capability → control_plane; correct layering).
  • Validation runs outside the mutation lock: the declared command is pre-read and run before exclusive_file_lock(..., operation="todo_complete") is acquired, so a multi-second validation does not block concurrent todo operations on the same goal (the MUTATION lock deadline is 5s). Skipped on dry_run and on terminal replay.
  • Timeout under the outer budget: inner validation timeout is 20s, below the 30s outer CLI/MCP subprocess budget, so a timed-out command still yields a typed receipt.
  • No new settlement adapter — formal SettlementStep composition and the journaled replay-reuse fixture are deferred to land with fix(turn): fence remote execution and terminal writeback #3074, per "no new generic validator or framework is needed… a data field plus a caller for an existing step."

Two decisions I'd like your call on (happy to adjust)

  1. Command form: stored validation_command as a plain string (e.g. pytest -q tests/test_x.py), fed to the existing shlex-based handler. Your run-once precedent uses --validation-command-json (JSON argv). String was simpler for the metadata schema and matches the handler directly; switching to JSON argv is a small change if you prefer it.
  2. Receipt schema: reused issue_fix_validation_command_v0 (the handler's native schema). Easy to switch to loopx_turn_task_validation_v0 if you want a single shared id.

Tests

tests/control_plane/test_todo_completion_validation.py — 7 cases: positive (with an execution spy so it fails if the gate is silently skipped), negative (fail → blocked, state unchanged, no spend), no-command parity, timeout, terminal-replay (no re-run), missing executable, malformed command. The typed failure is also surfaced in render_todo_markdown (a Validation section with command_label / passed / status / exit_code / summary). Maintainability ratchet baseline for loopx/todos.py bumped 2142 → 2318 to register the intentional module growth.

Heads-up: 9 pre-existing failures on main (not from this PR)

While verifying I found 9 failing tests on upstream main that are unrelated to this change — a --required-read-id drift between production (agent_scoped_evidence_log.py / should_run_packet.py append --required-read-id <id> to the evidence-log required-read command) and the replan/portfolio test fixtures (which assert the command without that flag). I confirmed they fail identically on the base commit with my changes excluded. Happy to file a separate issue if useful.

Non-goals (deferred)

Refs #3082.

…tion

When a todo declares a caller-approved `validation_command` (set at
`todo add` time), `complete_goal_todo` now runs it independently and
requires a passing receipt before the durable writeback commits; on
failure it returns ok=False with a typed validation receipt and the
state is left unchanged (the MCP complete_task quota spend, which keys
off the ok marker, is blocked for free). Todos without a declared
command keep the current fast path unchanged.

Implements huangruiteng#3082 per the maintainer's shape: reuse the existing
acceptance_loop handler (lifted to a shared
control_plane.runtime.validation_command module) rather than adding a
new validator, and model the result as a typed receipt.

Design notes (for review):
- validation_command / validation_label are stored on the todo metadata
  as plain string fields (JSON-argv form is a documented follow-up).
- The validation subprocess runs BEFORE the exclusive mutation lock is
  acquired (pre-read of the declared command), so a slow command does
  not block concurrent todo operations on the same goal; it is skipped
  on dry_run and on terminal replay.
- Inner validation timeout is 20s, kept under the 30s outer CLI/MCP
  subprocess budget so a timed-out command still yields a typed receipt.
- No new settlement adapter: formal SettlementStep composition and the
  journaled replay-reuse fixture are deferred to land with huangruiteng#3074.
- Maintainability ratchet baseline for loopx/todos.py bumped 2142 -> 2318
  to register the intentional module growth from this feature.

Tests: positive (with execution spy), negative (fail -> blocked),
no-command parity, timeout, terminal-replay (no re-run), missing
executable, and malformed command.

Refs huangruiteng#3082.

@huangruiteng huangruiteng left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

详细中文评审

精确评审头: 3142@a56fd4512cde6e2f206e215fd8da8e83ba78d121

动机

本 PR 实现 #3082(P0):把“caller-approved validation_command”接入 todo completion——完成前运行调用方声明的验证命令,验证未通过则不提交写回、不产生完成状态、不消耗 quota。这是把“done 必须被验证”从口号变成机器门禁的核心改动。

改动思路

复用 issue_fix acceptance-loop 的验证执行器(提升到 control_plane/runtime/validation_command.py,行为逐字保留、wire schema issue_fix_validation_command_v0 不变),在 complete_goal_todo 的 mutation 锁之前预读并运行验证:声明了命令的 todo 走“验证门禁”,未声明的 todo 保持原快速路径。验证失败返回 ok=False + typed receipt + validation_blocked_completion=True,状态不变,spend 也随之被挡住。

具体改动(关键代码讲解)

  • run_caller_validation(validation_command.py)shlex.split(无 shell)、cwd=workspace、只记录 exit_code/passed,stdout/stderr/本地路径均不采集;超时抛 TimeoutExpired 由调用方转 typed failure receipt。隐私边界保持。
  • _read_declared_validation / _run_declared_completion_validation(todos.py):命令只在 todo add 时写入、无 update 路径(不可漂移);workspace 缺失/超时/命令无法启动/格式错误全部转 passed=False 的 typed receipt,而不是让异常破坏完成路径。
  • 门禁位置:验证在 exclusive_file_lock 之前运行(避免多秒 subprocess 占用 5s mutation 锁);complete_goal_todopassed is not True 时直接返回,未触碰状态文件。dry-run 与已完成的终端重放跳过验证。
  • 契约与展示contract.py 增加 validation_command/validation_label 元数据字段;markdown.py 对被拦截的完成渲染 ## Validation 段(仅公开安全字段)。
  • acceptance_loop 重构:本地 _run_caller_validation/_require_passed 删除并改为从 control_plane 导入,行为逐字一致(capability → control_plane 分层正确)。

对主干的风险

风险低且方向正确:默认快速路径不变(无全局强制);新字段可选;验证失败 fail closed(不提交、不 spend)。两点 P2(非阻塞):(1) todos.py 增至 2318 行(+178),建议后续把“读取→运行→门禁”编排收敛进 bounded 模块;(2) canary baseline 的 todos.py 行数已同步更新(2142→2318),与 diff 一致,无作弊。

我的整体评价

APPROVE。 scope fit 成立(complete_goal_todo 是真实生产调用点);typed receipt 与 spend 阻断闭环完整;测试覆盖正/负/超时/命令缺失/快速路径;隐私边界保持。这是对 #3082 的一次扎实落地。


English Verdict

APPROVE — exact head a56fd4512cde6e2f206e215fd8da8e83ba78d121.

Implements #3082: caller-approved validation_command runs before todo-completion writeback, gating state changes and quota spend on a typed receipt; the no-command fast path is unchanged. Validation executes outside the mutation lock, failure paths are typed and fail-closed, the issue_fix handler is reused verbatim with a shared wire schema, and the canary baseline matches the diff. Verified: 7 new focused tests, 51 todo regression tests, turn fake-host smoke, and py_compile all pass. P2 only: consider moving the orchestration out of the now-2318-line todos.py in a follow-up. No blockers.

huangruiteng added a commit that referenced this pull request Aug 12, 2026
Security hardening release: fixes GHSA-2225/GHSA-c42j/GHSA-hfmf/
GHSA-vx2m/GHSA-p7c9 via #3137-#3140, caller-approved completion
validation (#3142), and the durable-smoke review gate (#3134).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants