Skip to content

Add tc eval review CLI wrapper (CR-RH-003)#87

Merged
coreytshaffer merged 1 commit into
mainfrom
cr-rh-003-eval-review-cli
Jul 9, 2026
Merged

Add tc eval review CLI wrapper (CR-RH-003)#87
coreytshaffer merged 1 commit into
mainfrom
cr-rh-003-eval-review-cli

Conversation

@coreytshaffer

Copy link
Copy Markdown
Owner

Summary

Adds CR-RH-003 for the evidence-bound review harness: a thin, read-only tc eval review CLI wrapper around the existing review_submission_v0 validator (CR-RH-001) and review_result_v0 deterministic checker (CR-RH-002).

The command validates a pre-tagged submission, runs the checker against an explicitly supplied context packet, and renders or writes the result. It adds no checking logic of its own.

Scope

  • New tc eval review subcommand (handler + subparser + dispatch)
  • New pure write_review_result(...) writer in review_result.py
  • New CLI doc under docs/operations/
  • New CLI unit tests
  • Submission-schema and result-contract doc updates (CLI now delivered)

Output convention

  • No --output: render the review result to stdout.
  • --output: write the JSON result to that path and print a short Success: line.
  • --print-json: print the JSON result to stdout in addition to normal behavior.
  • --fail-on-gate: exit 3 only after successful validation/checking when grounding_gate == fail.
  • Input/validation errors: exit 1.

Non-goals

  • Thin CLI wrapper only; no new checker logic
  • No model calls
  • No command execution
  • No raw-prose classification
  • No action approval
  • No citation-quality judgment beyond section-scoped anchor resolution
  • No correctness, safety, certification, or production-readiness verdict
  • No ledger, identity, generated artifact, scratch, backlog, changelog, or root README changes
  • Only a user-specified --output path may be written

Validation

  • python -m pytest tests/test_eval_review_cli.py tests/test_review_result.py tests/test_review_submission.py -q -> 61 passed
  • Real read-only smoke: python -m triage_core.tc_cli eval review --submission tests/fixtures/evals/model_review/review_submission_v0.example.json --context-packet tests/fixtures/evals/model_review/review_context_packet.example.md -> intentional grounding-gate FAIL, exit 0
  • git diff --cached --check -> clean before commit
  • staged file list matched exactly the six approved files

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@coreytshaffer
coreytshaffer merged commit 72c070f into main Jul 9, 2026
3 checks passed
@coreytshaffer
coreytshaffer deleted the cr-rh-003-eval-review-cli branch July 9, 2026 01:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant