Add robustness loops: quality validation, consensus, and critic-judge refinement

[BillJr99]

Summary

This PR introduces a comprehensive robustness framework that wraps agent dispatches with three layers of quality assurance and adaptive retry logic:

1. Response Quality Validation — scores responses for emptiness, malformed blocks, missing outputs, and low confidence
2. Auto-Consensus Fan-Out — stages marked careful: true (or all stages when configured) fan into N parallel samples with chief coalescing
3. Critic-Judge Refinement — draft → critic → revise inner loop that iterates until the critic approves or max rounds reached

Additionally, the Ralph and autoresearch loops now detect plateaus and regressions, escalating to consensus and reframing rather than burning the full iteration budget.

Key Changes

Core Robustness Layers

• response_quality.py (new) — Validator module that detects:

  • Empty or sub-threshold responses
  • Malformed ACTION/POST blocks
  • Missing declared outputs
  • Low confidence or needs-review flags
  • Refusal patterns
  • Returns ResponseQuality verdict with recoverable/non-recoverable distinction and repair hints

• agent.py — Refactored dispatch entry point:

  • run() now wraps _dispatch_once() with quality and consensus layers
  • _dispatch_with_quality_loop() — retries failed responses with repair preambles, escalates to consensus on final retry
  • _dispatch_auto_consensus() — fans out to N samples + chief coalescing
  • _should_auto_consensus() — proactive fan-out decision logic
  • _META_PHASES frozenset prevents recursion (consensus, checkpoint, recovery, critic, etc. never re-trigger loops)

• workflow.py — Critic-judge refinement:

  • WorkflowStage.refine field for explicit critic configuration
  • _refine_enabled() — decides when to run critic based on stage config or robustness.auto_refine policy
  • _refine_loop() — dispatches critic, scores response, re-dispatches worker with feedback up to max_rounds
  • Integrated into _run_stage() after initial dispatch

Adaptive Loop Control

• ralph_loop.py — Plateau and regression detection:

  • _progress_verdict() — classifies iterations as improving/plateau/regressing
  • _handle_plateau() — escalates to consensus fan-out, then reframes workflow
  • _should_terminate_for_plateau() — graceful termination when stuck
  • _adaptive_extra() — injects careful=true into next iteration's dispatches when escalating

• autoresearch_loop.py — Similar plateau detection for experiment loops

Blackboard & Q&A Protocol

• blackboard.py — Extended Post class:
  • target_agent and urgency fields for directed questions
  • find_unanswered_questions() filters by target agent
  • Questions with urgency: blocking answered inline; async surfaced to chief

Configuration & Documentation

• config.py — New robustness block in DEFAULT_CLK_CONFIG:

  • auto_consensus (off | on_careful | always)
  • auto_refine (off | careful_only | all)
  • max_quality_retries, min_response_chars
  • refine_max_rounds, refine_accept_threshold
  • max_qa_depth, plateau_window, plateau_action

• .env.example — Documented all CLK_ROBUSTNESS_* env vars with cost multipliers

• kickoff.sh — Added env-var override block that maps CLK_ROBUSTNESS_* into clk.config.json

• README.md — Comprehensive sections:

  • "Robustness loops" explaining all four layers
  • "Robustness-loop multipliers"

https://claude.ai/code/session_01XYLA51b49wz4S2THibSjaF