feat(context): add CPG-08 evaluation harness and rollout gates

[Wibias]

Summary

• Stack CPG-08 on CPG-07.
• Add a reproducible deterministic preflight harness for projection off, duplicate-only, bounded recovery, and a lab simple-control arm.
• Record synthetic performance/recoverability evidence in CI.
• Keep task success, provider cache billing, latency and cost-per-success claims explicitly unavailable when no live provider credentials/evidence exist.
• Encode the predeclared rollout gates so byte savings or an aggregate attempt count can never turn recovery default-on.
• Add docs/context-projection-live-evaluation.md with the exact live trial protocol and per-attempt evidence fields.

Scope decision after deep review

This PR is the evaluation framework + synthetic preflight, not the completed live rollout evaluation from the original roadmap. The checked-in evidence currently has zero live provider attempts, so it cannot justify default-on recovery or provider-stateful expansion.

Before the live gate can pass, the evaluation matrix must include provider-native continuation controls rather than comparing only context-reduction strategies. Required live controls are:

• stateless full replay with CPG off;
• stateless duplicate-only CPG;
• stateless bounded-recovery CPG;
• OpenAI Responses HTTP continuation with previous_response_id when the cell supports it;
• OpenAI Responses WebSocket continuation with previous_response_id when the cell supports it;
• provider-native compaction where the evaluated provider supports it;
• the existing simple head/tail control remains lab-only and can never become a production candidate.

The live workload must also include adversarial evidence placement and long agent trajectories: required evidence in the middle of a large result, root cause far from an error-log tail, near-identical outputs with one critical differing line, omitted edit anchors, recovery followed by a real mutating action, evidence needed many turns later, provider reconnect/state loss, tool-catalog changes, and 20-50 tool-call coding tasks.

Per-cell evidence gate

liveProviderAttempts and caller-supplied controlCoverageComplete are retained only for backward readability of old synthetic reports. They cannot approve rollout.

The gate now derives evidence from declared provider/workload/cache cells. Every required arm in every cell must have at least 30 completed attempts. OpenAI Responses cells can require both native HTTP and WebSocket continuation controls; provider cells can independently require native compaction. Missing controls and under-sampled arms fail closed.

Metric separation

Live evaluation must not collapse different optimisations into one "context savings" number. It must report separately:

1. transport bytes actually sent;
2. provider-visible logical input;
3. provider cache read/write/input-token behaviour where exposed;
4. CPG recovery overhead: hidden rounds, recovery calls/bytes, restarts, latency and output-token cost.

TDD / rollout gate

The original first commit is RED and pins the decision gate: zero protocol violations, 2pp task-success non-inferiority, at least 10% cost-per-success improvement, <=15% p90 total-latency regression, and live-provider evidence required.

The deep-review hardening adds tests that reject aggregate-only evidence, reject any required arm below 30 attempts, and reject missing OpenAI WebSocket or provider-compaction controls when a cell declares those capabilities.

If the evidence is missing or the gate fails, recovery remains opt-in and CPG-07 full sidecar composition plus CPG-09 provider-stateful recovery remain deferred.

Stack

• Base: agent/context-projection-cp07 / replacement PR feat(context): add CPG-07 explicit sidecar composition gates #25
• Do not merge before its parent.

[github-actions]

✅ Deterministic PR hygiene checks passed.

[github-actions]

✅ READY

• all PR quality gates passed.

Hygiene

✅ Deterministic PR hygiene checks passed.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review (2)

• ^dev$
• ^preview$

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 548e1178-cedb-4005-bd8c-356de2f07650

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}