Redesign the background reviewer as an eval-driven generator-reviewer loop

[5o1]

Problem

Mineprogress performs update generation, review, recovery, and submission as an unattended background workflow after the user explicitly creates or binds an item. The current reviewer has evolved through several incremental fixes, causing its prompt and checklist to accumulate overlapping rejection conditions without model-level regression evaluation.

This now produces excessive false rejections. In particular, reviewers may reject otherwise valid automatic updates because the background worker lacks fresh, current-turn user authorization—even though user participation is neither available nor expected in this workflow.

The current design also conflates two different conditions:

• Durable evidence exists but the generated plan failed to represent it.
• The source context genuinely lacks enough evidence.

missing currently describes the first condition, but the backend maps it to awaitingEvidence, causing Mineprogress to wait for future main-thread context that may never arrive. Reviewer feedback is not delivered to the main worker, so this cannot reliably resolve itself.

Non-negotiable product requirements

• After explicit create or bind, the entire background workflow must operate without user participation.
• Review must not require fresh user authorization, confirmation, manual correction, or future conversation input.
• Binding establishes the scope for automatic maintenance. The backend operation allowlist and static validation—not the reviewer—enforce whether an operation is permitted.
• The reviewer and generator must interact across bounded revision rounds. Review feedback must return directly to the background generator.
• Reviewer findings must accumulate across rounds; fixing the latest finding must not discard earlier requirements.
• The reviewer owns semantic privacy sanitization. It must identify personal or sensitive information, provide remediation without repeating the sensitive value, and verify the sanitized candidate.
• Deterministic privacy checks remain defense in depth rather than replacing semantic review.
• Fixable candidate defects must not enter awaitingEvidence. The generator should revise omissions, remove unsupported claims, narrow statements, or produce a safe no-op using the existing evidence.
• Reviewer findings must never depend on being surfaced to the foreground agent or user.
• The interaction and its unresolved findings must survive process interruption and resume automatically.

Required investigation and design work

This should be treated as a prompt-engineering and evaluation task, not another isolated checklist edit.

The implementation should:

1. Reconstruct the intent and regressions behind previous reviewer prompt changes.
2. Define clear responsibility boundaries between deterministic validation, semantic review, generation, and submission.
3. Replace the current coarse approve/reject + reason exchange with an evaluated interaction protocol that supports actionable, non-leaking findings and revision verification.
4. Remove the incorrect missing → awaitingEvidence behavior for generator omissions.
5. Distinguish hard safety failures, revisable content problems, legitimate no-ops, and infrastructure failures.
6. Establish an anonymized evaluation corpus covering normal approvals, safe no-ops, unsupported claims, irrelevant context expansion, status changes, preservation rules, authorization-related false rejections, and semantic privacy failures.
7. Compare candidate prompts and protocols against the current baseline using repeated real-model evaluations.
8. Document the evaluation method, selected metrics, trade-offs, and results. The contributor is responsible for proposing justified acceptance thresholds.

Changes may include reviewer prompts, schemas, interaction state, retry behavior, and evaluation tooling. They should not weaken deterministic operation boundaries or require user involvement.

Acceptance criteria

• A valid automatic update is not rejected solely because no fresh user authorization exists.
• Reviewer findings are fed back to the generator and verified within the same autonomous workflow.
• Personal information is semantically detected, remediated, and re-reviewed.
• Generator omissions do not cause the system to wait blindly for future main-thread evidence.
• Legitimate no-op plans can complete without repeated rejection.
• Findings remain durable across revision rounds and process interruption.
• Existing static safety constraints remain enforced.
• The final design includes reproducible model-level evaluation results, not only mocked unit tests.
• Documentation clearly explains the reviewer’s authority boundary and autonomous recovery behavior.

[5o1]

The key focus is on ensuring reliable, long-term operation without human intervention.