This repository demonstrates using Opencode and its subagents to generate targeted code mutations and then run the test-suite against those mutants to evaluate and improve coding-agent robustness.
- Purpose: measure how well coding agents (and test suites) detect and recover from realistic code mutations.
- Approach: have an automated
mutatorsubagent create small, focused changes tosrc/modules and atestersubagent runpytestagainst the mutated variants to surface behavioral divergences.
- A mutator subagent produces a mutated variant of an existing module
(for example
src/numerical_integration_mutated.py). - The mutator records metadata/handshake information for the tester.
- A tester subagent runs the test-suite (
pytest) against the mutated module and records pass/fail outcomes. - Failures indicate divergences; these are used to evaluate and harden the coding-agent workflows.
- Install dev deps (if needed):
pip install -e "[dev]" - Run tests:
pytest - Run opencode:
opencode - use prompt for simple test:
run @code-mutator on numerical_integration_orig.py then run @code-tester on the mutated script
src/— source modules (original and mutated variants)tests/— unit tests that validate expected behavior.opencode/— opencode configuration and agent scripts.opencode/opencode.json- agent configuration
pytestruff check .ruff format .ruff check . && pytest
- This repo is a local simulation of Opencode-style subagent interactions and is intended for experimentation and evaluation rather than production use.
- Python 3.10+ is recommended.
Provided as-is for experimentation.