Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions docs/cli-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ uv run yada --task-file issue.md --workspace /path/to/repository
| `--base-url URL` | DeepSeek-compatible API base URL. | `DEEPSEEK_BASE_URL` or `https://api.deepseek.com` |
| `--reasoning-effort high\|max` | Thinking effort. | `max` |
| `--thinking` / `--no-thinking` | Enable or disable thinking. | Enabled |
| `--editing-strategy patch-only\|replace-first` | Freeze the run-level editing policy and model-facing edit tools. | `replace-first` |
| `--max-steps N` | Maximum model turns. | `30` |
| `--max-output-tokens N` | Maximum tokens requested per completion. | `16384` |
| `--api-timeout SECONDS` | Timeout for one model request. | `300` |
Expand Down Expand Up @@ -289,12 +290,28 @@ container; Yada's automatic Agent command container applies only to the native
| `--max-steps N` | Model-turn budget. | `30` |
| `--wall-time SECONDS` | Comparable wall-time budget. | `1800` |
| `--max-output-tokens N` | Per-completion token limit. | `16384` |
| `--editing-strategy patch-only\|replace-first` | Native Yada editing policy. | `replace-first` |

The native agent also accepts the model, thinking, timeout, command-policy, and
trace-level options documented for `yada`. A deployment-level supervisor should
enforce a hard wall-time limit for an in-process native agent. Use the same task,
base commit, model budget, network policy, and grader when comparing agents.

For a controlled editing-policy comparison, run the same case once with each
strategy while holding the remaining options constant:

```bash
uv run yada eval --case CASE --agent yada --yes \
--editing-strategy patch-only --output results/patch-only.json

uv run yada eval --case CASE --agent yada --yes \
--editing-strategy replace-first --output results/replace-first.json
```

The result's `agent_run.details` records `editing_strategy` and
`editing_metrics`; the benchmark grader remains authoritative for resolved-task
status.

Evaluation exits with `0` for `resolved`, `1` for `unresolved`, and `2` for
errors or non-verdict outcomes such as skipped grading.

Expand Down
24 changes: 24 additions & 0 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,30 @@ When thinking is enabled, Yada retains `reasoning_content` across tool-calling
turns as required by DeepSeek. In non-thinking mode, Yada uses ordinary automatic
tool selection.

## Editing strategy

Editing strategy is frozen for the complete run:

| Strategy | Editing tools shown to the model | Policy |
| --- | --- | --- |
| `patch-only` | `apply_patch` | Express every edit as a checked unified diff. |
| `replace-first` | `replace_text`, `apply_patch` | Prefer exact replacement for localized edits and use patch for unsuitable operations. |

`replace-first` is the default. Select `patch-only` explicitly when every edit must
use a checked unified diff:

```bash
uv run yada "Fix the localized parser bug" \
--workspace /path/to/repository \
--editing-strategy patch-only
```

Both strategies allow at most one editing tool call per Assistant turn. This
prevents a replacement and a precomputed patch from acting as an opaque
same-turn fallback. See the
[editing strategy design](dev/editing-strategy.md) for routing and recovery
rules.

## Command execution policy

Repository commands are independently validated and then handled by one of
Expand Down
35 changes: 19 additions & 16 deletions docs/dev/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ stable system prompt + tool schemas
append assistant + tool observations
repeat or verified finish
repeat or verified finish_task
```

`Agent` in `agents/default.py` owns the append-only message list, step limit,
Expand All @@ -85,8 +85,8 @@ requests satisfy DeepSeek's thinking/tool-call contract.

The current `Planner` is deterministic; it is not another model call. It builds
the initial prompt, interprets assistant output, recovers from text-only turns,
and rejects unsafe batches such as `finish` mixed with another tool. It has no
workspace access.
and rejects unsafe batches such as `finish_task` mixed with another tool or more than
one editing operation in one turn. It has no workspace access.

The `Executor` parses tool arguments, preserves model-provided call order,
invokes `ToolRunner`, and records correlated `tool_call` and `tool_result`
Expand All @@ -104,10 +104,12 @@ Yada exposes six tools:
| `replace_text` | Apply exact unique replacements to existing UTF-8 files. |
| `apply_patch` | Validate and apply a Git-style unified diff. |
| `run_command` | Run an approved argv array and return bounded structured output. |
| `finish` | End only after verification of the latest revision. |
| `finish_task` | End only after verification of the latest revision. |

`ToolRunner` composes shared workspace, approval, output-limit, and verification
state. Handlers remain otherwise stateless. File paths are resolved through the
state. At run start it freezes either the `patch-only` interface or the
`replace-first` interface; the implementation still contains both editing tools.
Handlers remain otherwise stateless. File paths are resolved through the
workspace boundary, which rejects absolute paths, `..` escapes, symlink escapes,
and access to `.git` or `.yada` internals.

Expand Down Expand Up @@ -147,7 +149,7 @@ secret-looking environment variables and return stdout, stderr, exit code,
timeout, and duration through the same tool result.

The model labels a command as `inspect`, `test`, or `build`. Only a successful
`test` or `build` verifies the current workspace revision. `finish` rejects the
`test` or `build` verifies the current workspace revision. `finish_task` rejects the
run when:

- no verification succeeded;
Expand Down Expand Up @@ -223,16 +225,17 @@ load, workspace, grading, cache, and artifact sequence.

## Core invariants

1. System prompt and tool schemas stay stable during a run.
2. Conversation messages are append-only.
3. DeepSeek reasoning is preserved across tool-call turns.
4. File mutation occurs only through a checked unified diff.
5. Existing patch targets must match their last-read SHA-256.
6. Every patch invalidates previous verification.
7. `finish` requires verification of the latest revision.
8. Trace events are append-only and self-correlating.
9. Benchmark grading happens outside the agent's tool boundary.
10. Hidden grading inputs never enter the official Agent command container.
1. Editing strategy, system prompt, and tool schemas stay stable during a run.
2. At most one editing operation executes from one Assistant turn.
3. Conversation messages are append-only.
4. DeepSeek reasoning is preserved across tool-call turns.
5. File mutation occurs only through a checked unified diff.
6. Existing patch targets must match their last-read SHA-256.
7. Every patch invalidates previous verification.
8. `finish_task` requires verification of the latest revision.
9. Trace events are append-only and self-correlating.
10. Benchmark grading happens outside the agent's tool boundary.
11. Hidden grading inputs never enter the official Agent command container.

## Security boundary

Expand Down
6 changes: 3 additions & 3 deletions docs/dev/debugging.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ uv run --frozen pytest tests/ -v
```

The suite includes a fully offline fake-model path through read → patch → test →
finish. Prefer fake clients and temporary Git repositories for agent-loop tests;
finish_task. Prefer fake clients and temporary Git repositories for agent-loop tests;
unit tests must not require a DeepSeek key or network access.

## Capture a useful trace
Expand Down Expand Up @@ -156,7 +156,7 @@ Every schema v2 record contains `schema_version`, `run_id`, `sequence`, UTC

| Event | Meaning | Main correlation |
| --- | --- | --- |
| `run_start` | Task, workspace, model config, trace level, and provenance. | `run_id` |
| `run_start` | Task, workspace, model config, editing strategy, frozen tool names, trace level, and provenance. | `run_id` |
| `model_request` | Attempted model turn; debug adds `payload`. | `step`, `request_id` |
| `assistant` | Model message, usage, metadata, finish reason, and latency. | `step`, `request_id` |
| `model_error` | Model request exception instead of an assistant response. | `step`, `request_id` |
Expand Down Expand Up @@ -245,7 +245,7 @@ SWE-bench score. Use the official Docker grader for published results; see the

- **`DEEPSEEK_API_KEY is not set`**: export the key in the shell launching Yada.
- **No request payload in a trace**: rerun with `--trace-level debug`.
- **`finish` rejected**: run a successful `test` or `build` after the latest patch.
- **`finish_task` rejected**: run a successful `test` or `build` after the latest patch.
- **Tool reports `ok` but tests failed**: inspect the command `exit_code`.
- **No `run_end`**: the process was interrupted or raised outside a graceful path.
- **Step limit reached**: inspect repeated reminders, failed tools, context growth,
Expand Down
Loading