diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..e61210e --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,146 @@ +# Contributing to Yada + +Thanks for helping improve Yada. This guide covers local setup, validation, and +the Git workflow expected for pull requests. + +Before changing behavior, read the [architecture](docs/dev/architecture.md). +For failures and trace inspection, use the +[debugging guide](docs/dev/debugging.md). User-facing commands belong in the +[CLI reference](docs/cli-reference.md). + +## Development setup + +Fork `GenTang/Yada` on GitHub, then clone your fork and add the canonical +repository as `upstream`: + +```bash +git clone https://github.com/YOUR-USERNAME/Yada.git +cd Yada +git remote add upstream https://github.com/GenTang/Yada.git +git remote -v + +uv sync --locked --dev +``` + +Create a feature branch from the latest `upstream/main`: + +```bash +git fetch upstream +git switch -c feature/short-description upstream/main +``` + +Keep unrelated changes in separate branches and pull requests. Do not commit +generated workspaces, evaluation results, traces, virtual environments, caches, +API keys, or other secrets. + +## Make and validate a change + +Run focused tests while developing, then run the full local CI suite before +opening or updating a pull request: + +```bash +uv run --frozen ruff check . +uv run --frozen ruff format --check . +uv run --frozen pytest tests/ -v +``` + +To apply Ruff formatting locally: + +```bash +uv run --frozen ruff format . +``` + +CI runs lint, formatting, and tests on Python 3.11 and 3.12. If a behavior change +cannot be covered by a deterministic offline test, explain why in the pull +request and provide the smallest reproducible trace or benchmark case available. + +Documentation changes should keep `README.md` and `README-cn.md` structurally +aligned. Never put a real API key or an unreviewed debug trace in an issue or PR. + +## Keep your branch current with rebase + +Yada uses a rebase workflow. Update your feature branch from `upstream/main` +without creating merge commits: + +```bash +git fetch upstream +git rebase upstream/main +``` + +Do not use `git merge upstream/main` on a feature branch. A linear history keeps +review and later bisection focused on the actual change. + +### Resolve rebase conflicts + +When Git stops on a conflict: + +```bash +git status +# Edit each conflicted file and remove conflict markers. +git add path/to/resolved-file +git rebase --continue +``` + +Repeat until the rebase finishes. To abandon the entire attempt and restore the +branch to its pre-rebase state: + +```bash +git rebase --abort +``` + +Use `git rebase --skip` only when the stopped commit is genuinely redundant; it +drops that commit from the rewritten branch. + +## Clean up commits before the PR + +Use interactive rebase to reorder, reword, fix up, or squash noisy development +commits into a small set of logical changes: + +```bash +git fetch upstream +git rebase -i upstream/main +``` + +Do not squash unrelated behavior, tests, and documentation merely to reach one +commit. The goal is reviewable history, not a specific commit count. + +Then run the full validation suite again and push the branch: + +```bash +git push -u origin feature/short-description +``` + +Open a pull request against `GenTang/Yada:main` and complete the repository's PR +template with: + +- what changed and why; +- the exact validation commands and results; +- benchmark, token, latency, or trace impact, or `N/A` when not applicable. + +## Update a PR after review + +Make the requested changes, commit them, and rebase again before updating the +remote branch: + +```bash +git fetch upstream +git rebase upstream/main +uv run --frozen ruff check . +uv run --frozen ruff format --check . +uv run --frozen pytest tests/ -v +git push --force-with-lease origin feature/short-description +``` + +Rebase rewrites commit IDs, so a normal push will be rejected. Use +`--force-with-lease`, never plain `--force`: the lease refuses to overwrite +remote work you have not seen. + +If another contributor also writes to your branch, coordinate before rebasing +or force-pushing it. + +## Review scope + +A pull request is ready for review when it is focused, tested, documented, and +rebased onto the current `upstream/main`. Maintainers may ask for additional +benchmark evidence when a change affects prompts, tools, context, verification, +or model-call behavior. diff --git a/README-cn.md b/README-cn.md new file mode 100644 index 0000000..72f2c70 --- /dev/null +++ b/README-cn.md @@ -0,0 +1,61 @@ +# Yada + +**Yet Another DeepSeek Agent** 是一个为 DeepSeek V4 构建的小型、可审计 +Coding Agent Harness。给它一个任务和一个 Git 仓库,Yada 会检查代码、应用经过 +校验的 Patch、运行验证,并记录完整执行轨迹。 + +[English README](README.md) + +> 只想使用 Yada?直接阅读下面的**快速开始**。想贡献代码?请从 +> [CONTRIBUTING.md](CONTRIBUTING.md) 和[开发者文档](docs/dev/architecture.md) +> 开始。 + +Yada 目前处于 Alpha 阶段。仓库已经测试本地 Agent 闭环,但尚未宣称任何对比 +评测结果。 + +## 运行条件 + +- Python 3.11+ +- Git +- [uv](https://docs.astral.sh/uv/)(推荐) +- DeepSeek API Key + +## 快速开始 + +```bash +git clone https://github.com/GenTang/Yada.git +cd Yada +uv sync --locked --dev + +export DEEPSEEK_API_KEY="sk-..." + +uv run yada "修复 parser 的边界问题,并运行相关测试" \ + --workspace /path/to/repository +``` + +Yada 默认会在运行仓库命令前请求确认。只有在可信、一次性的隔离环境中才应使用 +`--yes`: + +```bash +uv run yada --task-file issue.md --workspace /workspace --yes +``` + +## 运行时会发生什么 + +Yada 会打印每轮 DeepSeek 调用和工具执行,最后报告任务是否通过验证门槛。默认 +Trace 保存在目标仓库的 `.yada/runs/` 目录下。 + +仓库测试可以执行任意代码。Yada 提供 Guardrail,但不是完整的操作系统沙箱; +处理陌生项目时请使用一次性 VM 或容器。 + +## 更多文档 + +- [配置](docs/configuration.md):其他安装方式、API Key、模型参数、命令策略和 + Trace Level。 +- [CLI 参考](docs/cli-reference.md):`yada`、`yada eval` 和 `yada-trace`。 +- [贡献指南](CONTRIBUTING.md):开发环境、验证命令和基于 Rebase 的 PR 流程。 +- [架构](docs/dev/architecture.md):Agent 循环、工具、Patch 事务、评测边界与 + 安全不变量。 +- [调试](docs/dev/debugging.md):测试、Trace 检查与可复现 Issue。 + +Yada 使用 [MIT License](LICENSE)。 diff --git a/README.md b/README.md index f1cfc3a..8f5802e 100644 --- a/README.md +++ b/README.md @@ -1,271 +1,64 @@ # Yada -**Yet Another DeepSeek Agent** — a small, auditable coding-agent harness built -specifically for DeepSeek V4. +**Yet Another DeepSeek Agent** is a small, auditable coding-agent harness built +for DeepSeek V4. Give it a task and a Git repository; Yada inspects the code, +applies a checked patch, runs verification, and records a trace of the run. -[中文说明](README.zh-CN.md) +[中文说明](README-cn.md) -Yada is deliberately narrow: one agent loop, separate planning and execution -boundaries, one append-only conversation, five tools, version-checked patches, -and a verification gate. The runtime has no third-party Python dependencies. -Development checks use Ruff and pytest. +> Just want to use Yada? Follow **Quick start** below. Want to contribute? +> Start with [CONTRIBUTING.md](CONTRIBUTING.md) and the +> [developer docs](docs/dev/architecture.md). -> Alpha status: the offline agent loop is tested, but no comparative benchmark -> result is claimed yet. +Yada is currently alpha software. The local agent loop is tested, but the +project does not claim a comparative benchmark result yet. -## Generic evaluation +## Requirements -Yada includes a benchmark-neutral evaluation layer. `EvalRunner` composes any -`BenchmarkAdapter` with any `AgentAdapter`; the initial adapters cover local -JSON manifests, SWE-bench, native Yada, and arbitrary external commands. - -The repository includes one portable SWE-bench Verified development case. Its -first run fetches the exact pytest commit and creates a locked Python 3.9 task -environment; later runs reuse those caches while keeping each agent workspace -fresh: - -```bash -uv run yada eval \ - --case benchmarks/swebench_verified/pytest-10051 \ - --agent yada \ - --yes -``` - -This produces a real local verdict from one FAIL_TO_PASS and 15 PASS_TO_PASS -tests, but it is not an official Docker score. The checkout lives under -`.yada/cache/evals/`; the task recipe and its own `uv.lock` are committed. - -For SWE-bench, Yada produces the patch and official `predictions.jsonl` while -delegating the verdict to the Docker-based `swebench.harness.run_evaluation`. -See [docs/evaluation.md](docs/evaluation.md) for manifests, external-agent -templates, Docker prerequisites, and fair-comparison constraints. - -## Why this exists - -General-purpose harnesses can run DeepSeek, but they are not necessarily shaped -around DeepSeek's tool-use and context behavior. Yada is a compact research -vehicle for testing model-native harness ideas with reproducible trajectories -and ablations. - -The current hypotheses are: - -1. A tiny, stable tool schema reduces tool-call failures. -2. SHA-bound unified diffs prevent stale and ambiguous edits. -3. Structured, bounded command observations improve recovery after test failures. -4. An append-only conversation preserves DeepSeek prefix-cache opportunities. +- Python 3.11+ +- Git +- [uv](https://docs.astral.sh/uv/) (recommended) +- A DeepSeek API key ## Quick start -Requirements: Python 3.11+, Git, and a DeepSeek API key. - -The recommended workflow uses [uv](https://docs.astral.sh/uv/): - ```bash +git clone https://github.com/GenTang/Yada.git cd Yada uv sync --locked --dev -export DEEPSEEK_API_KEY="sk-..." - -uv run yada "Fix the failing parser edge case and run the relevant tests" \ - --workspace /path/to/repository -``` - -The package also works with standard library tooling and pip: -```bash -cd Yada -python3 -m venv .venv -.venv/bin/python -m pip install -e . export DEEPSEEK_API_KEY="sk-..." -yada "Fix the failing parser edge case and run the relevant tests" \ +uv run yada "Fix the failing parser edge case and run the relevant tests" \ --workspace /path/to/repository ``` -Yada asks before every repository command by default. For autonomous execution -inside a disposable sandbox: - -```bash -yada "Fix the issue described in issue.md" \ - --workspace /workspace \ - --yes -``` - -You can also pass a task file: - -```bash -yada --task-file issue.md --workspace . -``` - -The default model is `deepseek-v4-pro`, thinking is enabled, and reasoning -effort is `max`. These can be changed with `--model`, `--no-thinking`, and -`--reasoning-effort`. - -## Docker - -The container limits filesystem exposure to the mounted repository. It is not a -network sandbox. +Yada asks before running repository commands. Use `--yes` only inside a trusted, +disposable environment: ```bash -docker build -t yada . -docker run --rm -it \ - -e DEEPSEEK_API_KEY \ - -v "/path/to/repository:/workspace" \ - yada "Fix the failing test" --workspace /workspace --yes +uv run yada --task-file issue.md --workspace /workspace --yes ``` -## The loop - -```text -stable prompt + tool schema - ↓ -DeepSeek tool call - ↓ -validate → approve → execute - ↓ -bounded structured observation - ↓ -append and repeat - ↓ -finish only after post-patch verification -``` - -Tools: - -- `search_code`: ripgrep-backed repository search, with a Python fallback. -- `read_file`: bounded numbered reads plus SHA-256. -- `apply_patch`: Git-style unified diffs checked against every file hash. -- `run_command`: argv-only command execution with an allowlist and approval gate. -- `finish`: rejected until a test or build succeeds after the latest patch. - -DeepSeek thinking-mode `reasoning_content` is retained in memory and passed back -after tool calls, as required by the API. The default `--trace-level summary` -records compact context metrics. `--trace-level debug` additionally records the -complete sanitized provider payload and reasoning text for every model turn. -Summary traces replace reasoning with its length and hash. Both levels redact -common API keys, authorization values, tokens, passwords, and secrets. Debug -traces contain sensitive model context and must be handled accordingly. - -Capture a replayable debug trace during an evaluation: - -```bash -uv run yada eval \ - --case benchmarks/swebench_verified/pytest-10051 \ - --agent yada \ - --yes \ - --trace-level debug -``` - -Inspect a completed or interrupted run without manually scanning JSONL: - -```bash -uv run yada-trace \ - .yada/runs/fix-parser-edge-case__2026-08-02_20-26.jsonl -uv run yada-trace \ - eval-results/pytest-dev__pytest-10051__2026-08-02_20-26.artifacts/yada-trace.jsonl \ - --step 8 -uv run yada-trace eval-results/__.artifacts/yada-trace.jsonl \ - --verbose -uv run yada-trace TRACE.jsonl --events -``` - -The default report groups each model request, response, planning decision, and -ordered tool executions into one agent step. Every summary includes physical -JSONL line references so the source evidence is immediately reachable with tools -such as `sed`. `--step` and `--verbose` expand sanitized model messages, tool -arguments, patches, stdout, and stderr inside grouped steps; `--events` retains a -line-prefixed flat timeline. The source JSONL remains the durable, -streaming-friendly record. Debug traces can contain source code and test output -even after secret redaction, so handle them as sensitive artifacts. See -[docs/tracing.md](docs/tracing.md) for the event reference, field-presence -semantics, lifecycle, and `jq` recipes. - -Default trace and evaluation paths use the system-local time at minute -precision. If a name already exists, Yada appends `(1)`, `(2)`, and so on before -the file or artifacts suffix, keeping the result JSON and artifacts directory on -the same number. - -## Safety model - -Yada provides guardrails, not a complete OS sandbox: - -- File tools reject workspace escapes, symlink escapes, `.git`, and `.yada`. -- Patches reject binary, rename, copy, mode, and symlink changes. -- Commands use argv arrays rather than a shell string. -- Shell `-c` and mutating Git subcommands are rejected. -- Secret-looking environment variables, including the DeepSeek key, are removed - from child command environments. -- Command execution asks for confirmation unless `--yes` is used. - -Repository tests are arbitrary code. Run unfamiliar repositories in a disposable -VM or a stronger sandbox. The included Dockerfile reduces filesystem exposure, -but repository code can still access the container network. - -## Development checks - -The test suite includes a fully offline fake-model run through read → patch → -test → finish, plus stale hash, path escape, secret environment, and verification -gate tests. Ruff provides the repository's lint and formatting gates. - -```bash -uv sync --locked --dev -uv run --frozen ruff check . -uv run --frozen ruff format --check . -uv run --frozen pytest tests/ -v -``` - -CI runs the same checks on Python 3.11 and 3.12. Without uv, install the runtime -project with `python3 -m pip install -e .`, install `pytest` and `ruff` separately, -then run the equivalent commands. These tools remain development dependencies and -do not increase Yada's runtime dependency footprint. - -## Project layout - -Yada uses a `src/` layout and keeps orchestration separate from execution: - -```text -src/yada/ -├── agents/ # thin loop, side-effect-free planner, and tool executor -├── models/ # model protocol and DeepSeek API adapter -├── environments/ # workspace boundary and command approval -├── tools/ # one module per tool plus the small dispatcher -├── traces/ # JSONL writer plus a human-readable diagnostic report -├── evals/ # generic runner plus benchmark and agent adapters -├── run/ # CLI entry point -└── utils/ # bounded-output helpers -benchmarks/ # portable recipes; generated checkouts stay in .yada/cache -tests/ -├── agents/ -├── evals/ -├── models/ -├── tools/ -├── traces/ -└── utils/ -``` - -`Planner` owns conversation policy and validates the next action without I/O. -`Executor` owns argument parsing, workspace side effects, and correlated tool -events. `Agent` only coordinates the two. This is a deliberately small seam—not -a second model call—but it prevents the main loop from accumulating every future -planning and execution policy. - -The package boundaries follow the useful parts of mini-SWE-agent's structure, -while Yada retains its own multi-tool protocol, SHA-bound patches, command -policy, and verification gate. +## What happens next -## Design lineage +Yada prints each DeepSeek turn and tool execution, then reports whether the task +passed its verification gate. Traces are written under the target repository's +`.yada/runs/` directory by default. -Yada learns from the simplicity of -[mini-SWE-agent](https://github.com/SWE-agent/mini-swe-agent), the reproducible -trajectory discipline of [SWE-agent](https://github.com/SWE-agent/SWE-agent), -and DeepSeek's official [thinking-mode](https://api-docs.deepseek.com/guides/thinking_mode) -and [tool-call](https://api-docs.deepseek.com/guides/tool_calls) contracts. The -implementation is original and intentionally smaller than those systems. +Repository tests can execute arbitrary code. Yada provides guardrails, not a +complete OS sandbox; use a disposable VM or container for unfamiliar projects. -See [docs/architecture.md](docs/architecture.md) for the detailed contracts and -planned ablations, and [docs/tracing.md](docs/tracing.md) for the trace schema. +## Learn more -## Current non-goals +- [Configuration](docs/configuration.md): installation alternatives, API key, + model settings, command policy, and trace levels. +- [CLI reference](docs/cli-reference.md): `yada`, `yada eval`, and `yada-trace`. +- [Contributing](CONTRIBUTING.md): development setup, validation, and the rebase + pull-request workflow. +- [Architecture](docs/dev/architecture.md): agent loop, tools, patch transaction, + evaluation boundaries, and safety invariants. +- [Debugging](docs/dev/debugging.md): tests, trace inspection, and reproducible + issue reports. -No TUI, IDE plugin, MCP, skills, subagents, web search, long-term memory, model -routing, automatic commits, or benchmark leaderboard. Those features should be -earned by evaluation evidence. +Yada is licensed under the [MIT License](LICENSE). diff --git a/README.zh-CN.md b/README.zh-CN.md deleted file mode 100644 index bbbea0c..0000000 --- a/README.zh-CN.md +++ /dev/null @@ -1,203 +0,0 @@ -# Yada - -**Yet Another DeepSeek Agent**:一个专为 DeepSeek V4 构建的小型、可审计 -Coding Agent Harness。 - -[English README](README.md) - -Yada 有意保持克制:单 Agent 循环、独立的规划/执行边界、追加式会话、5个工具、 -带版本校验的 Patch,以及“修改后必须通过测试才能完成”的验证门槛。运行时没有 -第三方 Python 依赖;开发验证使用 Ruff 和 pytest。 - -> 当前为 Alpha:离线 Agent 闭环已通过测试,但尚未宣称任何对比评测结果。 - -## 通用评测 - -Yada 内置了 Benchmark-neutral 的评测层。`EvalRunner` 将任意 -`BenchmarkAdapter` 与任意 `AgentAdapter` 组合起来;当前提供本地 JSON Manifest、 -SWE-bench、原生 Yada 和外部命令四个适配器。 - -仓库内置了一个可移植的 SWE-bench Verified 开发用例。首次运行会拉取 pytest 的 -精确 commit,并创建锁定的 Python 3.9 任务环境;后续运行复用缓存,但每个 Agent -仍获得全新的工作区: - -```bash -uv run yada eval \ - --case benchmarks/swebench_verified/pytest-10051 \ - --agent yada \ - --yes -``` - -该命令会运行 1 个 FAIL_TO_PASS 和 15 个 PASS_TO_PASS 测试,产生真实的本地 -resolved/unresolved 判定,但不等同于官方 Docker 成绩。源码缓存在 -`.yada/cache/evals/`,仓库只提交任务配方和任务自己的 `uv.lock`。 - -运行 SWE-bench 时,Yada 只生成 Patch 和官方 `predictions.jsonl`,评分仍委托给 -`swebench.harness.run_evaluation` 的 Docker Harness。使用 `--grade-mode none` 可以 -只检查任务准备和预测文件,不会产生虚假的 resolved 结果。Manifest Schema、外部 -Agent 命令模板和公平比较约束见 [docs/evaluation.md](docs/evaluation.md)。 - -## 快速开始 - -需要 Python 3.11+、Git 和 DeepSeek API Key。 - -推荐使用 `uv`: - -```bash -cd Yada -uv sync --locked --dev -export DEEPSEEK_API_KEY="sk-..." - -uv run yada "修复 parser 的边界问题,并运行相关测试" \ - --workspace /path/to/repository -``` - -也可以继续使用标准 venv 与 pip: - -```bash -cd Yada -python3 -m venv .venv -.venv/bin/python -m pip install -e . -export DEEPSEEK_API_KEY="sk-..." - -yada "修复 parser 的边界问题,并运行相关测试" \ - --workspace /path/to/repository -``` - -Yada 默认会在运行仓库命令前请求确认。在一次性沙箱中可以启用自动执行: - -```bash -yada --task-file issue.md --workspace /workspace --yes -``` - -默认使用 `deepseek-v4-pro`、开启思考模式,并把推理强度设为 `max`。 - -## 最小闭环 - -```text -稳定 Prompt + Tool Schema - ↓ -DeepSeek 工具调用 - ↓ -校验 → 审批 → 执行 - ↓ -结构化、限长的 Observation - ↓ -追加会话并继续 - ↓ -最新 Patch 通过验证后才允许 finish -``` - -工具只有5个: - -- `search_code`:优先使用 ripgrep,缺失时使用 Python 回退。 -- `read_file`:分段读取带行号内容,并返回 SHA-256。 -- `apply_patch`:使用 Git Unified Diff,并检查全部目标文件 Hash。 -- `run_command`:无 Shell 的 argv 执行、命令白名单和用户审批。 -- `finish`:最新修改后没有成功测试或构建时直接拒绝。 - -DeepSeek 思考模式要求工具轮次继续回传 `reasoning_content`。Yada 会在内存中 -保留并正确回传。默认的 `--trace-level summary` 只记录紧凑的上下文指标; -`--trace-level debug` 还会保存每轮模型请求的完整脱敏 provider payload。 -JSONL 默认只保留 reasoning 的长度和 Hash,并脱敏常见 API key、 -Authorization、token、password 和 secret。只有显式使用 -`--trace-reasoning` 才会落盘完整推理。 - -在评测中生成可还原的 debug trace: - -```bash -uv run yada eval \ - --case benchmarks/swebench_verified/pytest-10051 \ - --agent yada \ - --yes \ - --trace-level debug -``` - -无需手工翻阅 JSONL,可以直接生成关联后的诊断时间线: - -```bash -uv run yada-trace .yada/runs/20260801T120000.000000Z.jsonl -uv run yada-trace eval-results/.artifacts/yada-trace.jsonl --step 8 -uv run yada-trace eval-results/.artifacts/yada-trace.jsonl --verbose -uv run yada-trace TRACE.jsonl --events -``` - -默认报告会按 Agent step 归组模型请求、响应、规划决定和有序工具执行,并为 -step、模型调用、工具执行和协议事件显示真实 JSONL 行号。`--step` 和 -`--verbose` 会在分组内展开脱敏后的模型消息、工具参数、Patch、stdout 和 -stderr;`--events` 可切回带物理行号的平铺时间线。Debug trace 脱敏后仍可能 -包含源码和测试输出,应当作敏感 artifact 处理。 - -默认 trace 和评测路径使用精确到分钟的系统本地时间,不包含秒和小数秒。 -若名称已存在,Yada 会在文件或 `.artifacts` 后缀前依次添加 `(1)`、`(2)`; -同一次评测的结果 JSON 与 artifacts 目录始终使用相同编号。 - -## 安全边界 - -Yada 提供 Guardrail,但不是完整的操作系统沙箱: - -- 文件工具拒绝工作区逃逸、符号链接逃逸、`.git` 和 `.yada`。 -- Patch 拒绝二进制、重命名、复制、权限模式和符号链接变更。 -- 命令使用 argv 数组,拒绝 Shell `-c` 和修改性 Git 子命令。 -- 子进程环境会移除 API Key、Token、Secret 等变量。 -- 默认每条命令都需要确认;`--yes` 仅应在隔离环境使用。 - -仓库测试本身就是任意代码。陌生仓库应放在一次性 VM 或更强的沙箱中运行。 -Dockerfile 能限制文件系统暴露,但不会阻断容器网络。 - -## 开发验证 - -测试套件包含一个完全离线的 Fake Model 端到端流程,以及过期 Hash、路径 -逃逸、密钥环境变量和验证门槛测试;Ruff 负责 Lint 和格式检查: - -```bash -uv sync --locked --dev -uv run --frozen ruff check . -uv run --frozen ruff format --check . -uv run --frozen pytest tests/ -v -``` - -CI 会在 Python 3.11 和 3.12 上运行同样的检查。没有 uv 时,可使用 -`python3 -m pip install -e .` 安装运行时项目,再单独安装 `pytest` 和 `ruff` -并执行对应命令。它们只是开发依赖,不会增加 Yada 的运行时依赖。 - -## 项目结构 - -Yada 采用 `src/` 布局,并把 Agent 编排与具体执行分离: - -```text -src/yada/ -├── agents/ # 薄编排循环、无副作用 Planner 与工具 Executor -├── models/ # 模型协议与 DeepSeek API 适配器 -├── environments/ # 工作区边界与命令审批 -├── tools/ # 每个工具一个模块,以及小型分发器 -├── traces/ # JSONL 轨迹记录与人类可读诊断报告 -├── evals/ # 通用 Runner、Benchmark 与 Agent 适配器 -├── run/ # CLI 入口 -└── utils/ # 输出截断等通用逻辑 -benchmarks/ # 可移植任务配方;运行时 checkout 放在 .yada/cache -tests/ -├── agents/ -├── evals/ -├── models/ -├── tools/ -└── traces/ -``` - -`Planner` 只负责会话策略和下一步动作校验,不做 I/O;`Executor` 负责参数解析、 -工作区副作用和带关联 ID 的工具事件;`Agent` 只负责编排二者。它目前不是一次 -额外的模型调用,但已经为未来的独立规划模型留下替换点,避免主循环演变成上帝类。 - -边界设计参考了 mini-SWE-agent 的有效结构,但 Yada 保留自己的多工具协议、 -基于 SHA 的 Patch、命令策略和验证门槛。 - -## 借鉴与差异 - -Yada 借鉴了 [mini-SWE-agent](https://github.com/SWE-agent/mini-swe-agent) -的极简循环、[SWE-agent](https://github.com/SWE-agent/SWE-agent) 的可复现轨迹, -并直接遵循 DeepSeek 官方的[思考模式](https://api-docs.deepseek.com/guides/thinking_mode) -和[工具调用](https://api-docs.deepseek.com/guides/tool_calls)契约。代码为独立实现, -目标是成为一个便于实验和消融的 DeepSeek-native Harness。 - -当前不做 TUI、IDE、MCP、Skills、多 Agent、联网、长期记忆、模型路由或自动 -提交。详细设计见 [docs/architecture.md](docs/architecture.md)。 diff --git a/docs/architecture.md b/docs/architecture.md deleted file mode 100644 index ee215a6..0000000 --- a/docs/architecture.md +++ /dev/null @@ -1,129 +0,0 @@ -# Yada architecture - -Yada is a single-loop coding harness, not an orchestration framework. Its -directory boundaries make the minimal loop easier to test without turning it -into a framework of abstractions. - -## Package map - -```text -run/cli.py - ├── agents/default.py - ├── agents/planning.py - ├── agents/executor.py - │ └── tools/runner.py - │ ├── environments/workspace.py - │ ├── environments/approval.py - │ ├── tools/search.py - │ ├── tools/read.py - │ ├── tools/patch.py - │ ├── tools/command.py - │ └── tools/finish.py - ├── models/base.py ← models/deepseek.py - └── traces/jsonl.py ← traces/report.py - └── evals/cli.py - └── evals/runner.py - ├── evals/benchmarks/{local,swebench}.py - ├── evals/benchmarks/local_{source,environment}.py - └── evals/agents/{yada,command}.py -benchmarks/ - └── swebench_verified/pytest-10051/ # recipe only; no checkout or venv -``` - -- `agents/default.py`: coordinates state and the step limit; it owns no tool policy. -- `agents/planning.py`: side-effect-free conversation and batch protocol policy. -- `agents/executor.py`: parses, executes, and traces tool calls and their results. -- `models`: defines the completion boundary and implements DeepSeek transport. -- `environments`: owns access to the local workspace and command approval. -- `tools`: contains stateless handlers; `runner.py` composes shared tool state. -- `traces`: records correlated append-only events and renders diagnostic timelines. -- `evals`: composes benchmark preparation/grading with interchangeable agents. -- `benchmarks`: stores reproducible task recipes, canonical public inputs, - locked task environments, and external graders. -- `run`: parses user configuration and assembles the runtime. -- `utils`: holds small mechanics shared by otherwise independent modules. - -Dependencies point inward through these contracts. A tool handler does not know -about the model or agent, and the DeepSeek adapter does not know about tools. - -## Invariants - -1. The system prompt and tool schema stay stable for the whole run. -2. Messages are append-only; assistant `reasoning_content` is retained in memory. -3. File mutation is possible only through a checked unified diff. -4. Every existing patch target must have been read at the exact current SHA-256. -5. Every patch invalidates prior verification. -6. `finish` succeeds only if a `test` or `build` command passed at the latest revision. -7. All trace events are append-only JSONL records with a run ID and sequence. - -## Planner/executor seam - -The current `Planner` is a deterministic policy layer, not a second LLM call. It -builds the stable prompt prefix, recovers from text-only responses, and rejects -unsafe multi-call `finish` batches. It has no access to the workspace. The -`Executor` owns argument decoding, `ToolRunner` side effects, durations, and -tool-call correlation IDs. - -This split prevents `default.py` from becoming the home of every future policy. -An experiment can replace `Planner` with an explicit plan-producing model or -state machine without changing tool safety, and can replace `Executor` for a -remote sandbox without changing conversation policy. - -## Patch transaction - -`read_file` returns a content hash. `apply_patch` parses every `diff --git` -header and requires an exact set of `{path, sha256}` declarations. The patch is -rejected if a file changed after it was read, if a path leaves the workspace, or -if Git cannot apply it cleanly. A new file uses the sentinel `NEW`. - -This contract is intentionally stricter than a generic text-replace tool and -smaller than maintaining session-local snippet objects. - -## Command observations - -Commands use an argv array and capture stdout, stderr, exit code, duration, and -timeout separately. Long output keeps a larger prefix plus a suffix containing -the final error frames. Successful `inspect` commands do not satisfy the -verification gate; only `test` and `build` do. - -## Trace diagnostics - -JSONL is the crash-safe source of truth, but it is not the debugging interface. -Schema v2 has two capture levels. `summary` records context size and event timing; -`debug` also stores the sanitized provider payload built by the same client method -used for the HTTP request. Responses, planner decisions, tool calls, and tool -results remain correlated by step, request ID, and tool-call ID. `run_start` -records Yada version/commit, workspace base commit, case ID when available, and -the model configuration. - -`yada-trace PATH` renders one source-located section per agent step. `--step N` -expands one complete request → response → tools step, while `--verbose` expands -every grouped step and `--events` provides the line-prefixed flat timeline. -Reasoning is length/hash-redacted in summary traces and automatically retained in -debug traces. Common secret keys and bearer/API-key-like text are redacted in -both modes. A debug trace can still contain reasoning, source code, and test -output and must be handled as a sensitive artifact. -The complete event and field reference lives in [tracing.md](tracing.md). - -The MVP stores a full sanitized request snapshot per turn. This deliberately -favors deterministic inspection over delta complexity; content-addressed prompts -or message deltas can replace it later if measured trace size justifies the added -reader and compatibility cost. - -## Security boundary - -Path checks and command policy reduce accidental damage, but `python`, test -runners, build tools, and repository scripts execute arbitrary code. A real -benchmark deployment should run each task in a disposable container and run the -hidden grader in a separate container after the agent exits. - -## Evaluation-driven next steps - -Do not add a feature until a frozen baseline exposes a failure class. Candidate -ablations are: - -1. SHA-bound patch vs ordinary unified diff. -2. Bounded structured output vs raw terminal output. -3. Append-only stable prefix vs context rebuilding. -4. Free exploration vs mandatory plan. -5. One DeepSeek model vs a cheaper exploration model followed by a stronger repair model. diff --git a/docs/cli-reference.md b/docs/cli-reference.md new file mode 100644 index 0000000..cd6f6c1 --- /dev/null +++ b/docs/cli-reference.md @@ -0,0 +1,233 @@ +# CLI reference + +Yada installs two entry points: + +- `yada`: run the coding agent or an evaluation; +- `yada-trace`: inspect a JSONL execution trace. + +Examples below use `uv run`. After editable pip installation, omit `uv run`. +Run `uv run yada --help`, `uv run yada eval --help`, or +`uv run yada-trace --help` for the parser-generated reference. + +## `yada` + +```text +yada TASK [OPTIONS] +yada --task-file FILE [OPTIONS] +``` + +Provide exactly one task source. The workspace defaults to the current directory. + +```bash +uv run yada "Fix the parser boundary case and run its tests" \ + --workspace /path/to/repository + +uv run yada --task-file issue.md --workspace /path/to/repository +``` + +### Options + +| Option | Meaning | Default | +| --- | --- | --- | +| `TASK` | Natural-language coding task. | — | +| `--task-file PATH` | Read the task from a UTF-8 file. | — | +| `--workspace PATH` | Target Git workspace. | Current directory | +| `--model NAME` | DeepSeek model name. | `DEEPSEEK_MODEL` or `deepseek-v4-pro` | +| `--base-url URL` | DeepSeek-compatible API base URL. | `DEEPSEEK_BASE_URL` or `https://api.deepseek.com` | +| `--reasoning-effort high\|max` | Thinking effort. | `max` | +| `--thinking` / `--no-thinking` | Enable or disable thinking. | Enabled | +| `--max-steps N` | Maximum model turns. | `30` | +| `--max-output-tokens N` | Maximum tokens requested per completion. | `16384` | +| `--api-timeout SECONDS` | Timeout for one model request. | `300` | +| `--command-timeout SECONDS` | Default repository-command timeout. | `120` | +| `--command-policy ask\|allow\|deny` | Repository-command approval policy. | `ask` | +| `--yes` | Alias for command policy `allow`. | Off | +| `--trace PATH` | Exact JSONL trace path. | `.yada/runs/__