Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,8 @@ Before drawing any conclusion from a tool call, verify:
- __Don't conflate "test contamination" with "tool bug"__ — if a tool worked once and then fails, check the test state before assuming the tool is broken. Window switching, manual user interaction between calls, and stale controllers are common contamination sources.
- __`save_captured_image` writes into project bundle, not MaaMCP data dir__ — destination is `<bundle_root>/image/<subcategory>/<name>.png` (the path TemplateMatch's `template` field reads from). `bundle_root` is the directory passed to `Resource.post_bundle()`. For MAAGC it's `assets/resource/base/`; for MaaFramework sample it's `<repo>/sample/resource`. Default `overwrite=False` protects existing templates — pass `True` explicitly when updating.
- __`benchmark_node` measures wall-clock, not per-node timing__ — returned `latency_ms` is full `post_task → TaskDetail` time, including entry recognition overhead (~50-200ms). For per-node estimate subtract that baseline. `mean_score=None` with `successes=0` means the node never hit — threshold/ROI/template mismatch.
- __`run_pipeline` / `benchmark_node` accept per-run `pipeline_override`__ — field-level node overrides passed to `post_task` (same mechanism as interface.json's `pipeline_override`): tighten `roi`, tweak `expected`/`threshold`/`timeout` for single-node verification without editing pipeline files. Applies to one run only — the Resource is untouched. Node names not present in the loaded files show up in `warnings` (typo'd names silently no-op, so check them). `benchmark_node` rejects overriding `next` on entry/target (would break its isolation chain).
- __`run_pipeline` supports tool-side `timeout_seconds`__ — polls the task and calls `post_stop()` on expiry, returning `status="timeout"` + partial node details instead of blocking the MCP call indefinitely (a node that never matches burns its own `timeout`, 20s by default). Note: MaaFramework marks a stopped task's own status as succeeded — the tool reports `"timeout"` explicitly instead of trusting it. Recommended 5-15s for single-node verification; `None` (default) keeps the old unbounded behavior.

### Pipeline node tuning loop

Expand All @@ -180,7 +182,7 @@ When a TemplateMatch / OCR / ColorMatch node isn't reliable, iterate this loop (
4. `save_captured_image(cropped_path, bundle_root, subcategory, name)` → promote it to a TemplateMatch template
5. In pipeline JSON: `"recognition": "TemplateMatch", "template": "<subcategory>/<name>.png"`
6. `benchmark_node(cid, pipeline_path, node=<name>, iterations=10..50)` → inspect `mean_score`, `latency_ms`, `all_results_samples`
7. If `mean_score` < 0.85 or `successes < iterations`: tighten ROI (smaller `region`), raise `threshold`, or refresh the template with a fresh capture
7. If `mean_score` < 0.85 or `successes < iterations`: tighten ROI (smaller `region`), raise `threshold`, or refresh the template with a fresh capture — try candidate values via `pipeline_override={"<name>": {"roi": [...], "threshold": ...}}` first (no file edits), then write the winning values back into the pipeline JSON
8. Repeat 2-7 until stable

For MaaMCP-side pipeline infra testing, see `tests/test_dbg_pipeline.py` (gated by `@pytest.mark.integration`; skips if `MaaDbgControlUnit` DLL isn't shipped).
Expand Down
4 changes: 3 additions & 1 deletion CLAUDE_CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -168,6 +168,8 @@ OCR 模型和截图存储在平台特定的目录中:
- __别把"测试污染"误判成"工具 bug"__——工具用过一次后又失败,先检查测试状态(是否切窗、是否手动操作、controller 是否失效),再怀疑工具本身。
- __`save_captured_image` 写入项目 bundle,不是 MaaMCP 数据目录__——目标是 `<bundle_root>/image/<子分类>/<元素名>.png`(TemplateMatch 的 `template` 字段读取路径)。`bundle_root` 是传给 `Resource.post_bundle()` 的目录:MAAGC 是 `assets/resource/base/`;MaaFramework sample 是 `<repo>/sample/resource`。默认 `overwrite=False` 保护已有模板,更新时显式传 `True`。
- __`benchmark_node` 测的是 wall-clock,不是单节点耗时__——返回的 `latency_ms` 是 `post_task → TaskDetail` 总耗时,含 entry 识别开销(~50-200ms)。想估节点本身耗时减掉这段基线。`mean_score=None` 且 `successes=0` 表示一次都没命中——通常是阈值/ROI/模板漂移的信号。
- __`run_pipeline` / `benchmark_node` 支持单次 `pipeline_override`__——字段级节点覆盖,直接传给 `post_task`(与 interface.json 的 `pipeline_override` 同机制):单节点验证时收紧 `roi`、调 `expected`/`threshold`/`timeout`,不用改 pipeline 文件。只对单次运行生效——Resource 不被污染。覆盖的节点名若不在已加载文件中会出现在 `warnings`(拼错节点名时覆盖静默不生效,记得检查)。`benchmark_node` 拒绝覆盖 entry/target 的 `next`(会破坏隔离链路)。
- __`run_pipeline` 支持工具侧 `timeout_seconds`__——轮询任务状态,超时调 `post_stop()`,返回 `status="timeout"` + 部分节点详情,不再无限阻塞 MCP 调用(识别不命中的节点会烧满自己的 `timeout`,默认 20s)。注意:MaaFramework 把被 stop 的任务自身 status 标记为 succeeded——工具显式报 `"timeout"`,不信任它。单节点验证建议 5-15s;`None`(默认)保持旧的不限时行为。

### Pipeline 节点调参循环

Expand All @@ -179,7 +181,7 @@ TemplateMatch / OCR / ColorMatch 节点不稳定时,按这个循环迭代(is
4. `save_captured_image(cropped_path, bundle_root, subcategory, name)` → 提到 TemplateMatch 模板
5. pipeline JSON 里写:`"recognition": "TemplateMatch", "template": "<subcategory>/<name>.png"`
6. `benchmark_node(cid, pipeline_path, node=<name>, iterations=10..50)` → 看 `mean_score`、`latency_ms`、`all_results_samples`
7. `mean_score < 0.85` 或 `successes < iterations`:收紧 ROI(缩 `region`)、抬高 `threshold`、或重新截一张更准的模板
7. `mean_score < 0.85` 或 `successes < iterations`:收紧 ROI(缩 `region`)、抬高 `threshold`、或重新截一张更准的模板——先用 `pipeline_override={"<name>": {"roi": [...], "threshold": ...}}` 免改文件试参,确定后再把最终值写回 pipeline JSON
8. 重复 2-7 直到稳定

MaaMCP 侧 pipeline infra 集成测试见 `tests/test_dbg_pipeline.py`(标 `@pytest.mark.integration`;缺 `MaaDbgControlUnit` DLL 时自动 skip)。
Expand Down
24 changes: 23 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ Talk is cheap, 请看: **[🎞️ Bilibili 视频演示](https://www.bilibili.co
- `get_pipeline_protocol` - 获取 Pipeline 协议文档
- `save_pipeline` - 保存 Pipeline JSON 到文件(支持新建和更新)
- `load_pipeline` - 读取已有的 Pipeline 文件
- `run_pipeline` - 运行 Pipeline 并返回执行结果(支持单/多文件、Custom action agent 自动启动)
- `run_pipeline` - 运行 Pipeline 并返回执行结果(支持单/多文件、Custom action agent 自动启动、单次 `pipeline_override` 参数覆盖、工具侧 `timeout_seconds` 超时

### 🛑 Pipeline 终止

Expand Down Expand Up @@ -380,6 +380,28 @@ run_pipeline(
- 加载是原子的:所有预校验通过后才写入 Resource
- 节点驻留:已加载的节点持续驻留,切换 pipeline 集时调用 `clear_pipeline_resources()` 重置

### 单节点快速验证(pipeline_override + timeout_seconds)

调试单个节点时,不必反复改 pipeline 文件,也不必忍受整屏识别慢扫或无限阻塞:

```python
run_pipeline(
controller_id,
"main.json",
entry="点击设置", # 只从被测节点进入
start_agent=False, # 只测识别,不拉起 CustomAction 链
pipeline_override={ # 单次生效:不写文件、不污染 Resource
"点击设置": {"roi": [520, 20, 200, 80], "timeout": 3000}
},
timeout_seconds=10, # 工具侧超时:超过 10s 自动 post_stop
)
```

- `pipeline_override` 与 interface.json 的同名机制一致:**按字段合并**到已加载节点,
也可覆盖 `expected` / `threshold` / `enabled` 等任意字段;ROI 越准单次识别越快
- 超时返回 `status="timeout"` + 超时前已执行的节点详情,方便定位挂在哪个节点
- `benchmark_node` 同样支持 `pipeline_override`,调参循环免改文件

## 注意事项

📌 **Windows 自动化限制**:
Expand Down
23 changes: 22 additions & 1 deletion README_EN.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ Talk is cheap, see: **[🎞️ Bilibili Video Demo](https://www.bilibili.com/vid
- `get_pipeline_protocol` - Get Pipeline protocol documentation
- `save_pipeline` - Save Pipeline JSON to file (supports creating and updating)
- `load_pipeline` - Load an existing Pipeline file
- `run_pipeline` - Run Pipeline and return execution results (single/multi-file, auto-start Custom action agent)
- `run_pipeline` - Run Pipeline and return execution results (single/multi-file, auto-start Custom action agent, per-run `pipeline_override`, tool-side `timeout_seconds`)

### 🛑 Pipeline Termination

Expand Down Expand Up @@ -282,6 +282,27 @@ After Pipeline generation, AI automatically validates and optimizes:

If the Pipeline logic itself needs adjustment, AI can re-execute automation operations and combine old and new experiences to generate a more robust Pipeline.

### Fast Single-Node Verification (pipeline_override + timeout_seconds)

When debugging a single node, there is no need to repeatedly edit the pipeline file, suffer slow full-screen recognition, or let the call block forever:

```python
run_pipeline(
controller_id,
"main.json",
entry="ClickSettings", # enter from the node under test
start_agent=False, # recognition only, no CustomAction chain
pipeline_override={ # per-run only: no file edits, Resource untouched
"ClickSettings": {"roi": [520, 20, 200, 80], "timeout": 3000}
},
timeout_seconds=10, # tool-side cap: auto post_stop after 10s
)
```

- `pipeline_override` uses the same mechanism as interface.json's field: **field-level merge** into loaded nodes; any field (`expected` / `threshold` / `enabled` / ...) can be overridden — and the tighter the ROI, the faster each recognition
- On timeout the result carries `status="timeout"` plus the nodes executed before the stop, showing exactly where it hung
- `benchmark_node` accepts the same `pipeline_override`, making tuning loops file-edit-free

### Example Output

```json
Expand Down
Loading