Skip to content

docs(harbor): full pi-extension pairing for the nano arm (PR 16) #895

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
agentforce314 merged 1 commit into from
Aug 16, 2026
Merged

docs(harbor): full pi-extension pairing for the nano arm (PR 16) #895

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 9 additions & 6 deletions eval/harbor/RUN_NANO_TB21.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,12 +77,15 @@ what the A/B measures. Don't describe the runs as "same tools".

**Matching a pi run that used the TB extension** (vision_analyze +
websearch — e.g. `jobs/tb21-pi-flash-max-2`, which ran
`vision_model: gpt-5.6-luna`): add `--ak vision=openai:gpt-5.6-luna` to
the nano arm so nano registers its identically-named `vision_analyze`
tool. The 2026-08 run analysis found 4–5 of pi's exclusive wins used
vision/websearch (chess-best-move, video-processing, path-tracing…);
without the kwarg those tasks measure a capability gap, not the harness.
Nano has no websearch analog wired yet — note it when comparing.
`vision_model: gpt-5.6-luna` with TAVILY websearch): add
`--ak vision=openai:gpt-5.6-luna --ak websearch=1` to the nano arm
(runner: `NANO_VISION=openai:gpt-5.6-luna NANO_WEBSEARCH=1`) so nano
registers its identically-named `vision_analyze` plus WebSearch. The
2026-08 run analysis found 4–5 of pi's exclusive wins used
vision/websearch (chess-best-move ×3 vision, video-processing ×11+3,
mteb-leaderboard ×2 web, path-tracing…); without the kwargs those tasks
measure a capability gap, not the harness. Stock-vs-stock (no kwargs on
nano, `--ak tools=off` on pi) is the other clean pairing.


For a clean head-to-head against pi, pair **nano** with **stock pi**
Expand Down
Loading