Agent tool surface: submit_plan, one run_python, probe-loaded drafts - #161
Merged
Conversation
yichao-liang
force-pushed
the
bl-merge-09-run-refactor
branch
from
September 1, 2026 13:27
7c1e41b to
6fface9
Compare
yichao-liang
force-pushed
the
bl-merge-10-agent-tools
branch
2 times, most recently
from
September 1, 2026 13:56
5ba3975 to
ee86a9e
Compare
yichao-liang
force-pushed
the
bl-merge-09-run-refactor
branch
2 times, most recently
from
September 1, 2026 14:21
e58fc03 to
11bcbbe
Compare
yichao-liang
force-pushed
the
bl-merge-10-agent-tools
branch
from
September 1, 2026 14:21
ee86a9e to
621fd8b
Compare
yichao-liang
force-pushed
the
bl-merge-09-run-refactor
branch
from
September 1, 2026 14:43
11bcbbe to
e3de74e
Compare
yichao-liang
force-pushed
the
bl-merge-10-agent-tools
branch
from
September 1, 2026 14:43
621fd8b to
592db38
Compare
yichao-liang
force-pushed
the
bl-merge-09-run-refactor
branch
from
September 1, 2026 15:01
e3de74e to
77506ba
Compare
yichao-liang
force-pushed
the
bl-merge-10-agent-tools
branch
2 times, most recently
from
September 1, 2026 15:26
386d6af to
d073090
Compare
yichao-liang
force-pushed
the
bl-merge-09-run-refactor
branch
from
September 1, 2026 15:26
77506ba to
c1817b3
Compare
yichao-liang
changed the base branch from
bl-merge-09-run-refactor
to
master
September 1, 2026 15:39
…ing the cache get_or_create_env(CFG.env) returned whichever painting env an earlier test module had built - with the default 50 train tasks - so the assertion that reset_config's num_train_tasks=1 took effect failed whenever the test ran after another painting test in the same process. create_new_env honours the current config.
…oved The inspect_* / propose_* / retract_abstractions / generate_* / refine_plan_sketch / evaluate_predicate_on_trajectory tools were never offered to the learning arms and only survived as the fallback surface of arms that left agent_planner_use_explore_python at its False default. Every arm with a simulator now gets the same solve roster: evaluate_option_plan (+ evaluate_policy in policy mode), explore_python over the belief probe, and record_journal; a simulator-free planner gets none of them. The flag and its keep-replaced-tools companion are gone, along with the ProposalBundle plumbing (iteration_proposals, planning_results) that only the deleted tools ever filled, the proposed_code sandbox dir, the agent_option_learning approach that was built on the proposal tools, and the stale design doc describing them. The digest renderers the prompts and the probe share move from tools/inspection.py to tools/digests.py. The refine_plan_sketch tool tests now drive sim.refine on the probe (same parser and search core), including the evaluator-gated require_solved path. Claude-Session: https://claude.ai/code/session_01Pbv9TvedRD2iKyoKMJKx6H
The solve-phase explore_python and the synthesis-phase run_python were the same exec core under two names. Both sessions now offer run_python: the static instance binds the belief probe over the deployed model, a synthesis session attaches its own instance (fit data + the probe over the candidate simulator) under the same name and create_mcp_tools skips the static one when it is attached. The tool also accepts `path`, a .py file inside the sandbox executed in the same persistent namespace, so helpers and sweeps can be developed as files with Write/Edit instead of being pasted inline each call; a path that resolves outside the sandbox is refused (the PreToolUse hook does not see MCP calls). Settings and context fields follow the rename (agent_sdk_python_call_timeout, python_call_deadline). Claude-Session: https://claude.ai/code/session_01Pbv9TvedRD2iKyoKMJKx6H
evaluate_predicate_quality and evaluate_sampler were "reload the file, install the draft, report" tools; they are now sim.predicates() and sim.samplers() on the belief probe, so a synthesis session's only tool is run_python and every validation surface lives on `sim`. The loaders keep their behaviour (snapshot into *_versions/, validate, install into the approach so refinement sees the draft, milestone / sanity report) and are registered per session in ToolContext.probe_artifact_loaders; the probe raises a clear error where a session has no such file. The subclass hook that used to return extra tools becomes _install_extra_synthesis_surfaces. The converted evaluator-gate test now draws 200 samples so its certified and rejected paths are exercised at every probe seed. Claude-Session: https://claude.ai/code/session_01Pbv9TvedRD2iKyoKMJKx6H
_fit_params_after_synthesis used to refit from scratch after every learn session, silently replacing the values the agent's GO/NO-GO check had validated. sim.fit now publishes the full FitResult (point estimate plus the Laplace bundle the exploration ensemble is calibrated from), its SSE, and the physical values it applied; after the session the approach deploys exactly that fit when it ran on the final simulator.py content over the deployed parameter set. The harness fits itself only when no such fit exists (the session ended UNFITTED or out of turns, or an oracle sim program ran with no session) and logs a FIT FALLBACK warning saying the deployed values were never validated by the agent. The learn prompt states the contract. Claude-Session: https://claude.ai/code/session_01Pbv9TvedRD2iKyoKMJKx6H
record_journal is gone. The agent keeps ./journal.md with the ordinary file tools (agents were already bypassing the tool with Write), and the harness records each task's goal + initial state and every attempt's outcome and captured or best refused plan in its own ./attempts.md, which the agent is told not to edit. Solve prompts inject both (tail-capped), the facts-only guidance moved into the prompt, and the test-phase archive/rollback covers both files so an evaluation's notes never leak into the next one. Learn sessions can add to the journal the same way, so the learn roster is run_python alone. Claude-Session: https://claude.ai/code/session_01Pbv9TvedRD2iKyoKMJKx6H
…refinement The explorer used to run its own belief-model refinement on the returned sketch (search over unparameterized steps, info-seeking pooling, seeded fill of a failing tail) and derive the early-stop verdict from that pass, so there were two refinement surfaces and two verdict sources. Now the plan the agent hands back is grounded and executed exactly as written: every explicit parameter runs verbatim, a step left without parameters gets one uniform draw from the option's box and a warning, Wait steps keep their annotated targets, and the belief verdict is False unless the session's plan passed the capture gate (which still executes verbatim as a solve and is replayed for the cycle). The agent refines and validates in-session (sim.refine / sim.run / evaluate_option_plan); the explorer-only pinning and pool-size settings go with the harness pass. Claude-Session: https://claude.ai/code/session_01Pbv9TvedRD2iKyoKMJKx6H
…commit evaluate_option_plan conflated evaluation with submission: any goal-reaching run on the current task was captured as the answer, so the agent could not test a candidate there without committing to it, while task_idx and rollout_seed made the same tool a diagnostic probe. Evaluation now lives entirely on the probe (sim.run on any task, from any state, at any seed, with trials=N for the same gate as a preview) and submit_plan / submit_policy are the explicit commit: they run the plan or policy on the CURRENT task from its true initial state through the unchanged capture gate (validation repeats, physics and rule-param margins, evaluator legitimacy) and capture on pass. A FLAKY report now points at sim.run(plan, seed=S) to reproduce the failed rollout. Claude-Session: https://claude.ai/code/session_01Pbv9TvedRD2iKyoKMJKx6H
yichao-liang
force-pushed
the
bl-merge-10-agent-tools
branch
from
September 1, 2026 15:42
d073090 to
e81c5f7
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rewrites the agent tool surface: one solve surface for every arm with the legacy tool rosters removed, one run_python tool with a path argument, predicate and sampler drafts loading through the probe, evaluate_option_plan renamed to submit_plan as the one gated commit, the deployed model defined as the agent's published sim.fit, journal.md owned by the agent while the harness logs attempts.md, and the explorer executing the agent's sketch verbatim.
Commits:
Part of the bridge-learning merge stack.
🤖 Generated with Claude Code
https://claude.ai/code/session_01VQkk8ycwwwiSKraDw88FVm