diff --git a/agent_sdk_approach_overview.md b/agent_sdk_approach_overview.md deleted file mode 100644 index 357823e70..000000000 --- a/agent_sdk_approach_overview.md +++ /dev/null @@ -1,2397 +0,0 @@ -# Agent SDK Online Process Planning Approach - -## Overview - -A new approach that uses **Claude Agent SDK with Model Context Protocol (MCP)** to iteratively discover abstractions (predicates, processes, types, options) through interactive exploration rather than one-shot LLM prompting. - ---- - -## Key Innovation: Interactive Agent vs. Batch Prompting - -### Previous Approach (`OnlinePredicateInventionProcessPlanningApproach`) - -``` -Trajectory Data → Template Filling → LLM Prompt → Parse Response → Validate - ↓ - (all context provided upfront as string) -``` - -- **One-shot prompting**: Fills templates with full trajectory data, task info, types -- **Limited context window**: Must decide upfront what to include in prompt -- **No exploration**: Cannot test hypotheses before finalizing proposals -- **Rigid workflow**: Fixed sequence of prompting steps - -### New Approach (`AgentSDKOnlineProcessPlanningApproach`) - -``` -Agent ←→ MCP Tools ←→ ToolContext (trajectories, predicates, processes, tasks) - ↓ -Interactive: Query → Test → Propose → Validate -``` - -- **Multi-turn dialogue**: Agent can ask questions, inspect data selectively -- **On-demand access**: Only queries trajectory/task data it needs when needed -- **Interactive testing**: Can test predicates on states before proposing -- **Flexible exploration**: Agent decides its own discovery strategy - ---- - -## Architecture Comparison - -| Component | Old Approach | New Approach | -|-----------|--------------|--------------| -| **Input Method** | Template-based prompts | MCP tools (15 tools) | -| **Context Management** | All-at-once string dump | Selective on-demand queries | -| **Predicate Testing** | Parse & post-validate | Test before proposing via `test_predicate_on_states()` | -| **Process Learning** | Data-driven (learn from segments) | Agent-proposed (via tools) | -| **Code Execution** | Parse Python blocks from text | Structured execution context with safety | -| **Session Model** | Stateless per iteration | Persistent session across iterations | - ---- - -## MCP Tools Available to Agent (Detailed) - -The agent has access to 15 tools organized into three categories: - -### **Inspection Tools** (Read-Only) - Gather Information - -#### `inspect_types()` -**Returns:** List of all object types with their features and parent relationships -``` -Example output: -- robot[x, y, gripper_open]: parent=None -- block[x, y, z, on_table]: parent=object -- jug[x, y, water_amount, temperature]: parent=container -``` - -#### `inspect_predicates()` -**Returns:** All predicates with type signatures -``` -Example output: -- Holding(robot, block) -- OnTable(block) -- AtFaucet(jug) -- WaterBoiling(jug) -``` - -#### `inspect_processes(name: str)` -**Returns:** Detailed process conditions and effects -``` -Example output: -- FillJug - Conditions: {AtFaucet(?jug), GripperOpen(?robot)} - Add effects: {JugFilled(?jug)} - Delete effects: {} - Delay: ConstantDelay(5) -``` - -#### `inspect_options()` -**Returns:** Available parameterized actions -``` -Example output: -- Pick(robot, block), params_dim=3 -- Place(robot, block, location), params_dim=5 -- MoveToFaucet(robot, jug), params_dim=2 -``` - -#### `inspect_trajectories(traj_idx: int, include_states: bool = True, include_atoms: bool = False, max_timesteps: int = 10)` -**Most powerful tool** - Agent can selectively query trajectories without loading all data -```python -# Agent can request: -inspect_trajectories(traj_idx=3, max_timesteps=5, include_atoms=True) - -# Returns: -""" -Trajectory 3: 15 states, 14 actions - ---- Timestep 0 --- -State: { - "block1": {"x": 0.5, "y": 0.2, "on_table": 1.0}, - "gripper": {"x": 0.1, "y": 0.1, "open": 1.0} -} -Atoms: {OnTable(block1), GripperEmpty()} -Action: Pick(gripper, block1) - ---- Timestep 1 --- -State: { - "block1": {"x": 0.1, "y": 0.1, "on_table": 0.0}, - "gripper": {"x": 0.1, "y": 0.1, "open": 0.0} -} -Atoms: {Holding(gripper, block1)} -... -""" -``` - -#### `inspect_train_tasks(task_idx: Optional[int] = None)` -**Returns:** Task goals and initial conditions. If `task_idx` omitted, returns summary of all tasks -``` -Example output (specific task): -Task 5: - Goal: {OnTable(block2), Holding(gripper, block3)} - Initial atoms: {OnTable(block1), OnTable(block2), OnTable(block3), GripperEmpty()} - Objects: [gripper:robot, block1:block, block2:block, block3:block] -``` - -#### `inspect_planning_results()` -**Returns:** JSON of planning metrics from last test run -```json -{ - "success_rate": 0.67, - "avg_nodes_expanded": 245.3, - "avg_plan_length": 8.2, - "failure_summaries": "Task 2: goal not reachable. Task 5: timeout after 30s" -} -``` - -#### `inspect_past_proposals()` -**Returns:** Summary of all past iterations (what was proposed, what worked) - -### **Proposal Tools** (Write Access) - Submit Code - -These tools accept Python code and execute it safely. Each has specific requirements: - -#### `propose_types(code: str, description: str)` -**Required:** Code must define `proposed_types` as a list of `Type` objects - -```python -# Example agent call: -propose_types( - code=""" -proposed_types = [ - Type("grid_cell", ["row", "col", "occupancy"]), - Type("reference_frame", ["origin_x", "origin_y", "angle"]) -] -""", - description="Helper types for spatial reasoning" -) -``` - -**Validation:** Checks that each item is a `Type` instance - -#### `propose_predicates(code: str, description: str)` -**Required:** Code must define `proposed_predicates` as a list of `Predicate` objects - -```python -# Example agent call: -propose_predicates( - code=""" -proposed_predicates = [ - Predicate( - "InGripper", - [_block_type, _robot_type], - lambda s, objs: ( - abs(s.get(objs[0], "x") - s.get(objs[1], "x")) < 0.1 and - abs(s.get(objs[0], "y") - s.get(objs[1], "y")) < 0.1 and - s.get(objs[1], "gripper_open") < 0.5 - ) - ), - Predicate( - "OnTable", - [_block_type], - lambda s, objs: s.get(objs[0], "on_table") > 0.5 - ) -] -""", - description="Predicates for block manipulation" -) -``` - -**Validation:** -1. Executes code in safe context with current types/predicates available -2. Verifies each predicate's types reference valid types -3. Tests each predicate on `example_state` (from first trajectory) -4. Returns clear error messages if validation fails - -**Agent sees errors immediately:** -``` -Validation errors (2): -- InGripper: Predicate references unknown type 'gripper_type'. Did you mean '_robot_type'? -- OnTable: Predicate failed evaluation on example state: KeyError: 'on_table' -``` - -#### `propose_processes(code: str, description: str)` -**Required:** Code must define `proposed_processes` as a list of `CausalProcess` objects - -```python -# Example agent call: -propose_processes( - code=""" -v_jug = Variable("?jug", _jug_type) -v_robot = Variable("?robot", _robot_type) - -proposed_processes = [ - ExogenousProcess( - name="FillJug", - parameters=[v_jug, v_robot], - condition_at_start={ - LiftedAtom(AtFaucet, [v_jug]), - LiftedAtom(GripperHolding, [v_robot, v_jug]) - }, - condition_overall={ - LiftedAtom(AtFaucet, [v_jug]) - }, - condition_at_end=set(), - add_effects={ - LiftedAtom(JugFilled, [v_jug]) - }, - delete_effects={ - LiftedAtom(JugEmpty, [v_jug]) - }, - delay_distribution=DiscreteGaussianDelay(mean=5, variance=1), - strength=torch.tensor([1.0]) - ) -] -""", - description="Exogenous process for jug filling - takes time at faucet" -) -``` - -**Key distinction from old approach:** Agent directly proposes process structure; OLD approach segments trajectories and induces processes from patterns - -#### `propose_object_augmentor(code: str, description: str)` -**Required:** Code must define `augment_task(task) -> Task` function - -```python -# Example: Add grid cell helper objects -propose_object_augmentor( - code=""" -def augment_task(task: Task) -> Task: - # Add grid cells to simplify spatial reasoning - grid_cells = [] - for row in range(5): - for col in range(5): - cell = Object(f"cell_{row}_{col}", _grid_cell_type) - grid_cells.append(cell) - - # Create new initial state with grid cells - augmented_init = task.init.copy() - for cell in grid_cells: - augmented_init.set(cell, "row", float(row)) - augmented_init.set(cell, "col", float(col)) - augmented_init.set(cell, "occupancy", 0.0) - - return Task(augmented_init, task.goal) -""", - description="Add discretized grid for spatial reasoning" -) -``` - -**This is powerful:** Agent can add helper objects that aren't in the environment - -#### `propose_options(code: str, description: str)` -**Required:** Code must define `proposed_options` as list of `ParameterizedOption` objects -(Currently less used since options are typically provided) - -### **Testing Tools** - Validate Hypotheses - -#### `test_predicate_on_states(predicate_name: str, traj_idx: int, object_names: List[str])` -**Critical for iterative refinement** - Agent can test before proposing - -```python -# Agent workflow: -# 1. Inspect trajectory -inspect_trajectories(traj_idx=0, max_timesteps=5) - -# 2. Form hypothesis about "InGripper" predicate -# 3. Test it (even before officially proposing!) -test_predicate_on_states( - predicate_name="InGripper", - traj_idx=0, - object_names=["block1", "gripper"] -) - -# Returns: -""" -Predicate InGripper(block1, gripper) over trajectory 0: -t=0: False -t=1: False -t=2: True # After Pick action -t=3: True -t=4: False # After Place action -""" - -# 4. If looks good, officially propose it -propose_predicates(code="...", description="...") -``` - -**This prevents wasted proposals** - agent can debug before committing - -#### `test_planning(task_idx: int, timeout: int = 30)` -**Runs actual task planner** with current abstractions - -```python -# Agent can test if new predicates help planning -test_planning(task_idx=2) - -# Returns: -""" -Planning succeeded for task 2! -Plan length: 6 -Nodes expanded: 124 -Plan: Pick(gripper, block1) -> Move(gripper, loc2) -> Place(gripper, block1) -> ... -""" - -# Or on failure: -""" -Planning failed for task 2. -Reason: ApproachTimeout: Exceeded 30s timeout -""" -``` - -**Agent uses this to validate proposals help planning before finalizing** - ---- - -## Example: Agent Workflow - -### Old Approach Workflow -```python -# Cycle 0: -1. Fill template with ALL offline trajectories (token-heavy) -2. Prompt LLM: "Here are 50 states, propose predicates" -3. Get back code blocks, parse, validate -4. Learn processes from data (data-driven) -5. Done - wait for next cycle -``` - -### New Approach Workflow -```python -# Cycle 0: -Agent: inspect_train_tasks() - → "Task 0 has goal: {OnTable(block1)}, Task 1 has goal: {Holding(block2)}" - -Agent: inspect_trajectories(traj_idx=0, max_timesteps=3, include_atoms=True) - → Gets first 3 timesteps only - -Agent: "I notice blocks transition from table to gripper. Let me test a hypothesis..." -Agent: test_predicate_on_states("InGripper", traj_idx=0, object_names=["block1"]) - → [t=0: False, t=1: False, t=2: True, ...] - -Agent: "Good! Now let me propose this predicate with confidence..." -Agent: propose_predicates(code="...", description="Tracks gripper contents") - → Predicate validated and integrated - -Agent: propose_processes(code="...", description="Pick process") - → Process integrated directly (no data-driven learning needed) - -# Agent can iterate within ONE cycle! -``` - ---- - -## Key Differences Summary - -### What's Better in New Approach - -✅ **Token efficiency**: Queries only needed data vs. dumping everything -✅ **Interactive refinement**: Test hypotheses before proposing -✅ **Flexible exploration**: Agent decides strategy, not hardcoded templates -✅ **Direct process proposals**: Skip data-driven learning loop -✅ **Broader abstractions**: Can propose types, task augmentors, options (not just predicates) -✅ **Persistent context**: Agent remembers across iterations -✅ **Built-in validation**: Tools validate proposals before integration -✅ **Planning integration**: Can test if predicates help planning before committing - -### What's Preserved - -✔️ Multi-cycle online learning -✔️ ProcessPlanning base (still does bilevel planning) -✔️ Interaction with environment (still collects trajectories) -✔️ Save/load functionality - ---- - -## Technical Details (Deep Dive) - -### Process Learning: Data-Driven vs. Agent-Proposed - -#### Old Approach: Data-Driven Process Induction -```python -def _select_predicates_and_learn_processes(self, all_trajs, proposed_predicates): - # 1. Score predicates based on planning utility - candidates = self._score_predicates(proposed_predicates, all_trajs) - - # 2. Select best predicates via greedy search - selected = self._select_predicates_by_score_optimization(candidates, ...) - - # 3. Segment trajectories using selected predicates - segmented_trajs = [ - segment_trajectory(traj, selected_predicates) - for traj in all_trajs - ] - - # 4. INDUCE processes from segment patterns - for segment in segmented_trajs: - if segment has unexplained transition: - # Extract conditions from segment.init_atoms - conditions = segment.init_atoms - # Extract effects from segment.final_atoms - segment.init_atoms - add_effects = segment.final_atoms - segment.init_atoms - delete_effects = segment.init_atoms - segment.final_atoms - # Create process - proc = ExogenousProcess(name=f"process_{id}", ...) - - # 5. Filter processes by coverage/precision metrics - self._processes = filter_high_quality_processes(induced_processes) -``` - -**Problems:** -- Segmentation can miss processes if predicates are incomplete -- Process induction is heuristic-based, may miss structure -- Requires many trajectories to see process patterns -- Can't propose novel process structures (limited to what segmentation finds) - -#### New Approach: Agent Directly Proposes -```python -def _learn_processes(self, *args, **kwargs): - """Override parent's data-driven process learning.""" - # No-op! Agent proposes processes directly via MCP tools - if not hasattr(self, '_proc_name_to_results'): - self._proc_name_to_results = {} - logging.debug("Skipping data-driven process learning - agent proposes directly") - -# Agent uses propose_processes tool instead: -# Agent sees trajectories, reasons about causality, proposes structured processes -``` - -**Benefits:** -- Agent can propose processes with limited data (uses reasoning) -- Can propose novel structures (e.g., conditional delays, complex conditions) -- Uses `test_planning` to validate process helps before committing -- Faster: skip expensive segmentation/induction - -### Predicate Validation: Post-Hoc vs. Pre-Validation - -#### Old Approach: Parse and Hope -```python -def _get_predicate_proposals_from_fm(self, proposal_method, trajectories): - # 1. Fill prompt template with trajectory data - prompt = template.format( - STRUCT_DEFINITION=..., - TYPES_IN_ENV=_get_types_str(types), - LISTED_STATES=state_str, # Could be 1000s of lines - PREDICATE_SPECS=spec_response - ) - - # 2. Get LLM response (one-shot) - impl_response = self._llm.sample_completions(prompt, temperature=0)[0] - - # 3. Parse Python code blocks using regex - pattern = re.compile(r'```python(.*?)```', re.DOTALL) - python_blocks = list(pattern.finditer(impl_response)) - - # 4. Try to exec each block - primitive_preds = set() - for code_str in python_blocks: - exec(code_str, context) # May fail! - pred_name = extract_name_from_code(code_str) - if pred_name in context: - primitive_preds.add(context[pred_name]) - - # 5. Post-validation (after all proposals made) - return primitive_preds # Some may be broken! -``` - -**Problems:** -- LLM has one shot, can't iterate -- Errors discovered late (after parsing) -- No structured error messages back to LLM -- Broken predicates discarded silently - -#### New Approach: Validate in Tool, Immediate Feedback -```python -@tool("propose_predicates") -async def propose_predicates(args: Dict[str, Any]) -> Dict[str, Any]: - code = args["code"] - - # 1. Build safe execution context with current types/predicates - exec_ctx = build_exec_context(ctx.types, ctx.predicates, ctx.options) - # exec_ctx includes: _block_type, _robot_type, Holding, OnTable, etc. - - # 2. Execute code safely - result, error = exec_code_safely(code, exec_ctx, "proposed_predicates") - if error: - return _error_result(f"Code execution failed:\n{error}") - - # 3. Type check result - if not isinstance(result, (list, set)): - return _error_result( - f"proposed_predicates must be list/set, got {type(result)}") - - # 4. Validate EACH predicate before accepting ANY - validated = [] - errors = [] - for pred in result: - if not isinstance(pred, Predicate): - errors.append(f"Not a Predicate: {type(pred)}") - continue - - # Check types reference valid types - for t in pred.types: - if t not in ctx.types: - errors.append(f"{pred.name}: references unknown type {t.name}") - continue - - # Test on example state - if ctx.example_state: - err = validate_predicate(pred, ctx.types, ctx.example_state) - if err: - errors.append(f"{pred.name}: {err}") - continue - - validated.append(pred) # Only add if passed all checks - - # 5. Update context with validated predicates - proposed = set(validated) - ctx.iteration_proposals.proposed_predicates |= proposed - - # 6. Return structured feedback - msg = f"Successfully proposed {len(proposed)} predicates: {[p.name for p in proposed]}" - if errors: - msg += f"\n\nValidation errors ({len(errors)}):\n" + "\n".join(errors) - return _text_result(msg) -``` - -**Agent sees and can respond to errors:** -``` -Agent: Let me propose predicates... -Tool: Validation errors (1): - - InGripper: KeyError: 'gripper_open' not found. Available features: ['x', 'y', 'open'] - -Agent: Ah, I used wrong feature name! Let me fix and repropose... -``` - -### Proposals Integration and State Management - -#### ProposalBundle Structure -```python -@dataclass -class ProposalBundle: - """Accumulates all proposals during ONE iteration""" - proposed_types: Set[Type] = field(default_factory=set) - proposed_predicates: Set[Predicate] = field(default_factory=set) - augment_task_fn: Optional[Callable[[Task], Task]] = None - augment_task_code: Optional[str] = None # For serialization - proposed_processes: Set[CausalProcess] = field(default_factory=set) - proposed_options: Set[ParameterizedOption] = field(default_factory=set) - errors: List[str] = field(default_factory=list) -``` - -#### Integration Flow -```python -def learn_from_interaction_results(self, results): - # 1. Add new trajectories - for result in results: - self._online_dataset.append(trajectory_from_result(result)) - - # 2. Sync ToolContext with current state - self._sync_tool_context(all_trajs) - # This makes latest trajectories available to agent via tools - - # 3. Reset proposal bundle for this iteration - self._tool_context.iteration_proposals = ProposalBundle() - - # 4. Run agent (agent calls tools, builds up proposals) - self._run_agent_iteration(all_trajs) - - # 5. Extract proposals made during agent run - proposals = self._tool_context.iteration_proposals - - # 6. Integrate validated proposals into approach state - self._integrate_proposals(proposals) - - # 7. Learn parameters for agent-proposed processes (optional) - if CFG.learn_process_parameters: - self._learn_process_parameters(all_trajs) - - # 8. Save everything - self.save(self._online_learning_cycle) - self._online_learning_cycle += 1 - -def _integrate_proposals(self, proposals: ProposalBundle): - """Actually update approach state with validated proposals""" - - # Types: Add to type set and track as helper types - if proposals.proposed_types: - self._types |= proposals.proposed_types - self._helper_types |= proposals.proposed_types # Track for save/load - logging.info(f"Integrated {len(proposals.proposed_types)} new types: " - f"{[t.name for t in proposals.proposed_types]}") - - # Predicates: Add to learned predicates - if proposals.proposed_predicates: - self._learned_predicates |= proposals.proposed_predicates - logging.info(f"Integrated {len(proposals.proposed_predicates)} predicates: " - f"{[p.name for p in proposals.proposed_predicates]}") - - # Task augmentor: Store function AND code (for save/load) - if proposals.augment_task_fn: - self._augment_task_fn = proposals.augment_task_fn - self._augment_task_code = proposals.augment_task_code - logging.info("Integrated task augmentor") - - # Processes: Store as agent-proposed (NOT data-learned) - if proposals.proposed_processes: - self._agent_proposed_processes |= proposals.proposed_processes - self._processes = set(self._agent_proposed_processes) - logging.info(f"Integrated {len(proposals.proposed_processes)} processes") - - # Options: Add to available options - if proposals.proposed_options: - self._agent_proposed_options |= proposals.proposed_options - logging.info(f"Integrated {len(proposals.proposed_options)} options") -``` - -#### Key Tracking Distinctions -```python -# Old approach: -self._learned_predicates # From LLM proposals -self._processes # From data-driven induction - -# New approach: -self._learned_predicates # From agent proposals (via tools) -self._processes # From agent proposals (via tools) -self._agent_proposed_processes # Explicitly track as agent-proposed -self._helper_types # Types not in environment (agent-created) -self._augment_task_fn # Runtime task modification function -self._augment_task_code # Code string (for serialization) -self._agent_proposed_options # Additional options from agent -``` - -### Execution Context and Safety - -#### Building Safe Exec Context -```python -def build_exec_context(types, predicates, options): - """Create namespace for exec() with controlled imports""" - context = {} - - # 1. Safe imports only - import numpy as np - import torch - context["np"] = np - context["torch"] = torch - context["Box"] = Box # from gym.spaces - - # 2. Predicate/process/type classes - from predicators.structs import Type, Predicate, ExogenousProcess, ... - context["Type"] = Type - context["Predicate"] = Predicate - context["ExogenousProcess"] = ExogenousProcess - # ... etc - - # 3. Current types (namespaced to avoid collision) - for t in types: - context[f"_{t.name}_type"] = t - # Agent uses: _block_type, _robot_type, etc. - - # 4. Current predicates (by name) - for p in predicates: - context[p.name] = p # Agent can reference Holding, OnTable, etc. - context[f"_{p.name}_holds"] = p._classifier # Access classifier - - # 5. Current options (by name) - for o in options: - context[o.name] = o - - # 6. NO access to: - # - Environment internals (env._physics, env.simulate, etc.) - # - File system operations - # - Network operations - # - Arbitrary imports - - return context -``` - -**Security:** Code executes in restricted namespace, can't import dangerous modules or access environment internals - -#### Safe Execution with Error Handling -```python -def exec_code_safely(code: str, context: Dict, expected_var: str): - """Execute and extract expected variable""" - try: - exec(code, context) - except Exception: - # Return full traceback to agent - return None, traceback.format_exc() - - # Check expected variable exists - if expected_var not in context: - return None, (f"Code executed but didn't define '{expected_var}'. " - f"Available: {[k for k in context if not k.startswith('_')]}") - - return context[expected_var], None # Success -``` - -**Agent gets detailed error messages:** -``` -NameError: name '_gripper_type' is not defined. Did you mean '_robot_type'? - -Available types: _block_type, _robot_type, _jug_type, _table_type -``` - ---- - -## Example Use Case: Boiling Water Domain (Detailed Trace) - -### Iteration 0 - Agent Discovers Structure - -**Context Message to Agent:** -``` -== Online Learning Iteration 0 == - -TRAJECTORY SUMMARY: -- 3 new trajectories collected this cycle -- 3 total trajectories (offline + online) -- Task success rate: 0.0% [No abstractions yet!] - -CURRENT ABSTRACTIONS: -- Types: jug[x, y, z, water_amount, temperature], burner[x, y, z, heat_level], - faucet[x, y, z, flow_rate], robot[x, y, z, gripper_open] -- Predicates (0): [None yet - this is the first iteration] -- Processes (0): [None yet] -- Options (4): MoveToFaucet, MoveToPosition, GraspJug, ReleaseJug - -PLANNING PERFORMANCE: -- Success: Not yet evaluated [Need predicates first] -- Failures: Cannot run planner without predicates - -Your task: Inspect the trajectory data and propose predicates that capture the -important states and relationships. Focus on what conditions enable actions and -what effects those actions have. -``` - -**Agent's Exploration (Actual Trace):** - -``` -[Agent reasoning] -"I need to understand the domain first. Let me inspect the training tasks to see -what goals we're trying to achieve." - -[Tool call 1] -inspect_train_tasks() - -[Tool result] -Total tasks: 3 - Task 0: goal={WaterBoiling(jug0)} - Task 1: goal={WaterBoiling(jug1), OnBurner(jug2)} - Task 2: goal={JugFilled(jug0), JugFilled(jug1)} - -[Agent reasoning] -"Interesting - goals reference predicates WaterBoiling, OnBurner, JugFilled that -don't exist yet! These are the target predicates. But I need to understand the -domain better. Let me look at a trajectory." - -[Tool call 2] -inspect_trajectories(traj_idx=0, max_timesteps=8, include_states=True) - -[Tool result] -Trajectory 0: 25 states, 24 actions - ---- Timestep 0 --- -State: { - "jug0": {"x": 1.2, "y": 0.5, "z": 0.0, "water_amount": 0.0, "temperature": 20.0}, - "faucet0": {"x": 0.5, "y": 0.5, "z": 0.3, "flow_rate": 0.0}, - "burner0": {"x": 1.5, "y": 1.0, "z": 0.0, "heat_level": 0.0}, - "robot0": {"x": 1.2, "y": 0.5, "z": 0.0, "gripper_open": 1.0} -} -Action: MoveToFaucet(robot0, jug0) - ---- Timestep 1 --- -State: { - "jug0": {"x": 0.5, "y": 0.5, "z": 0.0, "water_amount": 0.0, "temperature": 20.0}, - "faucet0": {"x": 0.5, "y": 0.5, "z": 0.3, "flow_rate": 0.0}, - ... -} -Action: GraspJug(robot0, jug0) - ---- Timestep 2 --- -State: { - "jug0": {"x": 0.5, "y": 0.5, "z": 0.0, "water_amount": 0.0, "temperature": 20.0}, - "robot0": {"x": 0.5, "y": 0.5, "z": 0.0, "gripper_open": 0.0}, # Grasped! - ... -} -[No action - waiting] - ---- Timestep 3-7 --- -State: { - "jug0": {"water_amount": 0.2, ...}, # t=3 - "jug0": {"water_amount": 0.4, ...}, # t=4 - "jug0": {"water_amount": 0.6, ...}, # t=5 - "jug0": {"water_amount": 0.8, ...}, # t=6 - "jug0": {"water_amount": 1.0, ...}, # t=7 - FILLED! - "faucet0": {"flow_rate": 0.2, ...} -} -[No actions - filling happens passively] - ---- Timestep 8 --- -Action: MoveToPosition(robot0, jug0, 1.5, 1.0, 0.0) # Move to burner -... - -[Agent reasoning] -"Aha! A few key observations: -1. At t=0-1: Jug moves to faucet location (x,y ~= 0.5, 0.5) -2. At t=2: Gripper closes (gripper_open: 1.0 -> 0.0) -3. At t=2-7: Water fills PASSIVELY (no actions!) - this is an exogenous process! -4. After filling, jug moves to burner - -I should propose: -- Predicates: AtFaucet, Holding, JugFilled, OnBurner -- Process: FillJug (exogenous, happens when jug held at faucet) - -But first, let me test if my intuition about 'AtFaucet' is correct." - -[Tool call 3] -test_predicate_on_states( - predicate_name="AtFaucet", # Doesn't exist yet, but I can test my idea - traj_idx=0, - object_names=["jug0", "faucet0"] -) - -[Tool result - ERROR] -Predicate 'AtFaucet' not found. - -[Agent reasoning] -"Right, it doesn't exist yet. I need to propose it first. Let me define my -predicates carefully." - -[Tool call 4] -propose_predicates( - code=""" -# Jug is at faucet location (within distance threshold) -def _at_faucet_holds(state, objects): - jug, faucet = objects - dist = ((state.get(jug, "x") - state.get(faucet, "x"))**2 + - (state.get(jug, "y") - state.get(faucet, "y"))**2)**0.5 - return dist < 0.1 - -# Robot is holding jug (gripper closed and positions match) -def _holding_holds(state, objects): - robot, jug = objects - pos_match = (abs(state.get(robot, "x") - state.get(jug, "x")) < 0.05 and - abs(state.get(robot, "y") - state.get(jug, "y")) < 0.05) - gripper_closed = state.get(robot, "gripper_open") < 0.5 - return pos_match and gripper_closed - -# Jug has water -def _jug_filled_holds(state, objects): - jug = objects[0] - return state.get(jug, "water_amount") > 0.9 - -# Jug is on burner -def _on_burner_holds(state, objects): - jug, burner = objects - dist = ((state.get(jug, "x") - state.get(burner, "x"))**2 + - (state.get(jug, "y") - state.get(burner, "y"))**2)**0.5 - return dist < 0.1 - -# Water is boiling (high temperature) -def _water_boiling_holds(state, objects): - jug = objects[0] - return (state.get(jug, "temperature") > 95.0 and - state.get(jug, "water_amount") > 0.1) - -proposed_predicates = [ - Predicate("AtFaucet", [_jug_type, _faucet_type], _at_faucet_holds), - Predicate("Holding", [_robot_type, _jug_type], _holding_holds), - Predicate("JugFilled", [_jug_type], _jug_filled_holds), - Predicate("OnBurner", [_jug_type, _burner_type], _on_burner_holds), - Predicate("WaterBoiling", [_jug_type], _water_boiling_holds), -] -""", - description="Core predicates for jug manipulation and water states" -) - -[Tool result] -Successfully proposed 5 predicates: ['AtFaucet', 'Holding', 'JugFilled', -'OnBurner', 'WaterBoiling'] - -[Agent reasoning] -"Great! Now let me test these on the trajectory to verify they work correctly." - -[Tool call 5] -test_predicate_on_states( - predicate_name="AtFaucet", - traj_idx=0, - object_names=["jug0", "faucet0"] -) - -[Tool result] -Predicate AtFaucet(jug0, faucet0) over trajectory 0: -t=0: False [Jug not at faucet yet] -t=1: True [After MoveToFaucet action] -t=2: True -t=3: True -... -t=7: True [Still at faucet during filling] -t=8: False [Moved to burner] -... - -[Agent reasoning] -"Perfect! AtFaucet activates after the move and stays true during filling. -Now let me test Holding:" - -[Tool call 6] -test_predicate_on_states(predicate_name="Holding", traj_idx=0, - object_names=["robot0", "jug0"]) - -[Tool result] -Predicate Holding(robot0, jug0) over trajectory 0: -t=0: False -t=1: False [Moved to faucet but not grasped yet] -t=2: True [After GraspJug] -t=3-7: True [Held during filling] -t=8-15: True [Held during move to burner] -t=16: False [After ReleaseJug] - -[Agent reasoning] -"Excellent! Now I see the pattern: -- MoveToFaucet → AtFaucet becomes true -- GraspJug → Holding becomes true -- THEN water fills passively (exogenous process) -- Then move to burner, release jug - -I should propose the FillJug exogenous process now. It should have: -- Condition: AtFaucet AND Holding -- Effect: JugFilled becomes true -- Delay: Takes ~5 timesteps" - -[Tool call 7] -propose_processes( - code=""" -# Variables for process parameters -v_robot = Variable("?robot", _robot_type) -v_jug = Variable("?jug", _jug_type) -v_faucet = Variable("?faucet", _faucet_type) - -proposed_processes = [ - ExogenousProcess( - name="FillJug", - parameters=[v_robot, v_jug, v_faucet], - - # Conditions must hold at START to trigger process - condition_at_start={ - LiftedAtom(AtFaucet, [v_jug, v_faucet]), - LiftedAtom(Holding, [v_robot, v_jug]) - }, - - # Conditions must hold THROUGHOUT process - condition_overall={ - LiftedAtom(AtFaucet, [v_jug, v_faucet]), - LiftedAtom(Holding, [v_robot, v_jug]) - }, - - # No end conditions (can finish any time) - condition_at_end=set(), - - # Effects when process completes - add_effects={ - LiftedAtom(JugFilled, [v_jug]) - }, - delete_effects=set(), - - # Takes 5 timesteps with some variance - delay_distribution=DiscreteGaussianDelay(mean=5, variance=1), - - # Deterministic process - strength=torch.tensor([1.0]) - ) -] -""", - description="Exogenous process for filling jug at faucet" -) - -[Tool result] -Successfully proposed 1 processes: ['FillJug'] - -[Agent reasoning] -"Great! Now I have predicates and processes. Let me test if these abstractions -enable successful planning on one of the tasks." - -[Tool call 8] -test_planning(task_idx=0, timeout=30) - -[Tool result] -Planning succeeded for task 0! -Plan length: 4 -Nodes expanded: 47 -Plan: - MoveToFaucet(robot0, jug0, faucet0) - GraspJug(robot0, jug0) - [WAIT for FillJug process] - MoveToPosition(robot0, jug0, burner0) - [WAIT for HeatWater process - NOT YET DEFINED!] - -[Agent reasoning] -"Planning works! But I see it expects a 'HeatWater' process that I haven't -defined yet. Looking at task 0's goal {WaterBoiling(jug0)}, I need to model -the heating process too. Let me check a trajectory that accomplishes boiling." - -[Tool call 9] -inspect_trajectories(traj_idx=0, max_timesteps=25, include_atoms=False) - -[Sees timesteps 15-20 where jug on burner and temperature rises] - -[Tool call 10] -propose_processes( - code=""" -v_jug = Variable("?jug", _jug_type) -v_burner = Variable("?burner", _burner_type) - -proposed_processes = [ - ExogenousProcess( - name="HeatWater", - parameters=[v_jug, v_burner], - condition_at_start={ - LiftedAtom(OnBurner, [v_jug, v_burner]), - LiftedAtom(JugFilled, [v_jug]) - }, - condition_overall={ - LiftedAtom(OnBurner, [v_jug, v_burner]), - LiftedAtom(JugFilled, [v_jug]) - }, - condition_at_end=set(), - add_effects={ - LiftedAtom(WaterBoiling, [v_jug]) - }, - delete_effects=set(), - delay_distribution=DiscreteGaussianDelay(mean=8, variance=2), - strength=torch.tensor([1.0]) - ) -] -""", - description="Exogenous process for heating water on burner" -) - -[Tool result] -Successfully proposed 1 processes: ['HeatWater'] - -[Agent reasoning] -"Perfect! Now let me re-test planning with both processes:" - -[Tool call 11] -test_planning(task_idx=0, timeout=30) - -[Tool result] -Planning succeeded for task 0! -Plan length: 4 -Nodes expanded: 52 -Plan: MoveToFaucet → GraspJug → [FillJug] → MoveToPosition → PlaceJug → [HeatWater] - -[Agent final message] -"I've successfully proposed abstractions for this domain: - -Predicates (5): AtFaucet, Holding, JugFilled, OnBurner, WaterBoiling -Processes (2): FillJug, HeatWater - -These enable planning on task 0. The key insight was recognizing the two -exogenous processes (filling and heating) that happen passively when certain -conditions are met. Testing confirmed these work correctly." -``` - -**Iteration 0 Results:** -- **Proposals integrated:** 5 predicates, 2 processes -- **Planning test:** 1/3 tasks solved (need to test others) -- **Agent turns:** 11 tool calls -- **Cost:** ~$0.15 (Claude Sonnet 4) - -**Compare to old approach:** -``` -Old approach Iteration 0: -1. Dump all 3 trajectories into prompt (5000 tokens) -2. LLM proposes predicates (no testing) -3. Parse code blocks → 3/5 predicates fail validation -4. Data-driven process learning → induces 0 processes (not enough data) -5. Result: 2 working predicates, 0 processes, cannot plan -``` - -### Iteration 1 - Agent Refines Based on Failures (Detailed) - -**New Data:** 5 additional trajectories collected from online interaction - -**Context Message:** -``` -== Online Learning Iteration 1 == - -TRAJECTORY SUMMARY: -- 5 new trajectories collected this cycle -- 8 total trajectories (3 offline + 5 online) -- Task success rate: 60% (3/5 tasks solved) - -CURRENT ABSTRACTIONS: -- Types: jug[x,y,z,water_amount,temperature], robot[x,y,z,gripper_open], ... -- Predicates (5): AtFaucet(jug,faucet), Holding(robot,jug), JugFilled(jug), - OnBurner(jug,burner), WaterBoiling(jug) -- Processes (2): FillJug, HeatWater -- Options (4): MoveToFaucet, MoveToPosition, GraspJug, ReleaseJug - -PLANNING PERFORMANCE: -- Success: 2/3 tasks planned successfully -- Avg nodes expanded: 156.3 -- Failures: Task 2 failed with "Process FillJug activated but not completed - - jug moved away from faucet at timestep 5" - -PREVIOUS ITERATION OUTCOMES: -{ - "cycle": 0, - "proposed_predicates": ["AtFaucet", "Holding", "JugFilled", "OnBurner", "WaterBoiling"], - "proposed_processes": ["FillJug", "HeatWater"], - "total_predicates": 5, - "total_processes": 2 -} -``` - -**Agent's Analysis:** - -``` -[Agent reasoning] -"The failure message says 'Process FillJug activated but not completed - jug -moved away from faucet'. This suggests the planner initiated the FillJug process -but then moved the jug before it finished. - -The issue is my FillJug process only checks conditions at START and OVERALL, but -the 'condition_overall' isn't strong enough - it needs to be checked continuously. -Actually, looking at my process definition, I have condition_overall but the -planner might not be enforcing it strictly enough. - -Let me inspect the failed trajectory to understand what happened." - -[Tool call 1] -inspect_planning_results() - -[Tool result] -{ - "success_rate": 0.67, - "avg_nodes_expanded": 156.3, - "failure_summaries": "Task 2: Process FillJug(robot0, jug1, faucet0) started - at t=3 but at t=5 condition_overall {AtFaucet(jug1, faucet0)} became false. - Process was aborted incompletely." -} - -[Agent reasoning] -"Ah! The planner IS checking condition_overall, but it's ABORTING the process when -the condition fails, rather than preventing the action that would break it. The -problem is the planner doesn't know that 'MoveToPosition' would break the AtFaucet -condition. - -Actually, this might be a predicate problem. Let me check if there's a better way -to model this. Or maybe I need an additional predicate that prevents movement during -filling? - -Wait - let me inspect the actual failed trajectory to see what actions were attempted." - -[Tool call 2] -inspect_trajectories(traj_idx=5, max_timesteps=10, include_atoms=True) - -[Tool result] -Trajectory 5: 12 states, 11 actions, _train_task_idx=2 - ---- Timestep 0 --- -Atoms: {AtFaucet(jug1, faucet0): False, Holding(robot0, jug1): False, ...} -Action: MoveToFaucet(robot0, jug1, faucet0) - ---- Timestep 1-2 --- -Atoms: {AtFaucet(jug1, faucet0): True, Holding(robot0, jug1): False} -Action: GraspJug(robot0, jug1) -Atoms: {AtFaucet(jug1, faucet0): True, Holding(robot0, jug1): True} -[FillJug process starts here] - ---- Timestep 3-4 --- -[Filling in progress, water_amount increasing] -Action: MoveToPosition(robot0, jug1, 1.5, 1.0, 0.0) # <-- PREMATURE MOVE! -[jug1.water_amount = 0.4, not yet filled] - ---- Timestep 5 --- -Atoms: {AtFaucet(jug1, faucet0): False, Holding(robot0, jug1): True, - JugFilled(jug1): False} -[FillJug process aborted] - -[Agent reasoning] -"I see the problem now! The planner generated a plan that included moving away -from the faucet TOO EARLY (before FillJug completed). The issue is that my -condition_overall is correct, BUT the planner doesn't model the temporal duration -of processes well enough - it thought it could move immediately. - -Hmm, actually this might be a planning algorithm issue, not an abstraction issue. -But let me think if there's a better way to model this... - -Option 1: Add a 'FillingInProgress' predicate that becomes true when FillJug starts -Option 2: Model this as an endogenous process (action-triggered) instead -Option 3: Add preconditions to MoveToPosition that prevent moving during filling - -Actually, the real issue might be my delay distribution. Let me check if the planner -knows FillJug takes 5 timesteps. Let me test planning with more debug info." - -[Tool call 3] -test_planning(task_idx=2, timeout=30) - -[Tool result] -Planning failed for task 2. -Reason: ApproachFailure: Generated plan with FillJug process but process requires -5 timesteps and planner scheduled MoveToPosition at timestep 4. Process cannot -complete due to violated condition_overall. - -[Agent reasoning] -"So the planner DOES know about the delay but isn't properly planning around it. -This could be a limitation of the planning algorithm's handling of process timing. - -Let me check the other tasks that succeeded to see what's different." - -[Tool call 4] -inspect_train_tasks(task_idx=2) - -[Tool result] -Task 2: - Goal: {JugFilled(jug0), JugFilled(jug1)} - Initial atoms: {AtFaucet(jug0, faucet0): False, AtFaucet(jug1, faucet0): False, ...} - Objects: [robot0:robot, jug0:jug, jug1:jug, faucet0:faucet, burner0:burner] - -[Agent reasoning] -"Task 2 requires filling TWO jugs. Perhaps the issue is that the planner is trying -to be efficient and interleave actions, but doesn't properly account for process -timing. - -Looking back at my FillJug process, I think the issue is that I need to make the -'WAIT for process' more explicit. Actually, I wonder if I should model this -differently... - -Wait, let me check something. In my process, I have condition_overall which -should prevent moving. But maybe I need to also add a precondition to the -MoveToPosition option that says 'cannot move if FillingInProgress'? - -Actually, that's not in my scope - I can only propose abstractions, not modify -the planning algorithm or option preconditions. - -Let me try a different approach: What if I add a 'RobotAtFaucet' predicate that's -more explicit about the robot's location, and use that in the process condition? -Or what if I add an explicit 'FillingActive' predicate that becomes true during -filling? - -Actually, I think the issue might be simpler. Let me look at the condition_overall -more carefully. I required BOTH AtFaucet AND Holding throughout. But maybe I should -add an additional constraint or make the delay explicit in another way. - -Hmm, actually maybe this is not an abstraction problem but a planning algorithm -limitation. Let me instead check if" there are other abstractions I'm missing for -other tasks." - -[Tool call 5] -test_planning(task_idx=0, timeout=30) -test_planning(task_idx=1, timeout=30) - -[Tool results] -Task 0: Success (plan length 6, nodes: 142) -Task 1: Success (plan length 8, nodes: 178) - -[Agent reasoning] -"Tasks 0 and 1 succeed. Let me see what's different... - -Task 0: {WaterBoiling(jug0)} - single jug -Task 1: {WaterBoiling(jug1), OnBurner(jug2)} - two jugs but different operations -Task 2: {JugFilled(jug0), JugFilled(jug1)} - two jugs, SAME operation - -Ah! The issue is that Task 2 requires filling two jugs using the same faucet. The -planner might be trying to optimize by starting both, but can't properly handle the -temporal sequencing. - -I don't think this is an abstraction problem - my abstractions are correct. This -is a planning algorithm limitation with temporal processes. I should note this -but not propose new abstractions. - -Actually, wait. Let me think if there's a way to model this better. What if I -explicitly model 'faucet occupancy' or 'water source availability'? That might -help the planner understand it can't use the faucet for two jugs at once... - -Let me propose a 'FaucetAvailable' predicate and modify the FillJug process to -reserve the faucet." - -[Tool call 6] -propose_predicates( - code=""" -# Faucet is not currently being used for filling -def _faucet_available_holds(state, objects): - faucet = objects[0] - # Check if flow_rate is zero (not actively filling) - return state.get(faucet, "flow_rate") < 0.01 - -proposed_predicates = [ - Predicate("FaucetAvailable", [_faucet_type], _faucet_available_holds) -] -""", - description="Track whether faucet is available for use" -) - -[Tool result] -Successfully proposed 1 predicates: ['FaucetAvailable'] - -[Tool call 7] -propose_processes( - code=""" -v_robot = Variable("?robot", _robot_type) -v_jug = Variable("?jug", _jug_type) -v_faucet = Variable("?faucet", _faucet_type) - -proposed_processes = [ - ExogenousProcess( - name="FillJugV2", # Updated version - parameters=[v_robot, v_jug, v_faucet], - condition_at_start={ - LiftedAtom(AtFaucet, [v_jug, v_faucet]), - LiftedAtom(Holding, [v_robot, v_jug]), - LiftedAtom(FaucetAvailable, [v_faucet]) # NEW: Check availability - }, - condition_overall={ - LiftedAtom(AtFaucet, [v_jug, v_faucet]), - LiftedAtom(Holding, [v_robot, v_jug]) - }, - condition_at_end=set(), - add_effects={ - LiftedAtom(JugFilled, [v_jug]) - }, - delete_effects={ - LiftedAtom(FaucetAvailable, [v_faucet]) # NEW: Reserve faucet - }, - delay_distribution=DiscreteGaussianDelay(mean=5, variance=1), - strength=torch.tensor([1.0]) - ) -] -""", - description="Updated FillJug with faucet availability tracking" -) - -[Tool result] -Successfully proposed 1 processes: ['FillJugV2'] - -[Agent reasoning] -"Good! Now the process explicitly models faucet availability. This should prevent -the planner from trying to fill two jugs simultaneously at the same faucet. Let me -test this." - -[Tool call 8] -test_planning(task_idx=2, timeout=30) - -[Tool result] -Planning succeeded for task 2! -Plan length: 10 -Nodes expanded: 203 -Plan: - MoveToFaucet(robot0, jug0, faucet0) - GraspJug(robot0, jug0) - [WAIT: FillJugV2(robot0, jug0, faucet0) - 5 timesteps] - MoveToPosition(robot0, jug0, table_position) - ReleaseJug(robot0, jug0) - MoveToFaucet(robot0, jug1, faucet0) - GraspJug(robot0, jug1) - [WAIT: FillJugV2(robot0, jug1, faucet0) - 5 timesteps] - ... - -[Agent final message] -"Success! I fixed the planning failure by adding explicit resource modeling: -- New predicate: FaucetAvailable -- Updated process: FillJugV2 (checks and reserves faucet availability) - -This prevents concurrent filling attempts and enables proper temporal sequencing. -All 3 tasks now plan successfully." -``` - -**Iteration 1 Results:** -- **New proposals:**1 predicate (FaucetAvailable), 1 process (FillJugV2) -- **Planning improvement:** 2/3 → 3/3 tasks solved (100%) -- **Agent turns:** 8 tool calls -- **Root cause:** Resource contention not modeled -- **Solution:** Explicit availability tracking - -**Compare to old approach Iteration 1:** -``` -Old approach: -1. Get new trajectories (including failed ones) -2. Re-prompt LLM with ALL 8 trajectories (10,000+ tokens) -3. LLM proposes predicates again (may propose same ones or different ones) -4. Data-driven process learning still struggles with sparse 2 process patterns -5. No explicit debugging of planning failures -6. May or may not discover the faucet availability issue -``` - ---- - -## Implementation Highlights (Detailed) - -### System Prompt: Guiding the Agent - -The agent receives a comprehensive system prompt that defines its role: - -```python -_SYSTEM_PROMPT = """ -You are an abstraction inventor for a bilevel process planning system. Your role -is to propose types, predicates, helper objects, processes, and options that help -a task planner solve planning problems. - -## What You Observe -You observe the world ONLY through: -- **Trajectory data**: sequences of states (feature vectors per object) and actions -- **Task goals**: symbolic goal descriptions -- **Planning metrics**: success rate, nodes expanded, failure reasons -- **Current abstractions**: types, predicates, processes, and options currently in use - -You do NOT have access to environment source code, simulator internals, or -ground-truth models. You must infer useful abstractions from observed data. - -## Code Conventions -When writing proposal code, the following are available: - -### Current abstractions (injected into exec context) -- Each type T is available as _T_type (e.g., _domino_type, _robot_type) -- Each predicate P is available by name (e.g., Fallen, Standing) -- Each predicate classifier is available as _P_holds -- Each option O is available by name (e.g., Push) - -### Expected outputs -- propose_types: must define proposed_types (list of Type objects) -- propose_predicates: must define proposed_predicates (list of Predicate objects) -- propose_processes: must define proposed_processes (list of CausalProcess objects) -... - -## Iteration Protocol -At each learning iteration: -1. **Inspect** trajectory data and planning results -2. **Form hypotheses** about missing abstractions -3. **Propose** new abstractions -4. **Test** proposals interactively -5. **Refine** based on test results - -Focus on abstractions that help planning. Pay attention to: -- States where planning fails - what conditions are missing? -- Patterns in trajectories not captured by current predicates -- Whether helper objects could simplify the problem -""" -``` - -**Key aspects:** -- Agent knows it's a **discovery agent**, not a question-answerer -- Explicitly told to use **tools to explore** before proposing -- Understands the **code conventions** (how to reference types/predicates) -- Has clear **iteration protocol** to follow - -### Iteration Message: Context Updates - -Each cycle, agent receives a status update: - -```python -def build_iteration_message(cycle, num_new_trajs, num_total_trajs, - task_success_rate, types, predicates, processes, - planning_success, failures, prev_outcomes): - return f""" -== Online Learning Iteration {cycle} == - -TRAJECTORY SUMMARY: -- {num_new_trajs} new trajectories collected this cycle -- {num_total_trajs} total trajectories (offline + online) -- Task success rate: {task_success_rate:.1%} - -CURRENT ABSTRACTIONS: -- Types: {types} -- Predicates ({len(predicates)}): {predicates} -- Processes ({len(processes)}): {processes} -- Options ({len(options)}): {options} - -PLANNING PERFORMANCE: -- Success: {planning_success} -- Avg nodes expanded: {avg_nodes} -- Failures: {failures} - -PREVIOUS ITERATION OUTCOMES: -{prev_outcomes} - -Your task: Inspect the new trajectory data, analyze planning failures, and -propose abstractions that will improve planning success. -""" -``` - -**Agent uses this to:** -- See what changed since last iteration (# new trajectories) -- Know current abstraction inventory -- Identify planning problems to fix -- Build on previous iteration's work - -### Session Management: Persistent Agent - -```python -class AgentSessionManager: - """Manages persistent Claude SDK session across iterations""" - - def __init__(self, system_prompt, mcp_server, log_dir, model_name): - self._system_prompt = system_prompt - self._mcp_server = mcp_server # Contains all 15 tools - self._client = None # Lazy initialization - self._session_id = None - self._total_cost_usd = 0.0 - self._total_turns = 0 - - async def start_session(self): - """Start Claude SDK client with MCP tools""" - from claude_agent_sdk import ClaudeSDKClient, ClaudeAgentOptions - - # Define which tools agent can access - tool_prefix = "mcp__predicator_tools__" - allowed_tools = [ - f"{tool_prefix}inspect_types", - f"{tool_prefix}inspect_trajectories", - f"{tool_prefix}propose_predicates", - # ... all 15 tools - ] - - options = ClaudeAgentOptions( - allowed_tools=allowed_tools, - mcp_servers={"predicator_tools": self._mcp_server}, - permission_mode="bypassPermissions", # No manual approval - system_prompt=self._system_prompt, - model=self._model_name, # e.g., "claude-sonnet-4" - max_turns=CFG.agent_sdk_max_agent_turns_per_iteration, - ) - - self._client = ClaudeSDKClient(options=options) - await self._client.connect() - self._started = True - - async def query(self, message: str) -> List[Dict[str, Any]]: - """Send message, collect all responses (text + tool calls)""" - if not self._started: - await self.start_session() - - collected = [] - - # Send message to agent - await self._client.query(message) - - # Iterate through agent responses - async for msg in self._client.receive_response(): - if isinstance(msg, AssistantMessage): - # Agent's reasoning text and tool calls - entry = {"type": "assistant", "content": []} - for block in msg.content: - if isinstance(block, TextBlock): - entry["content"].append({ - "type": "text", - "text": block.text - }) - elif isinstance(block, ToolUseBlock): - entry["content"].append({ - "type": "tool_use", - "name": block.name, - "input": block.input - }) - collected.append(entry) - - elif isinstance(msg, ResultMessage): - # Final result with cost/turn metadata - self._total_cost_usd += msg.total_cost_usd - self._total_turns += msg.num_turns - collected.append({ - "type": "result", - "num_turns": msg.num_turns, - "total_cost_usd": msg.total_cost_usd - }) - - return collected - - async def _recover_session(self, last_message): - """Automatically recover from errors""" - logging.warning("Session error, attempting recovery...") - try: - if self._client: - await self._client.disconnect() - self._started = False - await self.start_session() # Fresh session - logging.info("Recovery successful") - except Exception as e: - logging.error(f"Recovery failed: {e}") -``` - -**Key features:** -- **Persistent session**: Agent's context preserved across iterations -- **Cost tracking**: Know exactly how much each iteration costs -- **Auto-recovery**: Handles errors gracefully -- **Async design**: Efficient I/O for tool calls - -### ToolContext: Shared Mutable State - -```python -@dataclass -class ToolContext: - """Shared state accessible to all MCP tools""" - # Current abstractions - types: Set[Type] = field(default_factory=set) - predicates: Set[Predicate] = field(default_factory=set) - processes: Set[CausalProcess] = field(default_factory=set) - options: Set[ParameterizedOption] = field(default_factory=set) - - # Task and trajectory data - train_tasks: List[Task] = field(default_factory=list) - offline_trajectories: List[LowLevelTrajectory] = field(default_factory=list) - online_trajectories: List[LowLevelTrajectory] = field(default_factory=list) - example_state: Optional[State] = None - - # Planning feedback - planning_results: Dict[str, Any] = field(default_factory=dict) - iteration_history: List[Dict[str, Any]] = field(default_factory=list) - - # Proposals accumulator (reset each iteration) - iteration_proposals: ProposalBundle = field(default_factory=ProposalBundle) -``` - -**Design rationale:** -- Tools are **closures** over ToolContext: `create_mcp_tools(ctx)` -- All tools share same context → agent's proposals accumulate -- Context updated by approach → tools always see latest state -- Avoids passing tons of arguments to each tool - -### MCP Server Creation - -```python -def create_mcp_tools(ctx: ToolContext) -> list: - """Create all tools as closures over ctx""" - from claude_agent_sdk import tool - - @tool("inspect_types", "List all object types and features", {}) - async def inspect_types(args): - lines = [] - for t in sorted(ctx.types, key=lambda t: t.name): - features = ", ".join(t.feature_names) - lines.append(f"- {t.name}[{features}]") - return _text_result("\n".join(lines)) - - @tool("propose_predicates", "Propose new predicates", {...schema...}) - async def propose_predicates(args): - code = args["code"] - exec_ctx = build_exec_context(ctx.types, ctx.predicates, ctx.options) - result, error = exec_code_safely(code, exec_ctx, "proposed_predicates") - # ... validation ... - ctx.iteration_proposals.proposed_predicates |= validated - return _text_result(f"Proposed {len(validated)} predicates") - - # ... 13 more tools ... - - return [inspect_types, inspect_trajectories, propose_predicates, ...] - -# In approach: -tools = create_mcp_tools(self._tool_context) -mcp_server = create_sdk_mcp_server( - name="predicator_tools", - version="1.0.0", - tools=tools -) -``` - -**Key points:** -- Each tool is an **async function** (for SDK compatibility) -- Tools **close over** ToolContext → share state -- Tools have **JSON schemas** defining inputs ( SDK validates) -- Returns dict with `{"content": [...]}` format - ---- - -## Logging & Observability (Comprehensive) - -The new approach provides **dramatically better logging** for debugging and analysis: - -### Directory Structure -``` -logs/agent_sdk/ -├── session_info.json # Overall session metadata -└── iteration_0/ - ├── context_message.txt # What we told agent this iteration - ├── agent_responses.jsonl # Line-delimited JSON of all agent activity - └── proposals/ - ├── types.json # Names of proposed types - ├── predicates_validated.json # Names of validated predicates - ├── processes_code.json # Names of proposed processes - └── augmentor_code.py # Code for task augmentation (if any) -├── iteration_1/ - └── ... -└── iteration_N/ -``` - -### session_info.json -```json -{ - "session_id": "session_abc123_20260212_143022", - "total_cost_usd": 3.47, - "total_turns": 127, - "model": "claude-sonnet-4" -} -``` - -**Tracks cumulative costs** across all iterations - -### context_message.txt -``` -== Online Learning Iteration 1 == - -TRAJECTORY SUMMARY: -- 5 new trajectories collected this cycle -- 8 total trajectories (3 offline + 5 online) -- Task success rate: 60% - -CURRENT ABSTRACTIONS: -[Full state of system] - -PLANNING PERFORMANCE: -[Detailed metrics] - -Your task: [Specific guidance] -``` - -**Shows exactly what context agent received** - crucial for debugging why agent made certain decisions - -### agent_responses.jsonl -Each line is a JSON object representing one agent action: - -```jsonl -{"type": "assistant", "content": [{"type": "text", "text": "I need to understand..."}]} -{"type": "assistant", "content": [{"type": "tool_use", "name": "inspect_train_tasks", "input": {}}]} -{"type": "assistant", "content": [{"type": "text", "text": "I see that tasks involve..."}, {"type": "tool_use", "name": "inspect_trajectories", "input": {"traj_idx": 0, "max_timesteps": 5}}]} -{"type": "assistant", "content": [{"type": "text", "text": "Let me test my hypothesis..."}, {"type": "tool_use", "name": "test_predicate_on_states", "input": {"predicate_name": "AtFaucet", ...}}]} -{"type": "result", "num_turns": 11, "total_cost_usd": 0.23} -``` - -**Complete trace** of agent reasoning and tool usage - can reconstruct entire thought process - -### proposals/ directory files - -**predicates_validated.json:** -```json -[ - "AtFaucet", - "Holding", - "JugFilled", - "OnBurner", - "WaterBoiling" -] -``` - -**processes_code.json:** -```json -[ - "FillJug", - "HeatWater" -] -``` - -**augmentor_code.py:** (if proposed) -```python -def augment_task(task: Task) -> Task: - # Full code saved for reproducibility - grid_cells = [] - for row in range(5): - for col in range(5): - cell = Object(f"cell_{row}_{col}", _grid_cell_type) - ... - return Task(augmented_init, task.goal) -``` - -### Old Approach Logging (Comparison) - -``` -logs/online_predicate_invention_and_process_planning/ -├── ite0_b0_s1_spec.prompt # Prompt for spec generation -├── ite0_b0_s1_spec.response # LLM response -├── ite0_b0_s2_impl.prompt # Prompt for implementation -├── ite0_b0_s2_impl.response # LLM response (code blocks) -└── ite0_obs/ # Images if CFG.rgb_observation - ├── state_000.png - └── state_001.png -``` - -**Problems:** -- Only sees prompts/responses, not reasoning process -- No tool-by-tool trace of agent decisions -- Can't see why agent chose to query certain data -- No structured proposals tracking -- Harder to debug failures - -### Observability Benefits - -1. **Reproducibility**: Can replay exact agent reasoning from logs -2. **Debugging**: See where agent got stuck or made wrong hypothesis -3. **Cost tracking**: Know exactly how much each iteration costs -4. **Analysis**: Study agent strategies across different domains -5. **Failure analysis**: Identify when agent didn't use available tools effectively - -### Example Analysis Using Logs - -```python -# Script to analyze agent behavior -import json - -def analyze_iteration(iteration_dir): - with open(f"{iteration_dir}/agent_responses.jsonl") as f: - responses = [json.loads(line) for line in f] - - # Count tool uses - tool_counts = {} - for r in responses: - if r["type"] == "assistant": - for block in r["content"]: - if block["type"] == "tool_use": - tool_counts[block["name"]] = tool_counts.get(block["name"], 0) + 1 - - # Identify reasoning patterns - text_blocks = [ - block["text"] for r in responses if r["type"] == "assistant" - for block in r["content"] if block["type"] == "text" - ] - - return { - "tool_usage": tool_counts, - "num_reasoning_steps": len(text_blocks), - "inspection_vs_proposal_ratio": ( - sum(v for k, v in tool_counts.items() if "inspect" in k) / - sum(v for k, v in tool_counts.items() if "propose" in k) - ) - } - -# Results might show: -# { -# "tool_usage": { -# "inspect_trajectories": 3, -# "inspect_train_tasks": 1, -# "test_predicate_on_states": 2, -# "propose_predicates": 1, -# "propose_processes": 2, -# "test_planning": 2 -# }, -# "num_reasoning_steps": 11, -# "inspection_vs_proposal_ratio": 1.5 # Agent inspects 1.5x more than proposes (good!) -# } -``` - ---- - -## When to Use Each Approach - -### Use **Old Approach** if: -- You have well-defined prompt templates that work -- Context fits in prompt easily -- You want simple, debuggable prompting flow -- Don't need interactive exploration - -### Use **New Approach** if: -- Trajectory data is large (token concerns) -- Want agent to discover its own strategy -- Need testing/validation before commitment -- Want richer abstractions (types, task augmentors, options) -- Value persistent learning across iterations -- Want to leverage Claude's reasoning for exploration - ---- - -## Future Extensions - -The MCP tool architecture enables easy additions: - -- **`propose_heuristics`** - Let agent define domain-specific planning heuristics -- **`analyze_failure`** - Give agent access to execution traces on failed tasks -- **`suggest_training_tasks`** - Agent proposes informative tasks to try -- **`query_environment_model`** - (If available) Agent can test "what if" scenarios - -The old template-based approach would require significant refactoring for these capabilities. - ---- - -## Performance Considerations (Detailed Analysis) - -### Token Efficiency - -#### Old Approach (Per Iteration) -``` -Prompt composition: -- Template boilerplate: ~500 tokens -- Type definitions: ~200 tokens -- All trajectory states: ~8,000 tokens [MAJOR COST] -- Task specifications: ~300 tokens -- Example code: ~400 tokens -Total input: ~9,400 tokens per iteration - -LLM response: -- Spec generation: ~1,000 tokens -- Implementation: ~2,000 tokens -Total output: ~3,000 tokens - -Cost per iteration (Claude Sonnet 4): -Input: 9,400 tokens × $3/MTok = $0.028 -Output: 3,000 tokens × $15/MTok = $0.045 -Total: ~$0.07 per iteration -``` - -**Problem:** Scales linearly with trajectories. With 50 trajectories = 50,000+ input tokens! - -#### New Approach (Per Iteration) -``` -Agent multi-turn dialogue: -- Context message: ~800 tokens (summary, not data) -- Agent reasoning: ~200 tokens per turn -- Tool inputs: ~50 tokens per tool call -- Tool results: ~500 tokens per result (selective data) - -Example iteration (11 turns): -Input: 800 + (11 × 200) + (11 × 50) = ~3,550 tokens -Output: ~(11 × 500) = ~5,500 tokens - -Cost per iteration: -Input: 3,550 × $3/MTok = $0.011 -Output: 5,500 × $15/MTok = $0.083 -Total: ~$0.09 per iteration -``` - -**Advantages:** -- ✅ Input tokens don't scale with trajectory count (agent queries selectively) -- ✅ Agent can choose to inspect 1 trajectory instead of all 50 -- ❌ More output tokens (agent reasoning) but provides value - -**Crossover Analysis:** -``` -Old approach: Cost = $0.07 + ($0.001 × num_trajectories) # Scales with data -New approach: Cost ≈ $0.09 × (1 + 0.1 × num_tool_calls) # Scales with complexity - -For 10 trajectories: Old=$0.08, New=$0.09 (similar) -For 50 trajectories: Old=$0.12, New=$0.09 (new better) -For 100 trajectories: Old=$0.17, New=$0.09 (new much better) -``` - -### Latency Comparison - -#### Old Approach -``` -Single LLM call: -- Prompt construction: ~0.5s -- LLM inference: ~8s (long context) -- Response parsing: ~0.2s -Total: ~8.7s per iteration -``` - -#### New Approach -``` -Multi-turn dialogue (11 turns): -- Context message: ~1s -- Agent turn 1 (inspect_train_tasks): ~2s -- Agent turn 2 (inspect_trajectories): ~3s -- Agent turn 3-10 (reasoning + tools): ~2s each = ~16s -- Agent turn 11 (final proposal): ~3s -Total: ~25s per iteration -``` - -**Tradeoff:** -- ❌ New approach is 3x slower per iteration -- ✅ But fewer iterations needed (better proposals first try) -- ✅ Can run in background / async - -**Projected end-to-end:** -``` -Old approach: 5 iterations × 8.7s = ~44s (but may need more iterations) -New approach: 2-3 iterations × 25s = ~50-75s (better quality) -``` - -### Proposal Quality - -#### Old Approach -```python -# Empirical results from experiments: -Iteration 0: -- Predicates proposed: 8 -- Predicates valid: 3 (37% validation rate) -- Processes induced: 0-1 (data-driven, needs many examples) -- Planning success: 20-40% - -Iteration 1: -- Predicates proposed: 6 (some re-proposed) -- Predicates valid: 4 (67% validation rate) -- Processes induced: 1-2 -- Planning success: 40-60% - -Iteration 2: -- Planning success: 60-80% -``` - -#### New Approach (Projected) -```python -Iteration 0: -- Predicates proposed: 5 -- Predicates valid: 5 (100% - tested before proposing!) -- Processes proposed: 2 (agent-reasoned, not data-induced) -- Planning success: 60-80% [Better from start] - -Iteration 1: -- Predicates proposed: 1-2 (refinements only) -- Predicates valid: 1-2 (100%) -- Processes proposed: 1-2 (refinements) -- Planning success: 80-95% - -Iteration 2: -- Planning success: 95-100% -``` - -**Key difference:** Higher validation rate (test before propose) + better process proposals (reasoning vs. induction) - -### API Call Comparison - -#### Old Approach -``` -Per iteration: -- LLM API calls: 2 (spec generation + implementation) -- Total API calls per iteration: 2 -``` - -#### New Approach -``` -Per iteration: -- Agent API calls: ~5-15 (depends on exploration depth) -- Tool execution: local (no API costs) -- Total API calls per iteration: 5-15 -``` - -**Tradeoff:** -- ❌ More API calls (but faster due to shorter contexts) -- ✅ Can batch/parallelize tool results -- ✅ Early stopping if agent converges quickly - -### Memory and Compute - -#### Old Approach -``` -Memory usage: -- Load all trajectories into prompt: ~50MB (for 100 trajectories) -- LLM context window: ~100K tokens needed for large datasets - -Compute: -- Trajectory segmentation: ~2s per trajectory (CPU-heavy) -- Process induction: ~5-10s (graph search) -- Total offline compute: ~10-15s per iteration -``` - -#### New Approach -``` -Memory usage: -- In-memory ToolContext: ~5-10MB (just references) -- Agent context: ~20K tokens (selective queries) - -Compute: -- Tool executions: ~0.1s per tool call (mostly lookups) -- Process proposals: instant (no induction) -- Total offline compute: ~1s per iteration -``` - -**Winner:** New approach has lower compute costs - -### Cost Projections for Full Learning Run - -#### Old Approach (5 iterations) -``` -Iteration 0: 10 trajs → $0.08 -Iteration 1: 20 trajs → $0.09 -Iteration 2: 35 trajs → $0.11 -Iteration 3: 50 trajs → $0.12 -Iteration 4: 70 trajs → $0.14 -Total: ~$0.54 + compute costs -``` - -#### New Approach (3 iterations, higher quality) -``` -Iteration 0: 10 trajs → $0.09 -Iteration 1: 20 trajs → $0.11 (more tool calls to debug) -Iteration 2: 35 trajs → $0.09 (converged, fewer explorations) -Total: ~$0.29 + minimal compute -``` - -**Projected savings:** ~46% cost reduction + fewer iterations - -### Scalability Analysis - -| Metric | Old Approach | New Approach | Winner | -|--------|--------------|--------------|---------| -| **Token scaling w/ trajectories** | Linear O(n) | Constant O(1) | ✅ New | -| **Proposal quality** | 50-70% valid | 95-100% valid | ✅ New | -| **Time per iteration** | ~9s | ~25s | ✅ Old | -| **Iterations needed** | 4-6 | 2-3 | ✅ New | -| **Total wall time** | 36-54s | 50-75s | ≈ Tie | -| **Total cost (50 trajs)** | ~$0.45 | ~$0.29 | ✅ New | -| **Debuggability** | Low | High | ✅ New | -| **Process quality** | Data-limited | Reasoning-based | ✅ New | -| **Extensibility** | Hard | Easy | ✅ New | - -### Summary: When to Use Each - -**Use Old Approach when:** -- Small datasets (<20 trajectories) -- Well-understood domain with templates -- Minimizing iteration time is critical -- Don't need processes or only simple ones -- Cost is not a concern - -**Use New Approach when:** -- Large datasets (>30 trajectories) -- Complex domains requiring exploration -- Need high-quality processes -- Want interactive debugging -- Need extensibility for new abstraction types -- Long-term cost optimization matters - ---- - ---- - -## Configuration and Settings - -The new approach introduces several config flags for controlling agent behavior: - -### Core Settings - -```python -# Agent model selection -CFG.agent_sdk_model_name = "claude-sonnet-4" -# Options: "claude-sonnet-4", "claude-opus-4", "claude-haiku-3.5" - -# Max turns per iteration (prevents runaway loops) -CFG.agent_sdk_max_agent_turns_per_iteration = 15 - -# What abstractions can agent propose? -CFG.agent_sdk_propose_types = True # Allow new type proposals -CFG.agent_sdk_propose_predicates = True # Allow predicate proposals -CFG.agent_sdk_propose_objects = True # Allow task augmentation -CFG.agent_sdk_propose_processes = True # Allow process proposals -CFG.agent_sdk_propose_options = False # Usually False (options given) - -# Logging -CFG.agent_sdk_log_agent_responses = True # Save agent_responses.jsonl - -# Process parameter learning -CFG.learn_process_parameters = True # Learn params for agent processes -``` - -### Comparison to Old Approach Settings - -**Old approach used:** -```python -CFG.llm_model_name = "gpt-4" # Which LLM for prompting -CFG.vlm_predicator_num_proposal_batches = 3 # How many prompt batches -CFG.vlm_predicator_oracle_base_predicates = False # Use oracle predicates -CFG.predicate_invent_neural_symbolic_predicates = False # Not supported -``` - -**Key differences:** -- New approach doesn't need "proposal batches" (agent explores adaptively) -- Old approach had "oracle predicate" shortcuts; new approach learns from data only -- Old approach had many prompt template options; new approach uses system prompt - -### Recommended Configurations - -**For experimentation / development:** -```python -CFG.agent_sdk_model_name = "claude-sonnet-4" # Good balance cost/quality -CFG.agent_sdk_max_agent_turns_per_iteration = 20 # Allow thorough exploration -CFG.agent_sdk_log_agent_responses = True # Debug agent reasoning -``` - -**For production / evaluations:** -```python -CFG.agent_sdk_model_name = "claude-sonnet-4" # Optimal for most domains -CFG.agent_sdk_max_agent_turns_per_iteration = 12 # Prevent overly long iterations -CFG.agent_sdk_log_agent_responses = True # Keep for analysis -``` - -**For budget-constrained experiments:** -```python -CFG.agent_sdk_model_name = "claude-haiku-3.5" # 10x cheaper -CFG.agent_sdk_max_agent_turns_per_iteration = 8 # Limit turns -``` - ---- - -## Conclusion: A Paradigm Shift - -**AgentSDKOnlineProcessPlanningApproach** represents a fundamental shift from **batch prompting** to **interactive exploration** for abstraction learning. - -### The Core Innovation - -Traditional approach: "Here's all the data, please propose abstractions" -- Limited by context window -- Cannot test hypotheses -- One-shot, hoping for the best - -New approach: "You have tools to explore data; discover abstractions iteratively" -- Agent decides what to examine -- Tests before proposing -- Refines based on feedback - -### Key Advantages Realized - -1. **Superior Proposal Quality** - - 95-100% validation rate vs. 50-70% - - Predicates tested before proposal - - Processes reasoned, not just induced - -2. **Better Scalability** - - Token costs constant w.r.t. dataset size - - Old approach: O(n) with trajectories - - 46% cost savings projected on large datasets - -3. **Richer Abstractions** - - Can propose types (not just predicates) - - Can propose task augmentors (helper objects) - - Can propose options (if needed) - - Old approach: only predicates, data-induced processes - -4. **Interactive Debugging** - - Agent sees errors immediately - - Can test hypotheses with `test_predicate_on_states` - - Can validate abstractions help planning via `test_planning` - - Old approach: errors discovered post-facto - -5. **Extensibility** - - New tools can be added without changing agent code - - MCP architecture isolates concerns - - Old approach: new capabilities require template rewrites - -6. **Superior Observability** - - Complete reasoning trace in logs - - Tool-by-tool decision tracking - - Cost and timing metadata - - Old approach: only prompt/response pairs - -### Technical Achievements - -**Architecture:** -- Model Context Protocol provides clean abstraction boundary -- Tools as closures over ToolContext enable state sharing -- Async session management handles complexity gracefully -- Safe code execution prevents security issues - -**Agent Design:** -- System prompt provides clear guidance without over-constraining -- Iteration messages give contextual updates -- Testing tools enable hypothesis validation -- Proposal tools enforce validation before integration - -**Process Learning:** -- Agent-proposed processes skip expensive induction -- Can propose novel structures (conditional delays, complex conditions) -- Better with limited data (uses reasoning not just patterns) - -### Remaining Challenges - -1. **Latency:** 3x slower per iteration than old approach - - Mitigated by: fewer iterations needed, background execution possible - - Future work: parallel tool execution, streaming responses - -2. **Agent Reliability:** Depends on Claude SDK stability - - Mitigated by: auto-recovery, session persistence - - Future work: fallback mechanisms, local model support - -3. **Planning Algorithm Coupling:** Some failures are planner limitations, not abstraction issues - - Agent can recognize but not fix planner bugs - - Future work: give agent ability to propose heuristics - -### Success Metrics (Projected) - -Compared to old approach on standard benchmarks: - -| Domain | Old Success | New Success | Old Cost | New Cost | Winner | -|--------|-------------|-------------|----------|----------|---------| -| Blocks (20 trajs) | 75% | 85% | $0.09 | $0.10 | ≈ Tie | -| BoilWater (30 trajs) | 60% | 80% | $0.11 | $0.09 | ✅ New | -| Domino (50 trajs) | 40% | 70% | $0.15 | $0.10 | ✅✅ New | -| Complex (100 trajs) | 30% | 65% | $0.22 | $0.11 | ✅✅✅ New | - -Takeaway: **Bigger advantage on complex domains with more data** - -### When Each Approach Wins - -**Old Approach Best For:** -- ✅ Simple domains with <20 trajectories -- ✅ Well-understood domains with working templates -- ✅ When minimizing iteration latency is critical -- ✅ When LLM access is easier than Agent SDK setup - -**New Approach Best For:** -- ✅ Complex domains requiring exploration -- ✅ Large trajectory datasets (>30 trajs) -- ✅ When proposal quality matters most -- ✅ When processes are critical to planning -- ✅ Long-term projects where extensibility matters -- ✅ Research settings where observability needed - -### Future Directions - -The MCP tool architecture enables exciting extensions: - -**Near-term:** -- `propose_heuristics`: Let agent define domain-specific planning heuristics -- `analyze_failure_trace`: Give agent access to detailed execution failures -- `query_subgoal_library`: Agent can reference common subgoal patterns - -**Medium-term:** -- `simulate_action_outcome`: Agent can test "what if" scenarios -- `cross_domain_transfer`: Agent queries similar domains for inspiration -- `propose_derived_predicates`: Agent creates predicates as logical combinations - -**Long-term:** -- Multi agent collaboration (one agent proposes, another critiques) -- Continuous learning (agent improves abstractions during deployment) -- Human-in-the-loop refinement (ask human expert via tool) - -### The Bottom Line - -**AgentSDKOnlineProcessPlanningApproach** transforms abstraction learning from: -- A prompting problem → An interactive AI research problem -- One-shot generation → Iterative hypothesis testing -- String parsing → Structured tool use -- Black-box LLM → Observable agent reasoning - -**The approach mirrors human problem-solving:** explore, hypothesize, test, refine, validate. - -For complex domains, this **paradigm shift pays dividends** in proposal quality, scalability, and extensibility. The future of abstraction learning is **interactive agents**, not batch prompts. - ---- - -## Appendix: Quick Reference - -### Agent Workflow Summary -``` -1. Receive context message (current state + planning results) -2. Inspect relevant data via tools (selective queries) -3. Form hypotheses about missing abstractions -4. Test hypotheses interactively -5. Propose validated abstractions via tools -6. Proposals accumulate in ProposalBundle -7. Integrate validated proposals into approach state -8. Save state and logs -9. Next iteration with updated context -``` - -### Tool Categories Quick Ref -- **Inspection (8 tools):** inspect_types, inspect_predicates, inspect_processes, inspect_options, inspect_trajectories, inspect_train_tasks, inspect_planning_results, inspect_past_proposals -- **Proposal (5 tools):** propose_types, propose_predicates, propose_object_augmentor, propose_processes, propose_options -- **Testing (2 tools):** test_predicate_on_states, test_planning - -### Key Classes Quick Ref -- **ToolContext:** Shared state between approach and tools -- **ProposalBundle:** Accumulates proposals during one iteration -- **AgentSessionManager:** Manages persistent Claude SDK session -- **Safe execution:** `exec_code_safely()`, `build_exec_context()`, `validate_predicate()` - -### Common Pitfalls & Solutions -1. **Agent gets stuck in inspection loop** - - Solution: Set `CFG.agent_sdk_max_agent_turns_per_iteration` -2. **Proposals reference undefined types** - - Solution: Execution context includes `_typename_type` convention -3. **High costs** - - Solution: Use claude-haiku-3.5 or limit max turns -4. **Session crashes** - - Solution: Auto-recovery mechanism handles most cases; check logs - -### Performance Cheat Sheet -- Tokens: New O(1), Old O(n) in trajectories -- Latency: New ~25s/iter, Old ~9s/iter -- Quality: New 95-100% valid, Old 50-70% valid -- Iterations: New 2-3, Old 4-6 -- Cost (50 trajs): New ~$0.29, Old ~$0.45 -- **Overall winner: New approach for complex domains** diff --git a/docs/sysid/make_sysid_pipeline_fig.py b/docs/sysid/make_sysid_pipeline_fig.py index 0088c0e26..6c3ee06bc 100644 --- a/docs/sysid/make_sysid_pipeline_fig.py +++ b/docs/sysid/make_sysid_pipeline_fig.py @@ -84,7 +84,7 @@ ("D1", 3, 7.0, "Belief env", "applied to base env for planning;\n" "fresh validation envs re-apply;\n" "dropped params revert to registry", None), - ("D2", 3, 10.0, "Capture gate", "evaluate_option_plan: parse ->\n" + ("D2", 3, 10.0, "Capture gate", "submit_plan: parse ->\n" "legitimacy -> 3x/6x decorrelated\n" "validation -> 32-pt hull sweep\n" "-> PARAM-SENSITIVE on failure", None), diff --git a/predicators/agent_sdk/belief_probe.py b/predicators/agent_sdk/belief_probe.py index a203b0750..96e0f7509 100644 --- a/predicators/agent_sdk/belief_probe.py +++ b/predicators/agent_sdk/belief_probe.py @@ -1,17 +1,17 @@ -"""Exploration probe API exposed to agents via ``explore_python``. +"""Exploration probe API exposed to agents via ``run_python``. ``BeliefProbe`` is a thin facade over the machinery the curated tools already use - ``parse_sketch_from_text`` (plan grammar), ``execute_plan_forward`` (forward executor over the option model), the tools' state-modification and rendering helpers - so probe rollouts -behave identically to ``evaluate_option_plan`` rollouts. What it adds is +behave identically to ``submit_plan`` rollouts. What it adds is composability: the agent can set the sim to any task state (or a modified copy), read full-precision features, run partial plans, render, -snapshot/restore, and write sweep loops in one ``explore_python`` call +snapshot/restore, and write sweep loops in one ``run_python`` call instead of one tool round-trip per experiment. By construction nothing the probe executes can be captured as the -answer - submission happens only through ``evaluate_option_plan`` on the +answer - submission happens only through ``submit_plan`` on the true initial state. The task evaluator is reachable, but only as a read-only preview: ``run(trials>=2, solved=True)`` and ``refine(require_solved=True)`` score rollouts through the same gate the @@ -61,8 +61,8 @@ class ProbeBudgetExceeded(Exception): """A probe call ran past a wall-clock budget. Raised cooperatively at probe checkpoints (every sim call) when the - explore_python per-call limit or the solve attempt's wall clock has - expired. ``explore_python`` catches it specially: the code's printed + run_python per-call limit or the solve attempt's wall clock has + expired. ``run_python`` catches it specially: the code's printed output so far is returned with the budget message appended, so a stopped sweep still hands the agent its partial results. """ @@ -83,12 +83,12 @@ def _check_time_budget(ctx: "ToolContext") -> None: raise ProbeBudgetExceeded( "the attempt's wall-clock exploration budget is exhausted. Stop " "exploring NOW and submit your single best plan via " - "evaluate_option_plan on the current task (omit task_idx).") - call_dl = ctx.explore_call_deadline + "submit_plan on the current task (omit task_idx).") + call_dl = ctx.python_call_deadline if call_dl is not None and now > call_dl: - call_timeout = ToolSurfaceConfig.from_cfg().explore_python_call_timeout + call_timeout = ToolSurfaceConfig.from_cfg().python_call_timeout raise ProbeBudgetExceeded( - f"this explore_python call exceeded its " + f"this run_python call exceeded its " f"{call_timeout:.0f}s time limit " "and was stopped between sim calls; output printed so far is " "returned above. Large sweeps are expensive - narrow the " @@ -201,7 +201,7 @@ def __len__(self) -> int: class ProbeResult(_StrLikeResult): """Outcome of one ``BeliefProbe.run`` call. - Attributes mirror the ``evaluate_option_plan`` report: ``steps`` is + Attributes mirror the ``submit_plan`` report: ``steps`` is a list of per-step dicts (``option``, ``num_actions``, ``failure``, ``added``, ``deleted``, ``subgoals_missing`` - the step's ``-> {atoms}`` annotations that did NOT hold in the post-state (the @@ -413,7 +413,7 @@ class ProbeRefineResult(_StrLikeResult): as more than it means. ``plan_lines`` holds one line per sketch step with the refined params filled in (``[?]`` for steps the search never refined) - paste them into ``sim.run`` or - ``evaluate_option_plan``. ``near_miss`` is the deepest validation + ``submit_plan``. ``near_miss`` is the deepest validation failure (step index, the exact params that got furthest, and why they failed), also populated on timeout/exhaustion. ``note`` carries caveats. @@ -516,7 +516,7 @@ class BeliefProbe: The "current state" is just a ``State`` object; ``run`` executes from it (the option model resets the sim env from that state, exactly as - ``evaluate_option_plan`` does from a task init) and advances it to + ``submit_plan`` does from a task init) and advances it to the rollout's final state. """ @@ -663,7 +663,7 @@ def task(self, task_idx: Optional[int] = None) -> str: ``reset(task_idx)`` + ``render()`` for the scene image. """ # pylint: disable-next=import-outside-toplevel - from predicators.agent_sdk.tools.inspection import render_task_digest + from predicators.agent_sdk.tools.digests import render_task_digest ctx = self._ctx if task_idx is None and ctx.probe_option_model_provider is not None: raise ValueError( @@ -721,6 +721,48 @@ def fit(self, "during learning; the solve-time belief model is fixed.") return provider(path=path, traj_idxs=traj_idxs, fixed=fixed) + def predicates(self, + max_trajectories: int = 10, + max_groundings_per_predicate: int = 4) -> str: + """Reload ``predicates.py`` and report milestone behaviour. + + Predicate-invention synthesis sessions only. Loads + ``LEARNED_PREDICATES`` fresh from the file (snapshotting it into + ``predicates_versions/``), validates each entry, installs the + set so ``run`` / ``refine`` abstract with the draft, and + reports, per predicate x grounding over the recorded + trajectories: coverage (ever-true / ever-false), flip counts, + and monotonicity (a milestone flips False->True once and + stays true). Call it after every edit of ``predicates.py``. + """ + loader = self._artifact_loader("predicates") + return loader( + max_trajectories=max_trajectories, + max_groundings_per_predicate=max_groundings_per_predicate) + + def samplers(self) -> str: + """Reload ``samplers.py`` and install its per-skill samplers. + + Sampler-synthesis sessions only. Loads ``LEARNED_SAMPLERS`` + fresh from the file (snapshotting it into + ``samplers_versions/``), validates the option-name -> callable + map, installs it so ``refine`` draws from the draft samplers, + and reports a per-option sanity check (return shape, in-box + draws) on a representative train-task state. Call it after every + edit of ``samplers.py``. + """ + return self._artifact_loader("samplers")() + + def _artifact_loader(self, name: str) -> Callable[..., str]: + ctx = self._ctx + _check_time_budget(ctx) + loader = ctx.probe_artifact_loaders.get(name) + if loader is None: + raise RuntimeError( + f"sim.{name} is unavailable in this session: it has no " + f"{name}.py surface to load.") + return loader + def residuals(self, max_transitions: int = 100, abs_tol: float = 1e-4, @@ -816,8 +858,7 @@ def _features(obj: Any) -> Dict[str, float]: def atoms(self) -> List[str]: """Sorted ground atoms true in the current state.""" cur = self._require_state() - preds = (self._ctx.predicates - | self._ctx.iteration_proposals.proposed_predicates) + preds = self._ctx.predicates return [str(a) for a in sorted(utils.abstract(cur, preds))] def render( @@ -880,8 +921,8 @@ def _parse_sketch(self, plan_text: str) -> Any: the probe task starts at the current state and carries no evaluator, and ``notices`` lists parse caveats to surface (e.g. region annotations ignored because ground samplers are off). - Same grammar and parser as ``evaluate_option_plan`` / - ``refine_plan_sketch`` (``~ [w]`` search regions included). + Same grammar and parser as ``submit_plan`` (``~ [w]`` search + regions included). """ # pylint: disable=import-outside-toplevel from predicators.agent_sdk import bilevel_sketch @@ -894,9 +935,8 @@ def _parse_sketch(self, plan_text: str) -> Any: init=cur, evaluator=None) - all_options = ctx.options | ctx.iteration_proposals.proposed_options - all_predicates = (ctx.predicates - | ctx.iteration_proposals.proposed_predicates) + all_options = ctx.options + all_predicates = ctx.predicates types = set(ctx.types) for opt in all_options: types.update(opt.types) @@ -965,7 +1005,7 @@ def run( ) -> Union[ProbeResult, ProbeTrialsResult, ProbeSweepResult]: """Execute an option plan from the current state. - ``plan_text`` uses the same grammar as ``evaluate_option_plan``: + ``plan_text`` uses the same grammar as ``submit_plan``: one option per line, ``Option(obj:type, ...)[params]`` with exact continuous params (``[]`` for none); ``-> {atoms}`` subgoal annotations are optional but CHECKED - each step's @@ -976,7 +1016,7 @@ def run( diverges here means a rule is more permissive than the env). Advances the current state to the rollout's final state (``restore`` a snapshot to rewind). - Like ``evaluate_option_plan``, each step's post-state is + Like ``submit_plan``, each step's post-state is rendered to a saved image whose path lands in the step report; pass ``render=False`` inside tight sweep loops to skip that. Exploratory only: results are never captured. @@ -997,7 +1037,7 @@ def run( ``solved``/``reward``. Reaching the goal atoms is NOT the same as being scored a solve - the evaluator can reject a goal-reaching route - so check ``solved`` counts here BEFORE submitting via - ``evaluate_option_plan`` instead of discovering rejections one + ``submit_plan`` instead of discovering rejections one submission at a time. ``contacts=True`` (single-run mode only, ``trials=1``) records @@ -1031,7 +1071,7 @@ def run( ``seed=S`` overrides the base motion-planner seed for this call. Trials report the planner seed each ran at (trial ``i`` runs at ``S + i``; without ``seed=`` at ``base + i``), and - ``evaluate_option_plan``'s validation rollouts report theirs the + ``submit_plan``'s validation rollouts report theirs the same way. A single run (``trials=1``) executes entirely at ``S``; a physics sweep runs every point at ``S`` instead of the base. @@ -1215,7 +1255,7 @@ def _one_point( # Fresh physics per trial when the session provides the scope # (solve sessions do; a synthesis probe's candidate model has # its own env, which the scope does not manage). Same CFG gate - # as evaluate_option_plan's validation rollouts, so the two + # as submit_plan's validation rollouts, so the two # surfaces sample the same distribution. fresh_scope = (ctx.validation_env_scope if ValidationConfig.from_cfg().fresh_env @@ -1417,7 +1457,7 @@ def _on_step(i: int, outcome: Any) -> None: f"state exactly (features: {feats}) - a failure " "here may partly reflect start-state " "reconstruction error, not plan margin.") - # Same per-step audit image evaluate_option_plan saves; the + # Same per-step audit image submit_plan saves; the # env already sits at the post-step state here. img = render_scene_image( ctx, @@ -1510,7 +1550,7 @@ def run_policy( Policy-mode counterpart of ``run``: loads the sandbox's ``policy.py`` fresh (or executes ``source`` directly) and drives its ``get_option(state, memory)`` through the belief model with - the same failure-surfacing semantics as ``evaluate_policy`` and + the same failure-surfacing semantics as ``submit_policy`` and the real executor - option failures land in ``memory['last_failure']`` and the policy is asked again; get_option bugs end the episode. @@ -1520,7 +1560,7 @@ def run_policy( and check that the policy RECOVERS from off-nominal states, not just the initial one. ``trials=N`` repeats the rollout from the SAME current state on fresh envs (fresh policy memory per - trial). Never captures - deliver via ``evaluate_policy``. Also + trial). Never captures - deliver via ``submit_policy``. Also available in learn sessions (probing a candidate simulator); there it runs against the candidate model. """ @@ -1551,9 +1591,8 @@ def run_policy( probe_task = dataclasses.replace(self._base_task, init=cur, evaluator=None) - all_options = ctx.options | ctx.iteration_proposals.proposed_options - all_predicates = (ctx.predicates - | ctx.iteration_proposals.proposed_predicates) + all_options = ctx.options + all_predicates = ctx.predicates types = set(ctx.types) for opt in all_options: types.update(opt.types) @@ -1832,8 +1871,8 @@ def refine(self, require_solved: bool = False) -> "ProbeRefineResult": """Backtracking parameter search for a sketch FROM THE CURRENT STATE. - Same grammar and search core as ``refine_plan_sketch``, but - composable: refine a plan *suffix* from a snapshot where the + Same grammar as ``submit_plan``, and composable: + refine a plan *suffix* from a snapshot where the prefix already executed, so the search budget goes to the step that matters instead of re-descending through the whole plan. Annotate each step's ``-> {subgoals}`` - success means every @@ -1880,7 +1919,7 @@ def refine(self, evaluator = self._require_solved_evaluator("require_solved") # Same gate (and therefore same accept policy: coarse and # evaluator errors never block, non-terminated never blocks) - # as refine_plan_sketch, so identical params can't get + # as submit_plan, so identical params can't get # contradictory verdicts across the two surfaces. inner_check = make_solved_check( evaluator, getattr(self._option_model(), "sim_env", None)) @@ -1905,8 +1944,8 @@ def gated_solved_check(states: List[State], labels: List[Any], max_samples_per_step = \ RefinementConfig.from_cfg().max_samples_per_step self._refine_calls += 1 - # Deterministic but distinct from refine_plan_sketch's - # CFG.seed + attempt streams and from other probe instances, so + # Deterministic but distinct from the solver's CFG.seed + attempt + # streams and from other probe instances, so # "try a different random search" does not replay failed draws. rng = np.random.default_rng(CFG.seed + 100003 * (self._instance_id + 1) + @@ -2008,7 +2047,7 @@ def _require_state(self) -> State: def build_probe_namespace(ctx: "ToolContext") -> Dict[str, Any]: - """The persistent ``explore_python`` namespace (solve sessions). + """The persistent ``run_python`` namespace (solve sessions). The probe facade, numpy, and the collected REAL trajectories as read-only evidence (``trajectories`` plus a ``describe_trajectory`` @@ -2026,7 +2065,7 @@ def build_probe_namespace(ctx: "ToolContext") -> Dict[str, Any]: import numpy as np # pylint: disable-next=import-outside-toplevel - from predicators.agent_sdk.tools.inspection import render_trajectory_digest + from predicators.agent_sdk.tools.digests import render_trajectory_digest all_trajs = list(ctx.offline_trajectories) + list(ctx.online_trajectories) def describe_trajectory(traj_idx: int, diff --git a/predicators/agent_sdk/bilevel_sketch.py b/predicators/agent_sdk/bilevel_sketch.py index 16124ca47..6feed94c5 100644 --- a/predicators/agent_sdk/bilevel_sketch.py +++ b/predicators/agent_sdk/bilevel_sketch.py @@ -28,7 +28,7 @@ strip_subgoal_annotations from predicators.agent_sdk.sketch_prompts import build_solve_prompt from predicators.agent_sdk.sketch_refinement import DeepestFailure, \ - InfoScorer, RefineOutcome, StepProbeSuggestion, \ + InfoScorer, RefineOutcome, StepProbeSuggestion, ground_step, \ refine_and_validate_report, refine_sketch, resolve_refine_timeout, \ sample_params, suggest_probes from predicators.agent_sdk.sketch_types import GroundSampler, SketchStep @@ -47,6 +47,7 @@ "format_plan_lines", "format_sketch_lines", "format_step_line", + "ground_step", "parse_atoms", "parse_region_annotations", "parse_sketch_from_text", diff --git a/predicators/agent_sdk/config.py b/predicators/agent_sdk/config.py index ce59c3255..df139dfd4 100644 --- a/predicators/agent_sdk/config.py +++ b/predicators/agent_sdk/config.py @@ -55,9 +55,8 @@ def from_cfg(cls) -> "SessionConfig": class RefinementConfig: """Plan-sketch refinement: search budgets, gates, and ground samplers. - Consumed at handler entry by ``refine_plan_sketch`` / - ``evaluate_option_plan`` (tools.py) and by the probe's ``refine`` - (belief_probe.py). + Consumed at handler entry by ``submit_plan`` (tools/testing.py) and + by the probe's ``refine`` (belief_probe.py). """ ground_samplers: bool refinement_timeout_per_step: float @@ -65,7 +64,6 @@ class RefinementConfig: max_samples_per_step: int check_subgoals: bool log_state: bool - refine_evaluator_attempts: int use_llm_initial_params: bool @classmethod @@ -80,8 +78,6 @@ def from_cfg(cls) -> "RefinementConfig": max_samples_per_step=CFG.agent_bilevel_max_samples_per_step, check_subgoals=CFG.agent_bilevel_check_subgoals, log_state=CFG.agent_bilevel_log_state, - refine_evaluator_attempts=( - CFG.agent_bilevel_refine_evaluator_attempts), use_llm_initial_params=CFG.agent_bilevel_use_llm_initial_params, ) @@ -90,9 +86,9 @@ def from_cfg(cls) -> "RefinementConfig": class ValidationConfig: """Capture-validation rollouts and the cross-attempt journal. - Consumed at handler entry by ``evaluate_option_plan`` (tools.py) and - the probe's ``run(trials=N)`` (belief_probe.py); ``use_journal`` - gates the ``record_journal`` tool. + Consumed at handler entry by ``submit_plan`` (tools.py) and the + probe's ``run(trials=N)`` (belief_probe.py); ``use_journal`` gates + the journal / attempt-log channel. """ rollouts: int rollouts_after_flaky: int @@ -120,36 +116,19 @@ def from_cfg(cls) -> "ValidationConfig": class ToolSurfaceConfig: """Which optional tools a session offers, and their surface knobs. - Consumed by the tool builders in tools.py (gates + descriptions - baked at build time), the proposal handlers (call-time gates), image - sizing, and the sandbox CLAUDE.md builder (sandbox_prompts.py). + Consumed by the tool builders (descriptions baked at build time) and + image sizing. """ - use_explore_python: bool - explore_python_keep_replaced_tools: bool use_base_simulator: bool - explore_python_call_timeout: float + python_call_timeout: float image_max_px: int - propose_types: bool - propose_predicates: bool - propose_processes: bool - propose_options: bool - propose_objects: bool @classmethod def from_cfg(cls) -> "ToolSurfaceConfig": """Read the tool-surface flags from the live ``CFG``.""" # Flags keep their names for experiment-yaml compatibility. return cls( - use_explore_python=CFG.agent_planner_use_explore_python, - explore_python_keep_replaced_tools=( - CFG.agent_planner_explore_python_keep_replaced_tools), use_base_simulator=CFG.agent_planner_use_base_simulator, - explore_python_call_timeout=( - CFG.agent_sdk_explore_python_call_timeout), + python_call_timeout=(CFG.agent_sdk_python_call_timeout), image_max_px=CFG.agent_sdk_image_max_px, - propose_types=CFG.agent_sdk_propose_types, - propose_predicates=CFG.agent_sdk_propose_predicates, - propose_processes=CFG.agent_sdk_propose_processes, - propose_options=CFG.agent_sdk_propose_options, - propose_objects=CFG.agent_sdk_propose_objects, ) diff --git a/predicators/agent_sdk/docker_agent_runner.py b/predicators/agent_sdk/docker_agent_runner.py index dd8875a0c..560795a27 100644 --- a/predicators/agent_sdk/docker_agent_runner.py +++ b/predicators/agent_sdk/docker_agent_runner.py @@ -120,7 +120,6 @@ def _report_result(entry: Dict[str, Any]) -> None: return { "responses": collected, - "iteration_proposals": ctx.iteration_proposals, } @@ -318,7 +317,6 @@ def main() -> None: "type": "error", "error": str(e) }], - "iteration_proposals": None, } # Save output diff --git a/predicators/agent_sdk/docker_sandbox.py b/predicators/agent_sdk/docker_sandbox.py index e18563b77..16607556a 100644 --- a/predicators/agent_sdk/docker_sandbox.py +++ b/predicators/agent_sdk/docker_sandbox.py @@ -310,35 +310,6 @@ def _stream_stderr() -> None: query_output = pkl.load(f_in) responses = query_output.get("responses", []) - proposals = query_output.get("iteration_proposals") - - # 6. Merge proposals back into host ToolContext - if proposals is not None: - logger.info( - "Docker proposals: proposed_options=%s, " - "retract=%s", - [o.name for o in proposals.proposed_options], - sorted(proposals.retract_option_names), - ) - self._tool_context.iteration_proposals = proposals - # Sync proposed/retracted options into ctx.options so - # the host-side parser can find them. - self._tool_context.options |= proposals.proposed_options - if proposals.retract_option_names: - self._tool_context.options = { - o - for o in self._tool_context.options - if o.name not in proposals.retract_option_names - } - logger.info( - "After Docker sync: tool_context.options=%s", - sorted(o.name for o in self._tool_context.options), - ) - else: - logger.warning( - "Docker output has iteration_proposals=None; " - "no proposals synced.") - # Track costs/turns via the base delta accounting. Each # docker query is a fresh in-container session whose # cumulative cost restarts from zero, so reset the delta diff --git a/predicators/agent_sdk/journal.py b/predicators/agent_sdk/journal.py index 05b6ed5b9..b5bc3293f 100644 --- a/predicators/agent_sdk/journal.py +++ b/predicators/agent_sdk/journal.py @@ -1,29 +1,35 @@ -"""Persistent per-run solve journal. - -One markdown file per run (``/journal.md``) that accumulates -knowledge across solve attempts and test tasks: the harness auto-records -each task's goal + initial state (one entry, at the top of the task's -section) and each attempt's outcome and captured or best refused plan, -and the agent records lessons via the ``record_journal`` MCP tool. -Fresh-context solve sessions read -the journal from their prompt, so knowledge travels through this curated -channel instead of raw transcript history (which also carries the wrong -conclusions of failed attempts - the anchoring failure mode). - -Entries are size-capped and the tool guidance asks for facts and -measurements rather than verdicts: a recorded "X is impossible" from a -failed attempt would re-import exactly the anchoring the fresh context -is meant to shed, while "tried yaws 0-15 deg at x in [0.50, 0.54], all -stopped >=5 cm short" steers the next attempt without foreclosing it. - -Phase lifecycle: learning-phase entries persist for the whole run and -accumulate across online-learning cycles, so every evaluation starts -from all learning knowledge so far. Test-phase entries live only for -their own evaluation: at ``end_test_phase`` the approach archives the -full journal to the run's log dir (outside the sandbox, so the agent -cannot read it) and rolls the file back to its pre-test content via -:func:`read_raw` / :func:`restore` - entries recorded while solving one -evaluation's test tasks must not leak into the next evaluation. +"""Persistent per-run solve journal and attempt log. + +Two markdown files in the sandbox that carry knowledge across solve +attempts, test tasks, and learning cycles: + +- ``journal.md`` is the AGENT's notebook. Solve and learn sessions + append to it with the ordinary file tools (no dedicated tool): short + factual entries - what was tried with exact parameters, what was + measured, what to try differently. The prompts ask for facts and + measurements rather than verdicts: a recorded "X is impossible" + from a failed attempt would re-import exactly the anchoring a fresh + context is meant to shed, while "tried yaws 0-15 deg at x in + [0.50, 0.54], all stopped >=5 cm short" steers the next attempt + without foreclosing it. +- ``attempts.md`` is the HARNESS's log, never edited by the agent: + each task's goal + initial state (once per task) and each + attempt's outcome and captured or best refused plan, so the + essentials of every attempt are on record even when the agent + writes nothing. + +Fresh-context solve sessions read both from their prompt (tail-capped +so recent attempts stay intact), so knowledge travels through these +curated channels instead of raw transcript history. + +Phase lifecycle: learning-phase content persists for the whole run +and accumulates across online-learning cycles, so every evaluation +starts from all learning knowledge so far. Test-phase additions live +only for their own evaluation: at ``end_test_phase`` the approach +archives both files to the run's log dir (outside the sandbox, so the +agent cannot read them) and rolls them back to their pre-test content +via :func:`read_raw` / :func:`restore` - entries written while +solving one evaluation's test tasks must not leak into the next. """ from __future__ import annotations @@ -32,21 +38,19 @@ from typing import Optional JOURNAL_FILENAME = "journal.md" - -# Per-entry cap. Entries are meant to be skimmable bullet lists; a cap -# keeps one verbose attempt from crowding every later prompt. -MAX_ENTRY_CHARS = 2000 - -# Harness auto-entries get more room: the first entry per task embeds -# the init-state feature dict (the prompt's own representation) and a -# captured plan. The writer additionally orders the layout block last, -# so tail truncation at this cap can only ever cut layout, never the -# outcome or the captured plan. -MAX_AUTO_ENTRY_CHARS = 4000 - -# Cap on how much journal is injected into a solve prompt. Tail-biased: -# recent attempts (usually the same task) matter most. -MAX_PROMPT_CHARS = 8000 +# The harness-owned attempt log (task contexts, attempt outcomes). +ATTEMPTS_FILENAME = "attempts.md" + +# Per-entry cap for harness attempt-log entries: the first entry per +# task embeds the init-state feature dict (the prompt's own +# representation) and a captured plan. The writer orders the layout +# block last, so tail truncation at this cap can only ever cut layout, +# never the outcome or the captured plan. +MAX_ENTRY_CHARS = 4000 +MAX_AUTO_ENTRY_CHARS = MAX_ENTRY_CHARS +# Cap on how much of each file is injected into a solve prompt. +# Tail-biased: recent attempts (usually the same task) matter most. +MAX_PROMPT_CHARS = 6000 # The learn-phase-maintained domain strategy document. Unlike the # append-only journal (facts and measurements), strategy.md is a LIVING @@ -66,6 +70,11 @@ def journal_path(sandbox_dir: str) -> str: return os.path.join(sandbox_dir, JOURNAL_FILENAME) +def attempts_path(sandbox_dir: str) -> str: + """Host path of the run's harness-owned attempt log.""" + return os.path.join(sandbox_dir, ATTEMPTS_FILENAME) + + def strategy_path(sandbox_dir: str) -> str: """Host path of the run's domain strategy document.""" return os.path.join(sandbox_dir, STRATEGY_FILENAME) @@ -96,8 +105,9 @@ def read_strategy(sandbox_dir: Optional[str], def append_entry(sandbox_dir: str, header: str, body: str, - max_chars: int = MAX_ENTRY_CHARS) -> Optional[str]: - """Append one entry; returns a truncation notice or None. + max_chars: int = MAX_ENTRY_CHARS, + filename: str = ATTEMPTS_FILENAME) -> Optional[str]: + """Append one harness entry; returns a truncation notice or None. ``header`` becomes a ``###
`` line; ``body`` is written verbatim below it, truncated at ``max_chars`` (default @@ -112,13 +122,14 @@ def append_entry(sandbox_dir: str, body += "\n[entry truncated at the per-entry size cap]" note = (f"entry truncated to {max_chars} chars - keep journal " "entries short and factual") - with open(journal_path(sandbox_dir), "a", encoding="utf-8") as f: + with open(os.path.join(sandbox_dir, filename), "a", encoding="utf-8") as f: f.write(f"### {header.strip()}\n{body}\n\n") return note -def read_raw(sandbox_dir: Optional[str]) -> Optional[str]: - """Exact journal file content, or None if no journal file exists. +def read_raw(sandbox_dir: Optional[str], + filename: str = JOURNAL_FILENAME) -> Optional[str]: + """Exact file content, or None if the file does not exist. Unlike :func:`read_journal` there is no prompt trimming and the absent-file case is distinguishable from an empty file, so the @@ -126,20 +137,22 @@ def read_raw(sandbox_dir: Optional[str]) -> Optional[str]: """ if not sandbox_dir: return None - path = journal_path(sandbox_dir) + path = os.path.join(sandbox_dir, filename) if not os.path.isfile(path): return None with open(path, "r", encoding="utf-8") as f: return f.read() -def restore(sandbox_dir: str, snapshot: Optional[str]) -> None: - """Reset the journal file to a :func:`read_raw` snapshot. +def restore(sandbox_dir: str, + snapshot: Optional[str], + filename: str = JOURNAL_FILENAME) -> None: + """Reset the file to a :func:`read_raw` snapshot. - A ``None`` snapshot means no journal file existed, so the file is - removed if present. + A ``None`` snapshot means the file did not exist, so it is removed + if present. """ - path = journal_path(sandbox_dir) + path = os.path.join(sandbox_dir, filename) if snapshot is None: if os.path.isfile(path): os.remove(path) @@ -150,15 +163,16 @@ def restore(sandbox_dir: str, snapshot: Optional[str]) -> None: def read_journal(sandbox_dir: Optional[str], - max_chars: int = MAX_PROMPT_CHARS) -> str: - """Journal content for prompt injection ('' if absent or empty). + max_chars: int = MAX_PROMPT_CHARS, + filename: str = JOURNAL_FILENAME) -> str: + """File content for prompt injection ('' if absent or empty). Over ``max_chars`` the head is dropped at an entry boundary with a truncation marker, keeping the most recent entries intact. """ if not sandbox_dir: return "" - path = journal_path(sandbox_dir) + path = os.path.join(sandbox_dir, filename) if not os.path.isfile(path): return "" with open(path, "r", encoding="utf-8") as f: diff --git a/predicators/agent_sdk/local_sandbox.py b/predicators/agent_sdk/local_sandbox.py index 38e65b45f..3b87e9397 100644 --- a/predicators/agent_sdk/local_sandbox.py +++ b/predicators/agent_sdk/local_sandbox.py @@ -162,7 +162,7 @@ async def query(self, await self.start_session() # Wall-clock backstop for the solve attempt deadline: the probe - # and explore_python enforce it cooperatively (tool calls refuse + # and run_python enforce it cooperatively (tool calls refuse # past the deadline), so normally the agent wraps up on its own; # interrupt only if the turn stream is still going long after. # The approach clears attempt_deadline before its final-submission @@ -190,20 +190,6 @@ async def _maybe_interrupt_on_deadline(_entry: Dict[str, Any]) -> None: kind=kind, on_entry=_maybe_interrupt_on_deadline) - # Log proposals (matches Docker sandbox logging) - proposals = self._tool_context.iteration_proposals - if proposals.proposed_options or proposals.retract_option_names: - logger.info( - "Local sandbox proposals: proposed_options=%s, " - "retract=%s", - [o.name for o in proposals.proposed_options], - sorted(proposals.retract_option_names), - ) - logger.info( - "After local sandbox query: tool_context.options=%s", - sorted(o.name for o in self._tool_context.options), - ) - return collected def _session_info_extras(self) -> Dict[str, Any]: diff --git a/predicators/agent_sdk/plan_execution.py b/predicators/agent_sdk/plan_execution.py index d5fe460ac..48bcdd146 100644 --- a/predicators/agent_sdk/plan_execution.py +++ b/predicators/agent_sdk/plan_execution.py @@ -127,7 +127,7 @@ def execute_plan_forward( """Execute a fully-grounded plan step by step through the option model. Shared forward-execution core behind ``validate_plan_forward`` (used - by ``refine_plan_sketch``) and the ``evaluate_option_plan`` tool. + by ``BeliefProbe.refine``) and the ``submit_plan`` tool. State carries forward across options — matching how the real env executes. Per step it mirrors ``run_backtracking_refinement``'s fixed-plan path: check ``initiable``, call diff --git a/predicators/agent_sdk/proposal_exec.py b/predicators/agent_sdk/proposal_exec.py index dbfb9037d..a67159925 100644 --- a/predicators/agent_sdk/proposal_exec.py +++ b/predicators/agent_sdk/proposal_exec.py @@ -1,29 +1,9 @@ """Safe execution and validation of agent-generated code proposals.""" import traceback -from dataclasses import dataclass, field from typing import Any, Callable, Dict, List, Optional, Sequence, Set, Tuple from predicators import utils -from predicators.structs import CausalProcess, ParameterizedOption, \ - Predicate, State, Task, Type - - -@dataclass -class ProposalBundle: - """Accumulates proposals made by the agent during a single iteration.""" - proposed_types: Set[Type] = field(default_factory=set) - proposed_predicates: Set[Predicate] = field(default_factory=set) - augment_task_fn: Optional[Callable[[Task], Task]] = None - augment_task_code: Optional[str] = None - proposed_processes: Set[CausalProcess] = field(default_factory=set) - proposed_options: Set[ParameterizedOption] = field(default_factory=set) - errors: List[str] = field(default_factory=list) - # Retractions: names of previously-proposed abstractions to remove - retract_type_names: Set[str] = field(default_factory=set) - retract_predicate_names: Set[str] = field(default_factory=set) - retract_task_augmentor: bool = False - retract_process_names: Set[str] = field(default_factory=set) - retract_option_names: Set[str] = field(default_factory=set) +from predicators.structs import ParameterizedOption, Predicate, State, Type def exec_code_safely(code: str, context: Dict[str, Any], @@ -89,7 +69,7 @@ def load_learned_samplers( ) -> Tuple[Dict[str, Any], List[str], Optional[str]]: """Exec sampler code and validate its ``LEARNED_SAMPLERS`` dict. - The single loader behind both the ``evaluate_sampler`` tool and + The single loader behind both ``sim.samplers()`` and ``SamplerLearningMixin._load_samplers_from_module_file``, so the two cannot drift. Keys must be known option names. """ diff --git a/predicators/agent_sdk/sandbox_prompts.py b/predicators/agent_sdk/sandbox_prompts.py index 17609573c..c98d916c9 100644 --- a/predicators/agent_sdk/sandbox_prompts.py +++ b/predicators/agent_sdk/sandbox_prompts.py @@ -50,25 +50,17 @@ Read ./session_logs/001_learn_*.md ## Scene Images -`evaluate_option_plan` automatically saves scene images to ./test_images/ +`submit_plan` automatically saves scene images to ./test_images/ after each step. You can Read them to inspect the spatial state of the environment. - -## Proposed Code -All proposal code and option source code is saved to ./proposed_code/. -Proposals are numbered (e.g. `001_propose_options_Pick.py`); saved -option source uses the option name (e.g. `Pick.py`): - - Glob ./proposed_code/*.py - Read ./proposed_code/001_propose_options_Pick.py """ _CLAUDE_MD_RULES = """\ ## Rules - Do NOT attempt to read or browse files outside the sandbox directory. - This is enforced for the file tools AND for Bash and the Python - execution tools (run_python / explore_python): commands or code + This is enforced for the file tools AND for Bash and the run_python + tool: commands or code containing absolute or `../` paths that leave the sandbox (or source introspection) are blocked. Use relative paths inside the sandbox. - Do NOT modify files in ./reference/ — they are for reading only @@ -89,7 +81,7 @@ - **Visualize liberally** — {visualize_hint} It's free (no physics, no failure modes). When stuck on a step, STOP testing and visualize the object at several candidate positions and orientations to find the - right region before spending more evaluate_option_plan calls. + right region before spending more submit_plan calls. - **Vary all parameters** — orientation and other non-position params affect both the outcome and whether the action succeeds. - **Search coarse-to-fine** — spread initial attempts across the full @@ -97,13 +89,10 @@ different region. """ -# The solve strategy's visualization pointer depends on whether the -# session has the probe: explore_python's sim.reset staging + -# sim.render overlays are the only visualization surface. -_VISUALIZE_HINT_PROBE = ("use explore_python (`sim.reset(mods={...})`, " +# The solve strategy's visualization pointer: run_python's +# sim.reset staging + sim.render overlays are the visualization surface. +_VISUALIZE_HINT_PROBE = ("use run_python (`sim.reset(mods={...})`, " "then `sim.render(...)`).") -_VISUALIZE_HINT_GENERIC = ("render candidate layouts with whatever " - "visualization your tools provide.") _CLAUDE_MD_SYNTHESIS_STRATEGY = """\ @@ -170,16 +159,11 @@ def build_claude_md(phase: Optional[str] = None) -> str: written into the sandbox so the agent reads phase-appropriate guidance every turn. """ - # pylint: disable-next=import-outside-toplevel - from predicators.agent_sdk.config import ToolSurfaceConfig if phase == "synthesis": strategy = _CLAUDE_MD_SYNTHESIS_STRATEGY else: - if ToolSurfaceConfig.from_cfg().use_explore_python: - hint = _VISUALIZE_HINT_PROBE - else: - hint = _VISUALIZE_HINT_GENERIC - strategy = _CLAUDE_MD_SOLVE_STRATEGY.format(visualize_hint=hint) + strategy = _CLAUDE_MD_SOLVE_STRATEGY.format( + visualize_hint=_VISUALIZE_HINT_PROBE) # The templates are authored as hard-wrapped markdown; render one # line per paragraph, consistent with the system prompts. return unwrap_prose_lines(_CLAUDE_MD_HEADER + strategy + _CLAUDE_MD_RULES) diff --git a/predicators/agent_sdk/sandbox_setup.py b/predicators/agent_sdk/sandbox_setup.py index 062b77eda..09144d402 100644 --- a/predicators/agent_sdk/sandbox_setup.py +++ b/predicators/agent_sdk/sandbox_setup.py @@ -196,7 +196,7 @@ def setup_sandbox_directory( - ``.claude/settings.json`` with PreToolUse hooks - ``.claude/validate_sandbox.py`` hook script - ``.git/`` marker so Claude CLI treats the sandbox as project root - - ``session_logs/``, ``test_images/``, ``proposed_code/`` subdirectories + - ``session_logs/``, ``test_images/`` subdirectories - ``full_system_prompt[_{phase}].md`` in *log_dir* for easy inspection Args: @@ -257,7 +257,7 @@ def setup_sandbox_directory( (sandbox / "CLAUDE.md").write_text(claude_md_content, encoding="utf-8") # 6. Create subdirectories and seed files - for subdir in ("session_logs", "test_images", "proposed_code"): + for subdir in ("session_logs", "test_images"): (sandbox / subdir).mkdir(exist_ok=True) # Seed empty scratchpad if enabled if seed_scratchpad: diff --git a/predicators/agent_sdk/sketch_prompts.py b/predicators/agent_sdk/sketch_prompts.py index 2ca644e46..697d97cce 100644 --- a/predicators/agent_sdk/sketch_prompts.py +++ b/predicators/agent_sdk/sketch_prompts.py @@ -28,6 +28,7 @@ def build_solve_prompt( ground_samplers: bool = False, journal: str = "", strategy: str = "", + attempts: str = "", physics_margin: bool = False, policy_mode: bool = False, ) -> str: @@ -52,7 +53,7 @@ def build_solve_prompt( ``[...]`` per step; the search refines them and samples on failure". ``require_tool_validation`` tells the agent it MUST submit a - goal-reaching ``evaluate_option_plan`` run on the current task (the + goal-reaching ``submit_plan`` run on the current task (the captured, validated plan is the only output) - used when the approach has no refinement fallback. When False, validation is merely encouraged. @@ -67,11 +68,11 @@ def build_solve_prompt( proving such a mechanism's absence instead of submitting the experiment that would let it be learned. - ``journal`` is the run's solve-journal content (see - ``predicators/agent_sdk/journal.py``): the curated record of earlier - attempts' outcomes and lessons, injected so fresh-context sessions - inherit what worked (and what was already swept) without inheriting - failed attempts' conclusions. + ``journal`` is the agent's own notebook (``journal.md``) and + ``attempts`` the harness's attempt log (``attempts.md``); see + ``predicators/agent_sdk/journal.py``. Both are injected so + fresh-context sessions inherit what worked (and what was already + swept) without inheriting failed attempts' conclusions. ``strategy`` is the learn-phase-maintained domain strategy document (``strategy.md``): the learn agent's best current natural-language @@ -81,8 +82,8 @@ def build_solve_prompt( re-verify rather than inherit. ``policy_mode`` (``CFG.agent_solve_policy_mode``): the deliverable - becomes a closed-loop ./policy.py validated via ``evaluate_policy`` - instead of a fixed evaluate_option_plan capture; the submit and + becomes a closed-loop ./policy.py validated via ``submit_policy`` + instead of a fixed submit_plan capture; the submit and closing guidance swap to the policy contract. ``physics_margin`` is the caller-threaded value of @@ -97,7 +98,7 @@ def build_solve_prompt( "explore_mode accepts an uncaptured experiment sketch, which " "contradicts the hard capture gate of require_tool_validation") assert not policy_mode or require_tool_validation, ( - "policy_mode is a hard capture gate (evaluate_policy), so it " + "policy_mode is a hard capture gate (submit_policy), so it " "requires require_tool_validation") init_state = task.init @@ -173,7 +174,7 @@ def build_solve_prompt( "not certify does not count. Once the belief model can " "validate a goal-reaching plan, submitting it (even " "unchanged) is how the loop concludes." + - (" A plan that passes evaluate_option_plan's validation " + (" A plan that passes submit_plan's validation " "gate (goal reached in every fresh belief rollout) is " "executed VERBATIM as this episode's solve attempt and " "replayed for the cycle's remaining episodes; only an " @@ -219,26 +220,18 @@ def build_solve_prompt( "sketch is a valid deliverable, and grinding for a validated " "plan the model cannot produce is wasted budget. An " "experiment's information comes from the steps the belief " - "model cannot predict. Before running, your sketch's " - "continuous parameters are refined in the belief model. " + - ("Every explicit parameter you propose executes exactly as " - "written (a step whose proposal fails a belief rollout is " - "retried, never re-sampled); refinement searches only the " - "steps you leave without parameters. " - if CFG.agent_explorer_pin_proposed_params else "") + - "When refinement cannot establish a step's annotated " - "subgoals the plan still runs in full: the refined prefix " - "executes as searched, the failing step executes its " - "closest attempt, and every LATER step falls back to the " - "explicit parameters you proposed for it - stopping only " - "at the first later step that has parameters to choose but " - "no proposal. So propose explicit parameters for every " - "step (any step after a model-refuted one runs exactly " - "those), and follow each uncertified step with a step " - "whose outcome reveals whether the mechanism worked - a " - "short plan that exercises the unknown beats a long one " - "that spends the episode's steps on what the model already " - "predicts.\n\n" + "model cannot predict. Your sketch runs in the real " + "environment EXACTLY as written: every explicit parameter " + "executes verbatim, nothing is searched or substituted, and " + "a step you leave without parameters gets one uniform draw " + "from the option's box - so propose explicit parameters for " + "every step. Validate in the belief model yourself where it " + "supports the plan (`sim.run`, `sim.refine`, then " + "`submit_plan` to submit), and follow each " + "uncertified step with a step whose outcome reveals whether " + "the mechanism worked - a short plan that exercises the " + "unknown beats a long one that spends the episode's steps " + "on what the model already predicts.\n\n" "Experiment design - one episode, many measurements. Before " "sketching, list the mechanisms the goal depends on and " "mark each KNOWN (the belief model has predicted it " @@ -257,14 +250,12 @@ def build_solve_prompt( "pinned this way is a cycle of drift-by-refit avoided " "later. When combining probes, annotate the subgoals of " "steps whose mechanism the belief model already CONTAINS " - "(annotations there drive boundary-probing refinement and " - "cost nothing). For a mechanism the model entirely LACKS, " - "annotate what SHOULD happen and give that step and every " - "step after it explicit parameters: refinement will fail " - "there and the rest of the plan runs on your proposed " - "parameters instead of searched ones - the annotation " - "documents the disagreement to measure, and your " - "parameters keep the later probes exactly as designed. " + "(annotations there let `sim.suggest_probes` rank probes and " + "let the execution monitor catch divergence, and cost " + "nothing). For a mechanism the model entirely LACKS, " + "annotate what SHOULD happen: the annotation documents the " + "disagreement to measure, and your explicit parameters keep " + "the later probes exactly as designed. " "Spend no steps re-demonstrating what the model already " "predicts well beyond what later probes need as setup.\n\n" "When the belief model has learned no dynamics at all yet " @@ -276,9 +267,9 @@ def build_solve_prompt( "sees each mechanism at least once, instead of spending the " "episode polishing a single goal attempt whose failure " "reveals only its first missing mechanism.\n\n" - "Ledger upkeep is part of the deliverable. Record " - "measurements with record_journal as you go (entries are " - "size-capped, so lead with the numbers), and when a result " + "Ledger upkeep is part of the deliverable. Append " + "measurements to ./journal.md as you go (a short entry per " + "experiment - lead with the numbers), and when a result " "settles an open question or opens a new one, edit " "open_questions.md directly with the file tools - the next " "learning phase designs its work from that file, and a " @@ -322,28 +313,39 @@ def build_solve_prompt( "depart from it whenever your own measurements disagree.\n\n" f"{strategy}\n") + attempts_section = "" + if attempts: + attempts_section = ( + "\n## Attempt Log (recorded by the harness)\n" + "Outcomes of earlier solve attempts and tasks in this run, " + "recorded automatically in ./attempts.md (do not edit it): " + "each task's goal and initial state, and per attempt the " + "outcome, the budget spent, and the captured or best refused " + "plan. Facts, not advice.\n\n" + f"{attempts}\n") journal_section = "" - if journal: + if journal or attempts: journal_section = ( - "\n## Solve Journal (record of earlier attempts)\n" - "You start with fresh context. The journal below is this run's " - "persistent record from earlier solve attempts and tasks: " - "auto-recorded outcomes (captured plans, rewards, budgets) " - "plus agent-recorded lessons. Use it - reproduce what worked, " - "do not repeat parameter sweeps it already covers - but treat " - "any recorded conclusion skeptically: re-verify cheap claims " - "rather than inheriting them, especially from failed " - "attempts.\n" + "\n## Solve Journal (./journal.md)\n" + "You start with fresh context. The journal is this run's " + "persistent notebook, written by earlier solve and learning " + "sessions with the file tools; with the attempt log it is " + "the record of what was tried. Use it - reproduce what " + "worked, do not repeat parameter sweeps it already covers - " + "but treat any recorded conclusion skeptically: re-verify " + "cheap claims rather than inheriting them, especially from " + "failed attempts.\n" "Journal protocol for this attempt:\n" - "- A design the journal records as having reached the goal in " - "the REAL environment is the INCUMBENT: reproduce it unless " - "the journal also records it failing since, or a model update " - "invalidates one of its steps. Every deviation from an " - "execution-validated design - reordering steps, dropping a " - "Wait, retargeting a parameter - is a NEW experiment carrying " - "first-execution risk that belief validation does NOT retire " - "(real option durations and placement scatter differ), so " - "deviate only for a recorded reason and record that reason.\n" + "- A design the attempt log records as having reached the " + "goal in the REAL environment is the INCUMBENT: reproduce it " + "unless the record also shows it failing since, or a model " + "update invalidates one of its steps. Every deviation from " + "an execution-validated design - reordering steps, dropping " + "a Wait, retargeting a parameter - is a NEW experiment " + "carrying first-execution risk that belief validation does " + "NOT retire (real option durations and placement scatter " + "differ), so deviate only for a recorded reason and record " + "that reason.\n" "- FIRST list the journal's untried leads, then execute or " "explicitly retire (with a measurement) each promising lead " "BEFORE re-opening a family an earlier attempt already marked " @@ -358,9 +360,13 @@ def build_solve_prompt( "recommends it), BOTH demote to open questions: design the " "cheap experiment that decides between them instead of " "silently trusting either.\n" - "Add your own lessons for future attempts with the " - "record_journal tool (facts and measurements only).\n\n" - f"{journal}\n") + "Add your own lessons for future attempts by appending a " + "short entry to ./journal.md with the file tools: a `### ` " + "header naming the task and attempt, then a few bullets of " + "facts and measurements only - exact parameters, what was " + "measured, what to try differently; no verdicts like " + "'impossible'.\n\n" + f"{journal or '(no journal entries yet)'}\n") goal_nl_section = "" if task.goal_nl: @@ -398,23 +404,18 @@ def build_solve_prompt( line += f" — {pred.natural_language_assertion(names)}" pred_strs.append(line) - # Tool-availability-aware references: when explore_python replaces - # the standalone refine tool (see - # agent_planner_explore_python_keep_replaced_tools), guidance must - # point at the probe equivalents instead of tools the session lacks. - # ``tool_names=None`` keeps the legacy all-tools wording. + # Tool-availability-aware references: guidance must not name a + # capability the session lacks (a simulator-free session has no + # probe). ``tool_names=None`` means the full surface. tool_set = set(tool_names) if tool_names is not None else None def _has_tool(name: str) -> bool: return tool_set is None or name in tool_set - probe_refine = (not _has_tool("refine_plan_sketch") - and _has_tool("explore_python")) - refine_ref = ("`sim.refine` (in `explore_python`)" - if probe_refine else "`refine_plan_sketch`") - if _has_tool("explore_python"): + refine_ref = "`sim.refine` (in `run_python`)" + if _has_tool("run_python"): visualize_advice = ( - "- Use `explore_python` (`sim.reset(mods={...})`, then " + "- Use `run_python` (`sim.reset(mods={...})`, then " "`sim.render(...)`) to move objects to candidate positions and " "orientations for free (no physics) and find the right region " "visually before testing.\n") @@ -428,7 +429,7 @@ def _has_tool(name: str) -> bool: # change the skeleton. deep_tune_advice = ( "deep-tune just that step (it needs precise values from you), then " - "re-test it. When deep-tuning a step with `evaluate_option_plan`:\n" + "re-test it. When deep-tuning a step with `submit_plan`:\n" "- Inspect the rendered images in `./test_images/` to see what " "actually happened.\n" "- For a failure like an IK error or collision, use the image and " @@ -528,7 +529,7 @@ def _has_tool(name: str) -> bool: "physical parameter - a design can pass just above and " "just below a value and fail exactly at it - so tune " "designs that pass the WHOLE range: pre-check with " - "`sim.run(plan_text, physics_sweep=True)` in explore_python " + "`sim.run(plan_text, physics_sweep=True)` in run_python " "(same points as the gate, one deterministic rollout each) " "instead of discovering rejections one submission at a " "time. ") @@ -551,17 +552,17 @@ def _has_tool(name: str) -> bool: "operating point to a learned threshold) and widen it if " "it is smaller than the measured execution scatter. ") submit_guidance = ( - "SUBMIT via `evaluate_option_plan`: pass your full plan as text " + "SUBMIT via `submit_plan`: pass your full plan as text " "(one option per line, `Option(obj:type)[params] -> {subgoals}`, " "with EXACT params) and run it on the CURRENT task (omit " "task_idx). When it reaches the goal, that plan is captured as " - "your answer, so do NOT finish until evaluate_option_plan " + "your answer, so do NOT finish until submit_plan " "CONFIRMS the capture. A goal-reaching plan is re-run several " "times before capture (simulation varies across runs; each " "rollout reports the motion-planner seed it ran at); if it is " "reported FLAKY, reproduce the failed rollout exactly (pass " - "its reported seed as rollout_seed to evaluate_option_plan, or " - "`sim.run(plan_text, seed=...)` in explore_python) to see WHY, " + "its reported seed as rollout_seed to submit_plan, or " + "`sim.run(plan_text, seed=...)` in run_python) to see WHY, " "then add margin to the fragile step and resubmit. For a plan " "you suspect is marginal, request a stricter gate up front " "with validation_rollouts=N (more repeats; never fewer than " @@ -584,8 +585,8 @@ def _has_tool(name: str) -> bool: "It runs your EXACT parameters with no sampling. To find " f"working parameters you MAY use {refine_ref} (it searches " "but is slower); read the parameters it reports and submit them " - "via evaluate_option_plan. If a step does not reach its subgoal, " - + stuck_advice) + "via submit_plan. If a step does not reach its subgoal, " + + stuck_advice) elif propose_params: submit_guidance = ( f"You may validate with {refine_ref} (it tries your " @@ -607,7 +608,7 @@ def _has_tool(name: str) -> bool: " def get_option(state, memory):\n" " ...\n\n" "- `state`: the current State object (read-only copy). Same " - "API as explore_python: `state.get(obj, 'feature')`, iterate " + "API as run_python: `state.get(obj, 'feature')`, iterate " "objects with `for obj in state`, `obj.name`, `obj.type`.\n" "- `memory`: a dict, initially empty, persisting across calls " "within ONE episode (phase flags, counters, cached " @@ -641,18 +642,18 @@ def _has_tool(name: str) -> bool: f"{CFG.agent_policy_max_repeated_noops} times in a row also " "ends the episode - if your stage logic is not advancing, " "fix the stage test, do not re-send the same command.\n" - "SUBMIT via `evaluate_policy` on the CURRENT task until it " + "SUBMIT via `submit_policy` on the CURRENT task until it " "reaches the goal across all validation rollouts - the " "validated policy.py snapshot (taken at call time; later " "edits need a new call) is your ONLY accepted output. Test " - "recovery behavior first: in explore_python, " + "recovery behavior first: in run_python, " "`sim.run_policy()` runs ./policy.py from the CURRENT probe " "state (including perturbed or mid-plan states), so check " "that the policy recovers from off-nominal states, not just " "the initial one. " + margin_guidance) closing_block = ( "Your answer is ONLY accepted from a goal-reaching " - "`evaluate_policy` run on the CURRENT task; final text alone " + "`submit_policy` run on the CURRENT task; final text alone " "is discarded, so never finish without that validated run. " "After the goal-reaching run, summarize the policy's strategy " "as your final text.") @@ -664,7 +665,7 @@ def _has_tool(name: str) -> bool: # text sketch and finish, wasting the whole attempt. closing_block = ( "Your answer is ONLY accepted from a goal-reaching " - "`evaluate_option_plan` run on the CURRENT task; final text " + "`submit_plan` run on the CURRENT task; final text " "alone is discarded, so never finish without that validated " "run. Tool calls are permitted on every turn of this " "conversation. If an earlier context summary says a turn was " @@ -714,7 +715,8 @@ def _has_tool(name: str) -> bool: ## Available Predicates (for subgoal annotations) {chr(10).join(pred_strs)} -{trajectory_summary}{tools_str}{strategy_section}{journal_section}\ +{trajectory_summary}{tools_str}{strategy_section}{attempts_section}\ +{journal_section}\ {scheduled_plans_section} ## Instructions Use your available tools to inspect the environment before producing the plan. diff --git a/predicators/agent_sdk/sketch_refinement.py b/predicators/agent_sdk/sketch_refinement.py index 6853d4c04..99d169f31 100644 --- a/predicators/agent_sdk/sketch_refinement.py +++ b/predicators/agent_sdk/sketch_refinement.py @@ -292,7 +292,7 @@ def _draw_params(search: _RefinementState, ctx: _RefineContext, return sample_params(step.option, rng_) -def _ground(step: SketchStep, params: np.ndarray) -> _Option: +def ground_step(step: SketchStep, params: np.ndarray) -> _Option: """Ground a step's option with the given params. Wait steps inject ``wait_target_atoms`` / ``wait_target_neg_atoms`` @@ -321,12 +321,12 @@ def ground_seeded_step(step: SketchStep) -> Optional[_Option]: """ box = step.option.params_space if box.shape[0] == 0: - return _ground(step, np.array([], dtype=np.float32)) + return ground_step(step, np.array([], dtype=np.float32)) if step.initial_params is None: return None params = np.clip(np.asarray(step.initial_params, dtype=np.float32), box.low, box.high).astype(np.float32) - return _ground(step, params) + return ground_step(step, params) def _info_seeking_applies(ctx: _RefineContext, step: SketchStep) -> bool: @@ -479,7 +479,7 @@ def _consider(grounded: _Option) -> None: idx not in search.llm_params_tried: search.llm_params_tried.add(idx) box = step.option.params_space - llm_grounded = _ground( + llm_grounded = ground_step( step, np.clip(np.asarray(step.initial_params, dtype=np.float32), box.low, box.high).astype(np.float32)) @@ -493,7 +493,8 @@ def _consider(grounded: _Option) -> None: if len(scored) > n_pooled_before else "infeasible — not pooled") while len(scored) < ctx.info_n_feasible_target and n_draws < draw_cap: - grounded = _ground(step, _draw_params(search, ctx, step, state, rng_)) + grounded = ground_step(step, + _draw_params(search, ctx, step, state, rng_)) n_draws += 1 _consider(grounded) pool.spent += n_draws @@ -559,7 +560,7 @@ def _sample_step(search: _RefinementState, ctx: _RefineContext, idx: int, box = step.option.params_space params = np.clip(np.asarray(step.initial_params, dtype=np.float32), box.low, box.high).astype(np.float32) - return _ground(step, params) + return ground_step(step, params) # Plain path: on the first arrival at this step, try the LLM-proposed # params (if any) before any sampling. Clipping avoids ground()'s # out-of-box ValueError; arity is already validated by the parser. @@ -571,8 +572,8 @@ def _sample_step(search: _RefinementState, ctx: _RefineContext, idx: int, box.low, box.high).astype(np.float32) logging.debug("[%s] step %d %s: trying LLM-proposed params %s", ctx.run_id, idx, step.option.name, params.tolist()) - return _ground(step, params) - return _ground(step, _draw_params(search, ctx, step, state, rng_)) + return ground_step(step, params) + return ground_step(step, _draw_params(search, ctx, step, state, rng_)) def _validate_step(search: _RefinementState, ctx: _RefineContext, idx: int, @@ -1108,7 +1109,7 @@ def _roll(grounded: _Option) -> Optional[State]: np.asarray(step.initial_params, dtype=np.float32), box.low, box.high).astype(np.float32) if step.initial_params is not None else np.array([], dtype=np.float32)) - nominal = _ground(step, params) + nominal = ground_step(step, params) nominal_next = _roll(nominal) if nominal is not None else None nominal_ok: Optional[bool] = None nominal_score: Optional[float] = None @@ -1127,8 +1128,8 @@ def _roll(grounded: _Option) -> Optional[State]: best_next: Optional[State] = None if has_params and atoms: for _ in range(max_draws): - grounded = _ground(step, - _draw_params(search, ctx, step, state, rng)) + grounded = ground_step( + step, _draw_params(search, ctx, step, state, rng)) n_draws += 1 nxt = _roll(grounded) if nxt is None or not atoms.issubset( diff --git a/predicators/agent_sdk/synthesis_backend.py b/predicators/agent_sdk/synthesis_backend.py index d16fe6fb1..7a0e77553 100644 --- a/predicators/agent_sdk/synthesis_backend.py +++ b/predicators/agent_sdk/synthesis_backend.py @@ -2,8 +2,8 @@ :class:`SynthesisBackend` declares exactly the approach surface that the synthesis tool factories in :mod:`predicators.agent_sdk.tools` -(``create_synthesis_tools``, ``create_predicate_synthesis_tools``, -``create_sampler_synthesis_tools``) and the approach-layer validation +(``create_synthesis_tools``, ``make_predicate_quality_loader``, +``make_sampler_loader``) and the approach-layer validation glue in :mod:`predicators.approaches.synthesis_validation` dereference. It exists so those modules can be typed against the contract instead of importing the concrete ``AgentSimLearningApproach`` - the import that @@ -64,8 +64,15 @@ class SynthesisBackend(Protocol): _residual_rules: Optional[List] _latent_init: Any - def _publish_probe_fit(self, params: Dict[str, float], version_tag: str, - simulator_file: str) -> None: + def _publish_probe_fit( + self, + params: Dict[str, float], + version_tag: str, + simulator_file: str, + fit_result: Optional[FitResult] = None, + sse: float = float("nan"), + applied_physical: Optional[Dict[str, float]] = None, + ) -> None: """Deploy a canonical ``sim.fit`` result to the candidate probe.""" # ── Vocabulary / engine accessors ──────────────────────────── @@ -142,7 +149,7 @@ def materialise_latent( class PredicateSynthesisBackend(SynthesisBackend, Protocol): - """The extra surface ``create_predicate_synthesis_tools`` needs. + """The extra surface ``make_predicate_quality_loader`` needs. Only the predicate-invention subclass provides these, so they live off the core protocol. @@ -157,7 +164,7 @@ class PredicateSynthesisBackend(SynthesisBackend, Protocol): class SamplerSynthesisBackend(Protocol): - """The narrow surface ``create_sampler_synthesis_tools`` needs. + """The narrow surface ``make_sampler_loader`` needs. ``SamplerLearningMixin`` satisfies this directly (its declared host- class contract covers every member), so the mixin can pass ``self`` diff --git a/predicators/agent_sdk/tools/__init__.py b/predicators/agent_sdk/tools/__init__.py index 28edd0261..6bdd8e555 100644 --- a/predicators/agent_sdk/tools/__init__.py +++ b/predicators/agent_sdk/tools/__init__.py @@ -9,14 +9,16 @@ - ``budget``: solve-attempt budget footer and watchdog. - ``scene``: scene rendering and state-manipulation helpers. - ``verdicts``: task-evaluator verdicts and ground-sampler loading. -- ``inspection`` / ``proposals`` / ``testing`` / ``planning`` / - ``exploration`` / ``journal_tools``: the static MCP tool builders, - assembled by ``assembly.create_mcp_tools``. +- ``testing`` / ``exploration``: the static MCP + tool builders, assembled by ``assembly.create_mcp_tools``. +- ``digests``: the type / option / task / trajectory digest renderers + shared by the prompts and the probe. - ``snapshots``: versioned write-time snapshots of agent-edited files. - ``python_exec``: shared python-exec core (run_python / - explore_python). -- ``synthesis`` / ``params_view`` / ``predicate_synthesis`` / - ``sampler_synthesis``: synthesis-session tool factories. + run_python). +- ``synthesis`` / ``params_view``: the synthesis-session tool factory. +- ``predicate_synthesis`` / ``sampler_synthesis``: the loaders behind + ``sim.predicates()`` / ``sim.samplers()``. This facade re-exports the package's public surface (plus a few underscore names kept for pre-split imports); new code should import @@ -27,19 +29,14 @@ from predicators.agent_sdk.tools.context import PlanCapture, ToolContext from predicators.agent_sdk.tools.params_view import _ParamsView from predicators.agent_sdk.tools.predicate_synthesis import \ - create_predicate_synthesis_tools + make_predicate_quality_loader from predicators.agent_sdk.tools.registry import ALL_TOOL_NAMES, \ - BUILTIN_TOOLS, EXPLORATION_TOOL_NAMES, INSPECTION_TOOL_NAMES, \ - JOURNAL_TOOL_NAMES, MCP_SERVER_NAME, PLANNING_TOOL_NAMES, \ - PREDICATE_SYNTHESIS_TOOL_NAMES, PROPOSAL_TOOL_NAMES, \ - RETRACTION_TOOL_NAMES, SAMPLER_SYNTHESIS_TOOL_NAMES, \ - SYNTHESIS_TOOL_NAMES, TESTING_TOOL_NAMES, explore_python_replaces_tools, \ - get_allowed_tool_list, list_session_tool_names + BUILTIN_TOOLS, EXPLORATION_TOOL_NAMES, MCP_SERVER_NAME, \ + SYNTHESIS_TOOL_NAMES, TESTING_TOOL_NAMES, get_allowed_tool_list, \ + list_session_tool_names from predicators.agent_sdk.tools.results import _make_coercing_tool, \ - _make_spilling_text_result, _save_option_to_sandbox, \ - session_log_filename -from predicators.agent_sdk.tools.sampler_synthesis import \ - create_sampler_synthesis_tools + _make_spilling_text_result, session_log_filename +from predicators.agent_sdk.tools.sampler_synthesis import make_sampler_loader from predicators.agent_sdk.tools.sandbox_guard import \ SANDBOX_HIDDEN_MODULES_PATTERN, SANDBOX_INTROSPECTION, \ SANDBOX_SYSTEM_ROOTS, _screen_text_for_sandbox_escape @@ -56,14 +53,7 @@ "ALL_TOOL_NAMES", "BUILTIN_TOOLS", "EXPLORATION_TOOL_NAMES", - "INSPECTION_TOOL_NAMES", - "JOURNAL_TOOL_NAMES", "MCP_SERVER_NAME", - "PLANNING_TOOL_NAMES", - "PREDICATE_SYNTHESIS_TOOL_NAMES", - "PROPOSAL_TOOL_NAMES", - "RETRACTION_TOOL_NAMES", - "SAMPLER_SYNTHESIS_TOOL_NAMES", "SANDBOX_HIDDEN_MODULES_PATTERN", "SANDBOX_INTROSPECTION", "SANDBOX_SYSTEM_ROOTS", @@ -74,17 +64,16 @@ "agent_render_resolution", "apply_state_modifications", "create_mcp_tools", - "create_predicate_synthesis_tools", - "create_sampler_synthesis_tools", "create_synthesis_tools", "draw_pybullet_annotation", "evaluate_states_with", - "explore_python_replaces_tools", "finalize_versioned_snapshot", "format_object_poses", "get_allowed_tool_list", "list_session_tool_names", "load_ground_sampler_fns", + "make_predicate_quality_loader", + "make_sampler_loader", "make_solved_check", "make_write_snapshot_hook", "render_pybullet_image", diff --git a/predicators/agent_sdk/tools/assembly.py b/predicators/agent_sdk/tools/assembly.py index 6d4d21670..29d8ce311 100644 --- a/predicators/agent_sdk/tools/assembly.py +++ b/predicators/agent_sdk/tools/assembly.py @@ -3,11 +3,6 @@ from predicators.agent_sdk.tools.context import ToolContext from predicators.agent_sdk.tools.exploration import _build_exploration_tools -from predicators.agent_sdk.tools.inspection import _build_inspection_tools -from predicators.agent_sdk.tools.journal_tools import _build_journal_tools -from predicators.agent_sdk.tools.planning import _build_planning_tools -from predicators.agent_sdk.tools.proposals import _build_proposal_tools, \ - _build_retraction_tools from predicators.agent_sdk.tools.results import _make_coercing_tool, \ _make_spilling_text_result from predicators.agent_sdk.tools.testing import _build_testing_tools @@ -34,18 +29,21 @@ def create_mcp_tools(ctx: ToolContext, # routes through the spiller with no call-site edits. _text_result = _make_spilling_text_result(ctx.sandbox_dir) + # A session-specific instance attached to ``ctx.extra_mcp_tools`` + # wins over the static builder of the same name: synthesis sessions + # attach their own ``run_python`` (fit data + candidate-simulator + # probe in one namespace), so the solve-phase instance is neither + # built nor offered there. + extra_names = {getattr(t, "name", "") for t in ctx.extra_mcp_tools} _all = { - **_build_inspection_tools(ctx, _text_result, tool), - **_build_proposal_tools(ctx, _text_result, tool), - **_build_retraction_tools(ctx, _text_result, tool), **_build_testing_tools(ctx, _text_result, tool), - **_build_planning_tools(ctx, _text_result, tool), - **_build_exploration_tools(ctx, _text_result, tool), - **_build_journal_tools(ctx, _text_result, tool), + **({} if "run_python" in extra_names else _build_exploration_tools( + ctx, _text_result, tool)), } if tool_names is None: tools = list(_all.values()) else: tools = [_all[n] for n in tool_names if n in _all] + tools = [t for t in tools if getattr(t, "name", "") not in extra_names] tools.extend(ctx.extra_mcp_tools) return tools diff --git a/predicators/agent_sdk/tools/budget.py b/predicators/agent_sdk/tools/budget.py index 397de782d..7540a32a0 100644 --- a/predicators/agent_sdk/tools/budget.py +++ b/predicators/agent_sdk/tools/budget.py @@ -122,7 +122,7 @@ def _arm_budget_watchdog(seconds: float) -> Callable[[], None]: """Schedule a ProbeBudgetExceeded in the CALLING thread after ``seconds``; returns an idempotent disarm callable. - ``explore_python``'s exec() runs on the event-loop thread, so + ``run_python``'s exec() runs on the event-loop thread, so pure-Python code that never reaches a probe checkpoint blocks every cooperative deadline check AND the sandbox's message-stream interrupt backstop - an async exception from a watchdog timer is the diff --git a/predicators/agent_sdk/tools/capture.py b/predicators/agent_sdk/tools/capture.py index a7d256526..70c323ea2 100644 --- a/predicators/agent_sdk/tools/capture.py +++ b/predicators/agent_sdk/tools/capture.py @@ -1,4 +1,4 @@ -"""The ``evaluate_option_plan`` capture decision, as a pure function. +"""The ``submit_plan`` capture decision, as a pure function. :func:`_decide_capture` encodes the run-verified capture gates in one side-effect-free place: the handler in ``testing.py`` computes the @@ -13,7 +13,7 @@ class CaptureDecision(enum.Enum): - """What ``evaluate_option_plan`` does with the submitted plan.""" + """What ``submit_plan`` does with the submitted plan.""" # Goal reached, evaluator-certified, every validation rollout passed: # captured and marked as a validated solve. VALIDATED_CAPTURE = "validated_capture" @@ -71,7 +71,7 @@ def _decide_capture(*, best_effort_mode: bool, have_validated_capture: bool, param_sensitive: bool = False) -> CaptureOutcome: - """Decide what ``evaluate_option_plan`` does with an evaluated plan. + """Decide what ``submit_plan`` does with an evaluated plan. Pure: no ctx access, no I/O - the caller supplies exactly what the gates read and applies the side effects the decision calls for. diff --git a/predicators/agent_sdk/tools/context.py b/predicators/agent_sdk/tools/context.py index f820652d0..702e4c6b9 100644 --- a/predicators/agent_sdk/tools/context.py +++ b/predicators/agent_sdk/tools/context.py @@ -4,7 +4,6 @@ from dataclasses import dataclass, field from typing import Any, Callable, Dict, Iterator, List, Optional, Set -from predicators.agent_sdk.proposal_exec import ProposalBundle from predicators.option_model import _OptionModelBase from predicators.settings import CFG from predicators.structs import CausalProcess, LowLevelTrajectory, \ @@ -43,7 +42,7 @@ class ToolContext: online_trajectories: List[LowLevelTrajectory] = field(default_factory=list) example_state: Optional[State] = None option_model: Optional[_OptionModelBase] = None - # Synthesis-session override for the explore_python probe: a lazy + # Synthesis-session override for the run_python probe: a lazy # builder over the CANDIDATE simulator.py (fresh MCMC fit, cached # until the file changes). When set, BeliefProbe executes against it # instead of ``option_model`` - which during synthesis is the stale @@ -60,6 +59,14 @@ class ToolContext: # sessions - the deployed belief model is fixed there, so the probe # rejects ``fit`` calls. probe_fit_provider: Optional[Callable[..., str]] = None + # Synthesis-session loaders behind ``sim.predicates()`` and + # ``sim.samplers()``: each reloads the agent-authored file fresh + # (predicates.py / samplers.py), installs the result into the + # approach so refinement sees the draft, and returns the report + # text. Empty in sessions that do not offer the artifact. + probe_artifact_loaders: Dict[str, + Callable[..., + str]] = field(default_factory=dict) # Synthesis sessions: which parameter values the candidate probe # model is running with - "fitted ()" after a canonical # ``sim.fit`` of the current simulator.py, or an UNFITTED notice @@ -86,9 +93,6 @@ class ToolContext: parameterized_samplers: Dict[str, ParameterizedSampler] = field( default_factory=dict) current_task: Optional[Task] = None - iteration_proposals: ProposalBundle = field(default_factory=ProposalBundle) - planning_results: Dict[str, Any] = field(default_factory=dict) - iteration_history: List[Dict[str, Any]] = field(default_factory=list) skill_factory_context: Dict[str, Any] = field(default_factory=dict) proposals_disabled: bool = False # set True during test-time solving log_dir: Optional[str] = None @@ -103,11 +107,11 @@ class ToolContext: # main.py's ``test_task_idx``. None outside the test phase. Threaded into # the saved session-log filename so test queries are attributable to a task. test_task_idx: Optional[int] = None - test_call_id: int = 0 # incremented per evaluate_option_plan call + test_call_id: int = 0 # incremented per submit_plan call # 0-based learning cycle (matching main.py's "ONLINE LEARNING CYCLE i"; # -1 = the offline pass) while a synthesis (learn) session is active, # None otherwise. Set/cleared around the synthesis query so tools that - # label output by phase (e.g. record_journal headers) can attribute + # label output by phase (e.g. attempt-log headers) can attribute # entries to the learning cycle instead of "pre-test phase". learn_cycle_index: Optional[int] = None # Managed by AgentSessionMixin: populated from @@ -160,7 +164,7 @@ class ToolContext: # so the next exploration targets the gaps. None ⇒ no fit ran yet # (or it had no weak spots). sysid_diagnostics: Optional[str] = None - # Set by refine_plan_sketch / evaluate_option_plan when a plan is verified + # Set by submit_plan / submit_policy when a plan is verified # to reach the goal on the CURRENT solve task: the simulator-verified plan # (grounded options with found params) and the parallel subgoal sketch. # The bilevel approach returns this directly instead of re-refining, so @@ -175,11 +179,11 @@ class ToolContext: solved_plan_reached_goal: Optional[bool] = None # Gate for the above: only approaches that consume captured plans # (AgentModelBasedApproach) set this True. Keeps the open-loop - # planner, which also uses evaluate_option_plan, from recording + # planner, which also uses submit_plan, from recording # spurious captures. capture_goal_reaching_plans: bool = False # Set (with capture_goal_reaching_plans) only for the final-submission - # nudge after an attempt exhausted its turn budget: evaluate_option_plan + # nudge after an attempt exhausted its turn budget: submit_plan # then captures the agent's submitted plan on the current task even if it # does not reach the goal, is scored a non-solve by the task evaluator, # or is flaky, so the approach executes the best-effort plan (for its @@ -205,7 +209,7 @@ class ToolContext: # ascending; empty when no fit with nonzero posterior width is # deployed). A callable rather than a stored list so the points # always track the LATEST applied fit. Installed by - # AgentSimLearningApproach; consumed by evaluate_option_plan under + # AgentSimLearningApproach; consumed by submit_plan under # agent_plan_validation_physics_margin and by the sim.run physics # sweep. physics_margin_provider: Optional[Callable[[], List[Dict[str, @@ -231,7 +235,7 @@ class ToolContext: rule_param_override_scope: Optional[Callable[[Dict[str, float]], Any]] = None # Capture-task keys (see ``_capture_task_key``) that have produced a - # FLAKY rejection in evaluate_option_plan. A flaky submission is direct + # FLAKY rejection in submit_plan. A flaky submission is direct # evidence the agent is tuning in a marginal region where a lucky # streak can pass the base rollout gate (run_20260717_182321: a # 20/20-swept placement validated 3/3, then failed the real episode), @@ -248,19 +252,19 @@ class ToolContext: # auto-entry. Cleared together with solved_plan. solved_plan_validation_summary: Optional[str] = None # Closed-loop policy mode (CFG.agent_solve_policy_mode): the captured - # policy.py source, SNAPSHOTTED at evaluate_policy call time so a + # policy.py source, SNAPSHOTTED at submit_policy call time so a # later edit of the file cannot swap unvalidated code into the # executed artifact. Mutually exclusive with solved_plan; cleared # together with it. solved_policy_source: Optional[str] = None # True while the current solve attempt's deliverable is a policy: - # evaluate_option_plan keeps its probing role but its CAPTURE gate is - # disabled, and evaluate_policy requires it. Set by _solve_attempt. + # submit_plan keeps its probing role but its CAPTURE gate is + # disabled, and submit_policy requires it. Set by _solve_attempt. policy_capture_mode: bool = False # Restart-loop attempt bookkeeping, set by AgentModelBasedApproach._solve # around each attempt. ``attempt_start``/``attempt_deadline`` are # time.monotonic() values; the deadline is enforced cooperatively by - # the probe (every sim call) and explore_python, and surfaced in tool + # the probe (every sim call) and run_python, and surfaced in tool # results as a budget footer. None ⇒ no attempt in flight / no wall # clock. The deadline is cleared before the final-submission nudge so # nothing blocks the submission itself. @@ -272,17 +276,17 @@ class ToolContext: # the budget footer so sweeps carry a visible price. attempt_rollout_count: int = 0 # Best submission on the current task this attempt that - # evaluate_option_plan evaluated but refused to capture (evaluator + # submit_plan evaluated but refused to capture (evaluator # scored it a non-solve, or it was flaky), ranked by evaluator # reward. Reset per attempt; the journal auto-entry records it so a # later attempt (or the final best-effort nudge) can resubmit it # instead of the attempt's work vanishing with its context. best_uncaptured_plan_lines: Optional[List[str]] = None best_uncaptured_reward: Optional[float] = None - # Per-call deadline for the explore_python call currently executing - # (agent_sdk_explore_python_call_timeout); enforced at the same + # Per-call deadline for the run_python call currently executing + # (agent_sdk_python_call_timeout); enforced at the same # probe checkpoints as attempt_deadline. None ⇒ no call in flight. - explore_call_deadline: Optional[float] = None + python_call_deadline: Optional[float] = None def begin_attempt(self, index: int, wall_clock: float) -> None: """Start restart-loop bookkeeping for solve attempt ``index``. diff --git a/predicators/agent_sdk/tools/digests.py b/predicators/agent_sdk/tools/digests.py new file mode 100644 index 000000000..df1d3bd4a --- /dev/null +++ b/predicators/agent_sdk/tools/digests.py @@ -0,0 +1,147 @@ +"""Digest renderers over ToolContext state. + +The single formatting source for the type / option / task / trajectory +summaries: the solve and synthesis prompt builders inject them directly, +and the probe serves the same text from ``sim.task()`` and +``describe_trajectory`` (so the wording cannot drift between surfaces). +""" +from typing import Any, Collection, Iterable, List, Optional, Union + +from predicators import utils + + +def render_types_digest(types: Iterable[Any]) -> str: + """One line per type: name, parent, feature names.""" + lines = [] + for t in sorted(types, key=lambda t: t.name): + features = ", ".join( + t.feature_names) if t.feature_names else "(no features)" + parent_str = f" (parent: {t.parent.name})" if t.parent else "" + lines.append(f"- {t.name}{parent_str}: [{features}]") + if not lines: + return "No types defined." + return "\n".join(lines) + + +def render_options_digest(options: Iterable[Any], + gt_options_ref_path: Optional[str] = None) -> str: + """One line per option: typed signature plus parameter box/descriptions. + + The ``obj:type`` signature and parameter listing here are what the + strict plan parser expects plans to match, so this exact rendering + is shared by every prompt surface. + """ + lines = [] + for opt in sorted(options, key=lambda o: o.name): + type_sig = ", ".join(t.name for t in opt.types) + params_dim = opt.params_space.shape[0] if opt.params_space.shape else 0 + if params_dim > 0: + low = opt.params_space.low.tolist() + high = opt.params_space.high.tolist() + if opt.params_description: + desc = ", ".join(opt.params_description) + param_info = f", params=[{desc}], low={low}, high={high}" + else: + param_info = (f", params_dim={params_dim}, " + f"low={low}, high={high}") + else: + param_info = "" + lines.append(f" {opt.name}({type_sig}{param_info})") + if not lines: + return "No options defined." + if gt_options_ref_path: + lines.append(f"\nOption definition source code: " + f"`{gt_options_ref_path}` (search for the option name).") + return "\n".join(lines) + + +def render_task_digest(task: Any, + task_idx: Union[int, str], + predicates: Collection[Any], + include_goal_query_hint: bool = False) -> str: + """Goal (NL preferred), initial atoms, objects, and init-state details for + one task. + + ``task_idx`` is the header label; a string (e.g. ``"(current solve + task)"``) is allowed for tasks outside the train list. + ``include_goal_query_hint`` adds the ``is_goal_state`` / + ``goal_holds`` pointer, which only makes sense (and is only + numerically valid) in sessions whose exec namespace binds those + names (synthesis ``run_python``), so pass it only with an integer + ``task_idx``. + """ + if task.goal_nl: + goal_line = f" Goal (natural language): {task.goal_nl}" + else: + goal_str = ", ".join(str(g) for g in sorted(task.goal)) + goal_line = f" Goal: {{{goal_str}}}" + init_atoms = utils.abstract(task.init, predicates) + atoms_str = ", ".join(str(a) for a in sorted(init_atoms)) + objects = sorted(task.init, key=str) + obj_str = ", ".join(f"{o.name}:{o.type.name}" for o in objects) + state_str = task.init.pretty_str() + hint_line = "" + if include_goal_query_hint: + hint_line = (f" Goal achievement: query " + f"`is_goal_state(state, {task_idx})` or " + f"`train_tasks[{task_idx}].goal_holds(state)`.\n") + return (f"Task {task_idx}:\n" + f"{goal_line}\n" + f"{hint_line}" + f" Initial atoms: {{{atoms_str}}}\n" + f" Objects: [{obj_str}]\n\n" + f"Initial state details:\n{state_str}") + + +def render_trajectory_digest(trajectories: List[Any], + train_tasks: List[Any], + predicates: Collection[Any], + traj_idx: int, + include_states: bool = True, + include_atoms: bool = False, + max_timesteps: int = 10) -> str: + """Header (provenance, goal, reached_goal) plus per-timestep state / atoms. + + / action for one trajectory. + + Raises ``ValueError`` on an out-of-range ``traj_idx``. + """ + if not trajectories: + raise ValueError("No trajectories available yet.") + if traj_idx < 0 or traj_idx >= len(trajectories): + raise ValueError(f"Invalid traj_idx {traj_idx}. " + f"Available: 0-{len(trajectories) - 1}") + traj = trajectories[traj_idx] + provenance = "demo" if traj.is_demo else "interaction" + task_idx = traj._train_task_idx # pylint: disable=protected-access + header = (f"Trajectory {traj_idx}: {len(traj.states)} states, " + f"{len(traj.actions)} actions " + f"[provenance={provenance}, task={task_idx}") + if task_idx is not None and 0 <= task_idx < len(train_tasks): + task = train_tasks[task_idx] + reached = task.goal_holds(traj.states[-1]) + goal_str = ", ".join(str(g) for g in sorted(task.goal)) + header += f", reached_goal={reached}]" + lines = [header, f"Goal: {{{goal_str}}}"] + else: + header += "]" + lines = [header] + + for t_step, state in enumerate(traj.states[:max_timesteps]): + lines.append(f"\n--- Timestep {t_step} ---") + if include_states: + lines.append("State:") + lines.append(state.dict_str(indent=2, num_decimal_points=4)) + if include_atoms: + atoms = utils.abstract(state, predicates) + atoms_str = ", ".join(str(a) for a in sorted(atoms)) + lines.append(f"Atoms: {{{atoms_str}}}") + if t_step < len(traj.actions): + act = traj.actions[t_step] + opt = act.get_option() + lines.append(f"Action: {opt.name}({opt.objects})") + + if len(traj.states) > max_timesteps: + lines.append( + f"\n... ({len(traj.states) - max_timesteps} more timesteps)") + return "\n".join(lines) diff --git a/predicators/agent_sdk/tools/exploration.py b/predicators/agent_sdk/tools/exploration.py index c74a0b123..e90a9eb9d 100644 --- a/predicators/agent_sdk/tools/exploration.py +++ b/predicators/agent_sdk/tools/exploration.py @@ -1,4 +1,4 @@ -"""The explore_python solve-phase exploration tool.""" +"""The solve-phase ``run_python`` tool over the belief probe.""" from typing import Any, Callable, Dict from predicators.agent_sdk.config import ToolSurfaceConfig @@ -11,9 +11,9 @@ def belief_probe_blurb(synthesis_probe: bool) -> str: """The BeliefProbe surface description, shared by every prompt/tool surface that offers the probe. - Solve sessions offer it through the standalone ``explore_python`` - tool; synthesis sessions bind the same facade as ``sim`` inside - ``run_python``'s namespace. One renderer so the two descriptions + Solve sessions bind it as ``sim`` in ``run_python``'s namespace over + the deployed belief model; synthesis sessions bind the same facade + over the candidate simulator. One renderer so the two descriptions cannot drift. ``synthesis_probe`` selects the candidate-simulator wording (task_idx-required resets, ``sim.fit``, and the fit/refine/forward-run validation protocol). @@ -89,7 +89,7 @@ def belief_probe_blurb(synthesis_probe: bool) -> str: "`sim.run(plan_text, render=True, trials=1, solved=False, " "contacts=False)` executes an option " "plan FROM THE CURRENT " - "STATE (same grammar as evaluate_option_plan; print the result " + "STATE (same grammar as submit_plan; print the result " "for per-step outcomes incl. saved per-step scene-image paths - " "view them with the Read tool; pass render=False inside tight " "sweep loops) and advances the state; `-> {subgoals}` " @@ -147,7 +147,7 @@ def belief_probe_blurb(synthesis_probe: bool) -> str: "`sim.refine(sketch_text, timeout=60, require_goal=False, " "require_solved=False)` runs " "backtracking parameter search FROM THE CURRENT STATE (same " - "grammar/search as refine_plan_sketch" + "grammar as submit_plan" f"{_region_syntax_blurb()}; " "success = each step establishes its `-> {subgoals}` " "annotation, and the result's Verdict line states what it " @@ -162,25 +162,21 @@ def belief_probe_blurb(synthesis_probe: bool) -> str: def _build_exploration_tools(ctx: ToolContext, _text_result: Callable, tool: Callable) -> Dict[str, Any]: - """Solve-phase ``explore_python`` over the BeliefProbe exploration facade. + """Solve-phase ``run_python`` over the BeliefProbe exploration facade. The namespace is the probe facade, numpy, and the collected real trajectories as read-only evidence (see ``build_probe_namespace`` - nothing evaluator-shaped beyond the probe's gated paths): the probe - reuses the exact machinery behind ``evaluate_option_plan`` (same + reuses the exact machinery behind ``submit_plan`` (same plan grammar, same option-model executor, same renderer) but carries no scoring surface - nothing run here can be captured as the answer, so it is safe to hand the agent as a freely composable - physics probe. Built only when the session's config opts in: the - ``tool_names=None`` legacy surface would otherwise grant every - default-configured session an in-process exec tool. Synthesis - sessions do not surface this tool at all - there the same facade is - merged into ``run_python``'s namespace (one exec namespace per - session; see ``_get_synthesis_tool_names``). + physics probe. Synthesis sessions attach their own ``run_python`` + (fit data + the candidate-simulator probe in one namespace; see + ``_get_synthesis_tool_names``), and ``create_mcp_tools`` skips this + instance when one is attached. """ surface_cfg = ToolSurfaceConfig.from_cfg() - if not surface_cfg.use_explore_python: - return {} # pylint: disable-next=import-outside-toplevel from predicators.agent_sdk.belief_probe import build_probe_namespace @@ -188,14 +184,15 @@ def _build_exploration_tools(ctx: ToolContext, _text_result: Callable, "EXPLORATORY " "ONLY: nothing run here is captured as your answer - preview " "the evaluator's verdict with sim.run(solved=True), then " - "validate and submit the final plan via evaluate_option_plan " + "validate and submit the final plan via submit_plan " "from the true initial state.") - explore_python = _make_python_exec_tool( + run_python = _make_python_exec_tool( tool, - name="explore_python", + name="run_python", description=( - "Execute Python code for cheap physics/geometry exploration in " - "a persistent namespace (variables survive across calls - " + "Execute Python code (`code`, or `path` to a .py file you " + "wrote in the sandbox) for cheap physics/geometry exploration " + "in a persistent namespace (variables survive across calls - " "define helpers and sweep loops once, reuse them). Available: " + belief_probe_blurb(synthesis_probe=False) + " Also bound: `np`; `trajectories` (the recorded REAL " @@ -207,19 +204,18 @@ def _build_exploration_tools(ctx: ToolContext, _text_result: Callable, "digest of one of them. " "print() output is " "returned; oversize output is spilled to " - "`tool_outputs/explore_python/` (Read/Grep it back). " + + "`tool_outputs/run_python/` (Read/Grep it back). " + (f"Each call has a " - f"{surface_cfg.explore_python_call_timeout:.0f}s " + f"{surface_cfg.python_call_timeout:.0f}s " "wall-clock limit (checked between sim calls, plus a hard stop " "for sim-free code; printed output up to the stop is " "returned): budget sweeps accordingly - " "prefer coarse-to-fine over exhaustive grids, and print " - "intermediate bests so partial results survive a stop. " - if surface_cfg.explore_python_call_timeout > 0 else "") + - f"{submit_desc}"), + "intermediate bests so partial results survive a stop. " if + surface_cfg.python_call_timeout > 0 else "") + f"{submit_desc}"), exec_ns=build_probe_namespace(ctx), sandbox_dir=ctx.sandbox_dir, text_result=_text_result, budget_ctx=ctx, ) - return {"explore_python": explore_python} + return {"run_python": run_python} diff --git a/predicators/agent_sdk/tools/inspection.py b/predicators/agent_sdk/tools/inspection.py deleted file mode 100644 index 4c27cad39..000000000 --- a/predicators/agent_sdk/tools/inspection.py +++ /dev/null @@ -1,430 +0,0 @@ -"""Read-only inspection tools (views over ToolContext state). - -The digest renderers at module level are the single formatting source -for this information: the tools below delegate to them, and the solve / -synthesis prompt builders inject the same digests directly into their -prompts (so sessions that drop the corresponding tools lose no -information and the wording cannot drift between surfaces). -""" -import json -import os -from typing import Any, Callable, Collection, Dict, Iterable, List, Optional, \ - Union - -from predicators import utils -from predicators.agent_sdk.tools.context import ToolContext -from predicators.agent_sdk.tools.results import _error_result, \ - _save_option_to_sandbox -from predicators.agent_sdk.tools.scene import render_pybullet_image - - -def render_types_digest(types: Iterable[Any]) -> str: - """One line per type: name, parent, feature names.""" - lines = [] - for t in sorted(types, key=lambda t: t.name): - features = ", ".join( - t.feature_names) if t.feature_names else "(no features)" - parent_str = f" (parent: {t.parent.name})" if t.parent else "" - lines.append(f"- {t.name}{parent_str}: [{features}]") - if not lines: - return "No types defined." - return "\n".join(lines) - - -def render_options_digest(options: Iterable[Any], - gt_options_ref_path: Optional[str] = None) -> str: - """One line per option: typed signature plus parameter box/descriptions. - - The ``obj:type`` signature and parameter listing here are what the - strict plan parser expects plans to match, so this exact rendering - is shared by the prompts and the ``inspect_options`` tool. - """ - lines = [] - for opt in sorted(options, key=lambda o: o.name): - type_sig = ", ".join(t.name for t in opt.types) - params_dim = opt.params_space.shape[0] if opt.params_space.shape else 0 - if params_dim > 0: - low = opt.params_space.low.tolist() - high = opt.params_space.high.tolist() - if opt.params_description: - desc = ", ".join(opt.params_description) - param_info = f", params=[{desc}], low={low}, high={high}" - else: - param_info = (f", params_dim={params_dim}, " - f"low={low}, high={high}") - else: - param_info = "" - lines.append(f" {opt.name}({type_sig}{param_info})") - if not lines: - return "No options defined." - if gt_options_ref_path: - lines.append(f"\nOption definition source code: " - f"`{gt_options_ref_path}` (search for the option name).") - return "\n".join(lines) - - -def render_task_digest(task: Any, - task_idx: Union[int, str], - predicates: Collection[Any], - include_goal_query_hint: bool = False) -> str: - """Goal (NL preferred), initial atoms, objects, and init-state details for - one task. - - ``task_idx`` is the header label; a string (e.g. ``"(current solve - task)"``) is allowed for tasks outside the train list. - ``include_goal_query_hint`` adds the ``is_goal_state`` / - ``goal_holds`` pointer, which only makes sense (and is only - numerically valid) in sessions whose exec namespace binds those - names (synthesis ``run_python``), so pass it only with an integer - ``task_idx``. - """ - if task.goal_nl: - goal_line = f" Goal (natural language): {task.goal_nl}" - else: - goal_str = ", ".join(str(g) for g in sorted(task.goal)) - goal_line = f" Goal: {{{goal_str}}}" - init_atoms = utils.abstract(task.init, predicates) - atoms_str = ", ".join(str(a) for a in sorted(init_atoms)) - objects = sorted(task.init, key=str) - obj_str = ", ".join(f"{o.name}:{o.type.name}" for o in objects) - state_str = task.init.pretty_str() - hint_line = "" - if include_goal_query_hint: - hint_line = (f" Goal achievement: query " - f"`is_goal_state(state, {task_idx})` or " - f"`train_tasks[{task_idx}].goal_holds(state)`.\n") - return (f"Task {task_idx}:\n" - f"{goal_line}\n" - f"{hint_line}" - f" Initial atoms: {{{atoms_str}}}\n" - f" Objects: [{obj_str}]\n\n" - f"Initial state details:\n{state_str}") - - -def render_trajectory_digest(trajectories: List[Any], - train_tasks: List[Any], - predicates: Collection[Any], - traj_idx: int, - include_states: bool = True, - include_atoms: bool = False, - max_timesteps: int = 10) -> str: - """Header (provenance, goal, reached_goal) plus per-timestep state / atoms. - - / action for one trajectory. - - Raises ``ValueError`` on an out-of-range ``traj_idx``. - """ - if not trajectories: - raise ValueError("No trajectories available yet.") - if traj_idx < 0 or traj_idx >= len(trajectories): - raise ValueError(f"Invalid traj_idx {traj_idx}. " - f"Available: 0-{len(trajectories) - 1}") - traj = trajectories[traj_idx] - provenance = "demo" if traj.is_demo else "interaction" - task_idx = traj._train_task_idx # pylint: disable=protected-access - header = (f"Trajectory {traj_idx}: {len(traj.states)} states, " - f"{len(traj.actions)} actions " - f"[provenance={provenance}, task={task_idx}") - if task_idx is not None and 0 <= task_idx < len(train_tasks): - task = train_tasks[task_idx] - reached = task.goal_holds(traj.states[-1]) - goal_str = ", ".join(str(g) for g in sorted(task.goal)) - header += f", reached_goal={reached}]" - lines = [header, f"Goal: {{{goal_str}}}"] - else: - header += "]" - lines = [header] - - for t_step, state in enumerate(traj.states[:max_timesteps]): - lines.append(f"\n--- Timestep {t_step} ---") - if include_states: - lines.append("State:") - lines.append(state.dict_str(indent=2, num_decimal_points=4)) - if include_atoms: - atoms = utils.abstract(state, predicates) - atoms_str = ", ".join(str(a) for a in sorted(atoms)) - lines.append(f"Atoms: {{{atoms_str}}}") - if t_step < len(traj.actions): - act = traj.actions[t_step] - opt = act.get_option() - lines.append(f"Action: {opt.name}({opt.objects})") - - if len(traj.states) > max_timesteps: - lines.append( - f"\n... ({len(traj.states) - max_timesteps} more timesteps)") - return "\n".join(lines) - - -def _build_inspection_tools(ctx: ToolContext, _text_result: Callable, - tool: Callable) -> Dict[str, Any]: - """Read-only inspection tools (views over ToolContext state).""" - - @tool("inspect_types", "List all object types and their features", {}) - async def inspect_types(_args: Dict[str, Any]) -> Dict[str, Any]: - digest = render_types_digest(ctx.types) - if digest == "No types defined.": - return _text_result(digest) - return _text_result("Current types:\n" + digest) - - @tool("inspect_predicates", - "List all predicates and their type signatures", {}) - async def inspect_predicates(_args: Dict[str, Any]) -> Dict[str, Any]: - lines = [] - for p in sorted(ctx.predicates, key=lambda p: p.name): - type_sig = ", ".join(t.name for t in p.types) - lines.append(f"- {p.name}({type_sig})") - if not lines: - return _text_result("No predicates defined.") - return _text_result("Current predicates:\n" + "\n".join(lines)) - - @tool("inspect_processes", - "List all processes with conditions, effects, and delays", {}) - async def inspect_processes(_args: Dict[str, Any]) -> Dict[str, Any]: - lines = [] - for proc in sorted(ctx.processes, key=lambda p: p.name): - conds = ", ".join(str(a) for a in sorted(proc.condition_at_start)) - adds = ", ".join(str(a) for a in sorted(proc.add_effects)) - dels = ", ".join(str(a) for a in sorted(proc.delete_effects)) - lines.append(f"- {proc.name}\n" - f" Conditions: {{{conds}}}\n" - f" Add effects: {{{adds}}}\n" - f" Delete effects: {{{dels}}}\n" - f" Delay: {proc.delay_distribution}") - if not lines: - return _text_result("No processes defined.") - return _text_result("Current processes:\n" + "\n".join(lines)) - - @tool( - "inspect_options", - "List all options, or inspect a specific option in detail. " - "When given an option_name, saves source code to " - "./proposed_code/.py in the sandbox for you to Read.", - { - "type": "object", - "properties": { - "option_name": { - "type": - "string", - "description": - "Name of a specific option to inspect. Saves its " - "source code to ./proposed_code/.py. " - "Omit to list all options.", - }, - }, - }, - ) - async def inspect_options(args: Dict[str, Any]) -> Dict[str, Any]: - option_name = args.get("option_name") - - if option_name is None: - # List all options (same digest the prompts inject). - digest = render_options_digest(ctx.options) - if digest == "No options defined.": - return _text_result(digest) - return _text_result("Current options:\n" + digest) - - # Detailed inspection of a specific option - opt_map = {o.name: o for o in ctx.options} - if option_name not in opt_map: - return _error_result(f"Unknown option '{option_name}'. " - f"Available: {sorted(opt_map.keys())}") - - opt = opt_map[option_name] - type_sig = ", ".join(t.name for t in opt.types) - dim = opt.params_space.shape[0] if opt.params_space.shape else 0 - - lines = [f"## {opt.name}({type_sig})", ""] - - # Params space - if dim > 0: - lines.append(f"params_dim: {dim}") - lines.append(f"params_low: {opt.params_space.low.tolist()}") - lines.append(f"params_high: {opt.params_space.high.tolist()}") - if opt.params_description: - lines.append(f"params_desc: {list(opt.params_description)}") - else: - lines.append("params_dim: 0 (no continuous parameters)") - - # Source code — point to existing file or reference - if ctx.show_option_source: - lines.append("") - code_path = os.path.join(ctx.sandbox_dir, "proposed_code", - f"{option_name}.py") \ - if ctx.sandbox_dir else None - - if code_path and os.path.exists(code_path): - # Already saved (e.g. from propose_options) - lines.append( - f"Source code: `./proposed_code/{option_name}.py`") - elif ctx.gt_options_ref_path: - # GT options — point to reference file instead of extracting - lines.append(f"Definition code: `{ctx.gt_options_ref_path}` " - f"(search for \"{option_name}\")") - else: - # Extract source and save it - # pylint: disable-next=import-outside-toplevel - import inspect as _inspect - code_parts = [] - for attr_name in ("policy", "initiable", "terminal"): - fn = getattr(opt, attr_name, None) - if fn is None: - continue - try: - src = _inspect.getsource(fn) - code_parts.append(f"# {attr_name}\n{src.rstrip()}") - except (TypeError, OSError): - code_parts.append( - f"# {attr_name}: (source not available)") - - if code_parts: - full_code = "\n\n".join(code_parts) - rel_path = _save_option_to_sandbox(ctx, option_name, - full_code) - if rel_path: - lines.append(f"Source code: `{rel_path}`") - else: - # No sandbox — inline as fallback - lines.append("### Source Code") - lines.append("```python") - lines.append(full_code) - lines.append("```") - - return _text_result("\n".join(lines)) - - @tool( - "inspect_trajectories", - "Inspect trajectory data. Returns state features and/or atoms.", - { - "type": "object", - "properties": { - "traj_idx": { - "type": "integer", - "description": "Trajectory index (0-based)" - }, - "include_states": { - "type": "boolean", - "description": "Include state feature dicts", - "default": True - }, - "include_atoms": { - "type": "boolean", - "description": "Include abstract atoms", - "default": False - }, - "max_timesteps": { - "type": "integer", - "description": "Max timesteps to show", - "default": 10 - }, - }, - "required": ["traj_idx"], - }, - ) - async def inspect_trajectories(args: Dict[str, Any]) -> Dict[str, Any]: - traj_idx = args["traj_idx"] - include_states = args.get("include_states", True) - include_atoms = args.get("include_atoms", False) - max_timesteps = args.get("max_timesteps", 10) - - all_trajs = ctx.offline_trajectories + ctx.online_trajectories - try: - digest = render_trajectory_digest(all_trajs, - ctx.train_tasks, - ctx.predicates, - traj_idx, - include_states=include_states, - include_atoms=include_atoms, - max_timesteps=max_timesteps) - except ValueError as e: - return _error_result(str(e)) - return _text_result(digest) - - @tool( - "inspect_train_tasks", - "Inspect training tasks (goals, initial atoms, objects, state " - "details, and optionally an image of the initial scene)", - { - "type": "object", - "properties": { - "task_idx": { - "type": - "integer", - "description": - "Task index (0-based). Omit to see summary of all.", - }, - "include_image": { - "type": - "boolean", - "description": - "If true and task_idx is given, render and return an " - "image of the initial state (PyBullet envs only).", - }, - }, - }, - ) - async def inspect_train_tasks(args: Dict[str, Any]) -> Dict[str, Any]: - task_idx = args.get("task_idx") - include_image = args.get("include_image", False) - - if task_idx is not None: - if task_idx < 0 or task_idx >= len(ctx.train_tasks): - return _error_result(f"Invalid task_idx {task_idx}. " - f"Available: 0-{len(ctx.train_tasks)-1}") - task = ctx.train_tasks[task_idx] - text = render_task_digest(task, - task_idx, - ctx.predicates, - include_goal_query_hint=True) - - content: List[Dict[str, Any]] = [{"type": "text", "text": text}] - - if include_image: - img_block = render_pybullet_image(ctx, - f"task_{task_idx}_init", - state=task.init) - if img_block is not None: - content.append(img_block) - - return {"content": content} - - lines = [f"Total tasks: {len(ctx.train_tasks)}"] - for i, task in enumerate(ctx.train_tasks[:10]): - if task.goal_nl: - lines.append(f" Task {i}: {task.goal_nl}") - else: - goal_str = ", ".join(str(g) for g in sorted(task.goal)) - lines.append(f" Task {i}: goal={{{goal_str}}}") - if len(ctx.train_tasks) > 10: - lines.append(f" ... ({len(ctx.train_tasks) - 10} more tasks)") - return _text_result("\n".join(lines)) - - @tool("inspect_planning_results", - "Get latest planning performance metrics", {}) - async def inspect_planning_results( - _args: Dict[str, Any]) -> Dict[str, Any]: - if not ctx.planning_results: - return _text_result("No planning results available yet.") - return _text_result( - json.dumps(ctx.planning_results, indent=2, default=str)) - - @tool("inspect_past_proposals", - "Get summaries of proposals and retractions from all past " - "iterations", {}) - async def inspect_past_proposals(_args: Dict[str, Any]) -> Dict[str, Any]: - if not ctx.iteration_history: - return _text_result("No past proposals available yet.") - lines = [] - for entry in ctx.iteration_history: - lines.append(json.dumps(entry, indent=2, default=str)) - return _text_result("\n---\n".join(lines)) - - return { - "inspect_types": inspect_types, - "inspect_predicates": inspect_predicates, - "inspect_processes": inspect_processes, - "inspect_options": inspect_options, - "inspect_trajectories": inspect_trajectories, - "inspect_train_tasks": inspect_train_tasks, - "inspect_planning_results": inspect_planning_results, - "inspect_past_proposals": inspect_past_proposals, - } diff --git a/predicators/agent_sdk/tools/journal_tools.py b/predicators/agent_sdk/tools/journal_tools.py deleted file mode 100644 index 67cfeb7ed..000000000 --- a/predicators/agent_sdk/tools/journal_tools.py +++ /dev/null @@ -1,81 +0,0 @@ -"""The record_journal solve-journal tool.""" -from typing import Any, Callable, Dict - -from predicators.agent_sdk.config import ValidationConfig -from predicators.agent_sdk.tools.context import ToolContext -from predicators.agent_sdk.tools.results import _error_result - - -def _build_journal_tools(ctx: ToolContext, _text_result: Callable, - tool: Callable) -> Dict[str, Any]: - """``record_journal`` - agent-authored entries in the run's solve - journal. - - Built only when the journal is enabled. The journal is the curated - cross-attempt/cross-task memory channel for fresh-context solve - sessions, so the tool guidance insists on facts and measurements: - recorded verdicts ("X is impossible") from a failed attempt would - re-import exactly the anchoring a restart is meant to shed - (run_20260717_230436 seed1 concluded a "hard collision boundary" - its sibling run placed through minutes later). - """ - if not ValidationConfig.from_cfg().use_journal: - return {} - # pylint: disable-next=import-outside-toplevel - from predicators.agent_sdk import journal as journal_mod - - @tool( - "record_journal", - ("Append a short entry to the run's persistent solve journal " - "(journal.md), which future solve attempts - starting with FRESH " - "context - read in their prompt. Record durable, transferable " - "facts: what you tried with exact parameters, what you measured, " - "what worked (and its load-bearing values), and what a fresh " - "attempt should try differently. Facts and measurements ONLY - do " - "NOT record conclusions like 'X is impossible' or 'the task " - "requires Y' (a wrong verdict anchors every later attempt; the " - "evidence lets them re-judge). State every negative result as the " - "exact family swept - parameters, orientations, regions, and any " - "formula the sweep assumed - plus what remains untested: 'X never " - "works' generalized from a partial sweep has buried the correct " - "mechanism for entire runs. In learning sessions, record what the " - "current simulator gets wrong and which experiment would " - "discriminate - future solve attempts read this. Keep it " - f"skimmable: a few bullets, under {journal_mod.MAX_ENTRY_CHARS} " - "chars."), - { - "type": "object", - "properties": { - "entry": { - "type": "string", - "description": "The journal entry (markdown bullets).", - } - }, - "required": ["entry"], - }, - ) - async def record_journal(args: Dict[str, Any]) -> Dict[str, Any]: - entry = (args.get("entry") or "").strip() - if not entry: - return _error_result("`entry` is required.") - if not ctx.sandbox_dir: - return _error_result("No sandbox directory in this session.") - if ctx.learn_cycle_index is not None: - if ctx.learn_cycle_index < 0: - where = "offline learning" - else: - where = f"learning cycle {ctx.learn_cycle_index}" - elif ctx.test_task_idx is not None: - where = f"test task {ctx.test_task_idx}" - else: - where = "pre-test phase" - attempt = f", attempt {ctx.attempt_index}" if ctx.attempt_index else "" - note = journal_mod.append_entry(ctx.sandbox_dir, - f"Agent notes ({where}{attempt})", - entry) - msg = "Recorded to the solve journal." - if note is not None: - msg += f" NOTE: {note}." - return _text_result(msg) - - return {"record_journal": record_journal} diff --git a/predicators/agent_sdk/tools/planning.py b/predicators/agent_sdk/tools/planning.py deleted file mode 100644 index 2e5165dc3..000000000 --- a/predicators/agent_sdk/tools/planning.py +++ /dev/null @@ -1,536 +0,0 @@ -"""Planning tools: generate_bilevel_plan, generate_abstract_plan, and -refine_plan_sketch.""" -import time -import traceback -from typing import Any, Callable, Dict, List, Optional - -import numpy as np - -from predicators import utils -from predicators.agent_sdk import bilevel_sketch -from predicators.agent_sdk.config import RefinementConfig -from predicators.agent_sdk.tools.context import ToolContext -from predicators.agent_sdk.tools.results import _error_result -from predicators.agent_sdk.tools.sandbox_guard import _scrub_host_paths -from predicators.agent_sdk.tools.tasks import _resolve_task -from predicators.agent_sdk.tools.verdicts import _belief_rollout_verdict, \ - _format_evaluator_verdict, _resolve_task_evaluator, \ - load_ground_sampler_fns, make_solved_check -from predicators.planning_with_processes import \ - run_task_plan_with_processes_once -from predicators.settings import CFG - - -def _build_planning_tools(ctx: ToolContext, _text_result: Callable, - tool: Callable) -> Dict[str, Any]: - """Planning tools (generate bilevel / abstract plans).""" - - @tool( - "generate_bilevel_plan", - "Generate a concrete option plan using the bilevel planner. Returns " - "grounded options with sampled continuous parameters, simulated " - "step-by-step via the option model.", - { - "type": "object", - "properties": { - "task_idx": { - "type": - "integer", - "description": - "Train task index. Omit to use the current " - "solve-time task (if available)." - }, - "timeout": { - "type": "integer", - "description": "Planning timeout in seconds", - "default": 30 - }, - }, - }, - ) - async def generate_bilevel_plan(args: Dict[str, Any]) -> Dict[str, Any]: - task_idx = args.get("task_idx") - timeout = args.get("timeout", 30) - - # Resolve task - resolved, task_err = _resolve_task(ctx, task_idx) - if task_err is not None: - return task_err - assert resolved is not None - task = resolved.task - task_label = resolved.description - - all_preds = ctx.predicates | ctx.iteration_proposals.proposed_predicates - all_procs = ctx.processes | ctx.iteration_proposals.proposed_processes - all_types = ctx.types | ctx.iteration_proposals.proposed_types - - # Get abstract plan - try: - plan, _atoms_seq, metrics = run_task_plan_with_processes_once( - task, - all_procs, - all_preds, - all_types, - timeout, - seed=CFG.seed, - _task_planning_heuristic=CFG.process_task_planning_heuristic, - max_horizon=float(CFG.horizon)) - except Exception as e: # pylint: disable=broad-except - return _text_result(f"Planning failed for {task_label}.\n" - f"Reason: {type(e).__name__}: {e}") - - if not plan: - return _text_result( - f"Planner returned empty plan for {task_label}.") - - # Sample options and simulate - rng = np.random.default_rng(CFG.seed) - state = task.init - lines = [ - f"Bilevel plan for {task_label} " - f"({len(plan)} steps, " - f"{metrics.get('num_nodes_expanded', '?')} nodes expanded):" - ] - - option_plan_lines = [] - for step_idx, ground_proc in enumerate(plan): - try: - option = ground_proc.sample_option(state, task.goal, rng) - except Exception as e: # pylint: disable=broad-except - lines.append( - f"Step {step_idx}: {ground_proc.name}" - f"({', '.join(str(o) for o in ground_proc.objects)}) " - f"- SAMPLE FAILED: {e}") - break - - # Format option - obj_strs = ", ".join(f"{o.name}:{o.type.name}" - for o in option.objects) - params_str = ", ".join(f"{p:.4f}" for p in option.params) - option_line = f"{option.name}({obj_strs})[{params_str}]" - option_plan_lines.append(option_line) - - # Simulate - if ctx.option_model is not None: - try: - next_state, num_actions = \ - ctx.option_model.get_next_state_and_num_actions( - state, option) - atoms_before = utils.abstract(state, all_preds) - atoms_after = utils.abstract(next_state, all_preds) - added = atoms_after - atoms_before - deleted = atoms_before - atoms_after - lines.append( - f"Step {step_idx}: {option_line} " - f"({num_actions} actions)" - f"\n Added: " - f"{{{', '.join(str(a) for a in sorted(added))}}}" - f"\n Deleted: " - f"{{{', '.join(str(a) for a in sorted(deleted))}}}") - state = next_state - except Exception as e: # pylint: disable=broad-except - lines.append(f"Step {step_idx}: {option_line} " - f"- SIMULATION ERROR: {e}") - break - else: - lines.append(f"Step {step_idx}: {option_line}") - - # Check goal via env-side classifiers so the result is robust - # to invented predicates that don't reuse env names. - if ctx.option_model is not None: - goal_achieved = task.goal_holds(state) - lines.append(f"\nGoal achieved: {goal_achieved}") - - lines.append("\n## Option Plan (copy-paste format):") - lines.extend(option_plan_lines) - - return _text_result("\n".join(lines)) - - @tool( - "generate_abstract_plan", - "Generate an abstract plan skeleton without continuous parameters. " - "Returns option names and objects with parameter space info so you " - "can fill in continuous parameters yourself.", - { - "type": "object", - "properties": { - "task_idx": { - "type": - "integer", - "description": - "Train task index. Omit to use the current " - "solve-time task (if available)." - }, - "timeout": { - "type": "integer", - "description": "Planning timeout in seconds", - "default": 30 - }, - }, - }, - ) - async def generate_abstract_plan(args: Dict[str, Any]) -> Dict[str, Any]: - task_idx = args.get("task_idx") - timeout = args.get("timeout", 30) - - # Resolve task - resolved, task_err = _resolve_task(ctx, task_idx) - if task_err is not None: - return task_err - assert resolved is not None - task = resolved.task - task_label = resolved.description - - all_preds = ctx.predicates | ctx.iteration_proposals.proposed_predicates - all_procs = ctx.processes | ctx.iteration_proposals.proposed_processes - all_types = ctx.types | ctx.iteration_proposals.proposed_types - - try: - plan, _atoms_seq, metrics = run_task_plan_with_processes_once( - task, - all_procs, - all_preds, - all_types, - timeout, - seed=CFG.seed, - _task_planning_heuristic=CFG.process_task_planning_heuristic, - max_horizon=float(CFG.horizon)) - except Exception as e: # pylint: disable=broad-except - return _text_result(f"Planning failed for {task_label}.\n" - f"Reason: {type(e).__name__}: {e}") - - if not plan: - return _text_result( - f"Planner returned empty plan for {task_label}.") - - lines = [ - f"Abstract plan for {task_label} " - f"({len(plan)} steps, " - f"{metrics.get('num_nodes_expanded', '?')} nodes expanded):", - "", - ] - - for step_idx, ground_proc in enumerate(plan): - obj_strs = ", ".join(f"{o.name}:{o.type.name}" - for o in ground_proc.option_objs) - option = ground_proc.option - params_dim = option.params_space.shape[0] - if params_dim > 0: - low = option.params_space.low.tolist() - high = option.params_space.high.tolist() - param_info = (f" params_dim={params_dim}, " - f"low={low}, high={high}") - else: - param_info = " (no continuous params)" - lines.append( - f"Step {step_idx}: {option.name}({obj_strs})\n{param_info}") - - # Include conditions for context - lines.append("\n## Process conditions:") - for step_idx, ground_proc in enumerate(plan): - conds = ", ".join( - str(a) for a in sorted(ground_proc.condition_at_start)) - adds = ", ".join(str(a) for a in sorted(ground_proc.add_effects)) - dels = ", ".join( - str(a) for a in sorted(ground_proc.delete_effects)) - lines.append(f"Step {step_idx} ({ground_proc.name}):" - f"\n Conditions: {{{conds}}}" - f"\n Add effects: {{{adds}}}" - f"\n Delete effects: {{{dels}}}") - - return _text_result("\n".join(lines)) - - _gs_refine_doc = ( - "Confine a step's sampling with a GROUND SAMPLER after its " - "`[params]`: either a region `~ [w1, w2]` (per-parameter " - "half-widths; the exact center is tried first, then ALL further " - "samples for the step are drawn uniformly from " - "`[center - w, center + w]` clipped to the option's range - a zero " - "width pins every draw to the center), or `~ my_sampler` naming an " - "entry of `GROUND_SAMPLERS` in the sandbox file " - "`ground_samplers.py`, which you Write/Edit and which is reloaded " - "fresh on every call (each entry is " - "`fn(state, subgoal_atoms, rng, objects) -> params`, so it can " - "shape any state-dependent distribution). A ground sampler " - "overrides any learned per-skill sampler for that step. " - if RefinementConfig.from_cfg().ground_samplers else "") - _gs_refine_plan_doc = ( - ", optionally followed by a ground sampler: `~ [w1, w2]` " - "half-widths around those params, or `~ my_sampler` naming a " - "GROUND_SAMPLERS entry in ground_samplers.py" - if RefinementConfig.from_cfg().ground_samplers else "") - - @tool( - "refine_plan_sketch", - "FIND continuous parameters for a plan SKETCH: run a backtracking " - "search over the option model, then — on success — forward-validate " - "the refined plan. Unlike evaluate_option_plan (which runs your EXACT " - "params with no search), this takes a sketch and lets the search find " - "params. You may seed it by appending `[p1, p2]` per step (use `[]` " - "for none); the search tries them first, then samples. " + - _gs_refine_doc + "`plan` is one " - "option call per line with typed object references (`obj:type`) and " - "every argument supplied; add `-> {Atom(obj:type, ...)}` subgoal " - "annotations (effectively required after open-ended skills like Place, " - "and for Wait to say when it should end — prefix an atom with NOT to " - "require it become false). When the task has an evaluator, success is " - "also gated on its scoring: a parameterization that reaches the goal " - "atoms but scores as a non-solve (no success credit in its reward) is " - "discarded and the search resamples. On SUCCESS it reports the exact " - "PARAMETERS it found per step — submit those via evaluate_option_plan, " - "which is the delivery path; refine_plan_sketch itself does NOT " - "submit. Also reports the verdict (SUCCESS / TIMEOUT / " - "SAMPLE_EXHAUSTED with the stuck step / FORWARD_VALIDATION_FAILED / " - "SCORED_NON_SOLVE) and time used. Requires a simulator (option " - "model). Slower than evaluate_option_plan — use it to find params " - "for hard steps, not to submit.", - { - "type": "object", - "properties": { - "plan": { - "type": - "string", - "description": - "Option-skeleton plan text, one option call per " - "line, typed `obj:type` references, every argument " - "supplied; optional `-> {Atom(...)}` subgoal per step, " - "and `[p1, p2]` proposed continuous params per step " - "(`[]` for none) when param-proposing is enabled" + - _gs_refine_plan_doc + ".", - }, - "task_idx": { - "type": - "integer", - "description": - "Train task index. Omit to use the current " - "solve-time task (if available).", - }, - "timeout": { - "type": - "number", - "description": - "Refinement timeout in seconds. Omit for an auto " - "value that scales with sketch length; the value " - "used is reported back.", - }, - }, - "required": ["plan"], - }, - ) - async def refine_plan_sketch(args: Dict[str, Any]) -> Dict[str, Any]: - refine_cfg = RefinementConfig.from_cfg() - if ctx.option_model is None: - return _error_result( - "refine_plan_sketch requires a simulator (no option model " - "in ToolContext).") - - # Resolve the task (mirrors evaluate_option_plan). - resolved, task_err = _resolve_task(ctx, args.get("task_idx")) - if task_err is not None: - return task_err - assert resolved is not None - task = resolved.task - task_idx = resolved.label - - all_options = ctx.options | ctx.iteration_proposals.proposed_options - all_predicates = (ctx.predicates - | ctx.iteration_proposals.proposed_predicates) - # Keep the option model's name map in sync with proposed options so - # refinement can ground them (matches evaluate_option_plan). - model = ctx.option_model - model._name_to_parameterized_option = ( # type: ignore[attr-defined] # pylint: disable=protected-access - {o.name: o - for o in all_options}) - # Union declared types with those reachable from options/predicates/ - # objects so typed `obj:type` references in the sketch resolve. - types = set(ctx.types) - for opt in all_options: - types.update(opt.types) - for pred in all_predicates: - types.update(pred.types) - types.update(o.type for o in task.init) - - plan_text = (args.get("plan") or "").strip() - if not plan_text: - return _error_result("`plan` is required (option-skeleton text).") - try: - # strict: the `plan` argument is pure sketch text (see - # evaluate_option_plan) - unparseable lines must error, not be - # silently dropped. - # Named `~ my_sampler` references resolve against the agent's - # ground_samplers.py, reloaded fresh so edits between calls - # take effect; a broken file is surfaced instead of silently - # falling back to uniform draws. - gs_fns, gs_err = load_ground_sampler_fns(ctx) - if gs_err is not None: - return _error_result(gs_err) - parse_notices: List[str] = [] - sketch = bilevel_sketch.parse_sketch_from_text( - plan_text, - task, - predicates=all_predicates, - options=all_options, - types=types, - parse_continuous_params=refine_cfg.use_llm_initial_params, - strict=True, - parse_ground_samplers=refine_cfg.ground_samplers, - ground_sampler_fns=gs_fns or None, - notices=parse_notices, - ) - except Exception as e: # pylint: disable=broad-except - return _error_result(f"Could not parse plan sketch: {e}") - if not sketch: - return _error_result( - "Parsed empty plan sketch. Check that every line names a " - "known option with typed `obj:type` arguments matching the " - "Options digest in your prompt.") - - timeout, timeout_source = bilevel_sketch.resolve_refine_timeout( - args.get("timeout"), - len(sketch), - per_step=refine_cfg.refinement_timeout_per_step, - minimum=refine_cfg.refinement_timeout_min) - - # Refinement accepts a parameterization only if the task evaluator - # also scores its rollout as a solve: a candidate that reaches the - # goal atoms yet earns no success credit (e.g. the poker, not the - # cascade, toppled the target) is discarded and refinement is - # resampled with a fresh rng, all attempts sharing the one timeout - # budget. Without this gate the search happily converges onto - # parameterizations the env would score as non-solves and reports - # SUCCESS on them (run_20260713_172854 seed0 task1 test034). The - # gate reads ONLY the public (terminated, reward, solved) triple - - # the standard RL end-of-episode observables - so it grants the - # search nothing the agent could not compute itself, and it never - # depends on a reward sign convention. - attempts = max(1, refine_cfg.refine_evaluator_attempts) - discarded_rewards: List[float] = [] - verdict_line: Optional[str] = None - non_solve = False - start = time.perf_counter() - success, report = False, "" - plan: List[Any] = [] - - # In-search version of the same gate: reject a goal-atom-reaching - # candidate DURING backtracking when the evaluator scores it as a - # non-solve, so the search keeps moving from the same node (its - # upstream samples intact) instead of converging onto uncertifiable - # parameters and needing a cold restart below. The restart loop - # stays as the safety net for verdict flakiness: the post-hoc - # check re-rolls the accepted plan, and a re-roll that scores - # differently (the sim is nondeterministic across runs) still - # triggers a resample. - _gate_evaluator = _resolve_task_evaluator(ctx, task_idx) - solved_check = None - if _gate_evaluator is not None: - solved_check = make_solved_check( - _gate_evaluator, - getattr(ctx.option_model, "sim_env", None), - on_reject=discarded_rewards.append) - - for attempt in range(attempts): - remaining = timeout - (time.perf_counter() - start) - if attempt and remaining < 5.0: - break - try: - success, report, plan = \ - bilevel_sketch.refine_and_validate_report( - task, - sketch, - ctx.option_model, - predicates=all_predicates, - timeout=remaining if attempt else timeout, - rng=np.random.default_rng(CFG.seed + attempt), - max_samples_per_step=refine_cfg.max_samples_per_step, - check_subgoals=refine_cfg.check_subgoals, - log_state=refine_cfg.log_state, - parameterized_samplers=ctx.parameterized_samplers - or None, - run_id="planner_refine", - timeout_source=timeout_source, - solved_check=solved_check, - strip_latent_wait_targets=( - not ctx.latent_tracking_available), - ) - except Exception: # pylint: disable=broad-except - tb = _scrub_host_paths(traceback.format_exc()) - return _error_result(f"Refinement raised:\n{tb}") - non_solve = False - if not (success and plan): - break - scored = _belief_rollout_verdict(ctx, task, task_idx, plan, - all_predicates) - if scored is None: - break - verdict, coarse = scored - if coarse or not verdict["terminated"] or verdict["solved"]: - verdict_line = _format_evaluator_verdict(verdict, - coarse=coarse) - break - discarded_rewards.append(verdict["reward"]) - success = False - non_solve = True - - # A failed search whose DEEPEST blocker was the in-search gate is - # the verdict the restart loop expresses as SCORED_NON_SOLVE: the - # sketch reaches the goal atoms but never certifiably, so say that - # (with the change-the-sketch advice). A search that rejected a - # candidate along the way but then failed on something else (an - # upstream IK wall, a timeout mid-descent) keeps its own headline - # - the near-miss line and the discard NOTE still surface the - # rejections. - if (not success and discarded_rewards - and "scored non-solve" in report): - non_solve = True - rewards_str = ", ".join(f"{r:.2f}" for r in discarded_rewards) - if non_solve: - report = ( - "FAILURE: SCORED_NON_SOLVE\n" - f" Refinement found goal-atom-reaching parameters " - f"{len(discarded_rewards)} time(s), but the task evaluator " - f"scored every such rollout as a non-solve (rewards: " - f"{rewards_str}; no success credit). The real env applies " - "the same scoring, so these parameters can never count as " - "a solve - change the sketch (e.g. different placements or " - "orientations), not just the parameters.\n" - "Last attempt detail:\n" + report) - elif discarded_rewards: - passed_tail = ( - "; the result above is from parameters that passed the " - "evaluator's scoring." if success else ".") - report += ( - f"\n NOTE: {len(discarded_rewards)} earlier " - f"parameterization(s) reached the goal atoms but scored as " - f"non-solves (rewards: {rewards_str}) and were discarded " - f"during the search{passed_tail}") - - # refine_plan_sketch is a parameter FINDER, not a submission path: on - # success, append the parameters the search found per step so the - # agent can submit these exact values via evaluate_option_plan (the - # only delivery path). It deliberately does NOT capture a solved plan. - if success and plan: - param_lines = [] - for i, gopt in enumerate(plan): - objs = ", ".join(o.name for o in gopt.objects) - par = ", ".join(f"{p:.4f}" for p in gopt.params) - param_lines.append(f" {i}: {gopt.name}({objs})[{par}]") - report += ("\n\nParameters found (submit these exact values via " - "evaluate_option_plan):\n" + "\n".join(param_lines)) - if verdict_line is not None: - report += "\n" + verdict_line - - if parse_notices: - report = "\n".join(f"NOTE: {n}" for n in parse_notices) + \ - "\n" + report - - return _text_result(f"Task {task_idx}:\n{report}") - - # ------------------------------------------------------------------ # - # Scene annotation - # ------------------------------------------------------------------ # - - return { - "generate_bilevel_plan": generate_bilevel_plan, - "generate_abstract_plan": generate_abstract_plan, - "refine_plan_sketch": refine_plan_sketch, - } diff --git a/predicators/agent_sdk/tools/predicate_synthesis.py b/predicators/agent_sdk/tools/predicate_synthesis.py index 0f6ebf33a..6cc7d7897 100644 --- a/predicators/agent_sdk/tools/predicate_synthesis.py +++ b/predicators/agent_sdk/tools/predicate_synthesis.py @@ -1,28 +1,25 @@ -"""Predicate-invention synthesis tools (predicate invention loop).""" -import os +"""The ``sim.predicates()`` loader for predicate-invention sessions.""" from typing import Any, Callable, Dict, List, Optional, Sequence, Tuple from predicators.agent_sdk.proposal_exec import build_exec_context, \ exec_code_safely, validate_predicate from predicators.agent_sdk.synthesis_backend import PredicateSynthesisBackend from predicators.agent_sdk.tools.params_view import _ParamsView -from predicators.agent_sdk.tools.results import _make_coercing_tool, \ - _make_spilling_text_result from predicators.agent_sdk.tools.sandbox_guard import _scrub_host_paths from predicators.agent_sdk.tools.snapshots import _ArtifactSnapshotter from predicators.structs import LowLevelTrajectory, Predicate, State, Type -def create_predicate_synthesis_tools( +def make_predicate_quality_loader( predicates_file: str, predicates_versions_dir: str, approach: PredicateSynthesisBackend, trajectories: List[LowLevelTrajectory], cycle_index_provider: Optional[Callable[[], int]] = None, -) -> list: - """Create the predicate-invention synthesis tool. +) -> Callable[..., str]: + """Build the ``sim.predicates()`` loader for one synthesis session. - Returns ``[evaluate_predicate_quality]``. The tool loads + The loader loads ``predicates.py`` fresh on each call (snapshotting into ``predicates_versions_dir`` as ``cycle_XXX_vers_YYY_predicates.py``), validates each @@ -46,16 +43,9 @@ def create_predicate_synthesis_tools( # pylint: disable=import-outside-toplevel import traceback # pylint: disable=redefined-outer-name,reimported - from claude_agent_sdk import tool as _sdk_tool - tool = _make_coercing_tool(_sdk_tool) - from predicators.code_sim_learning.fit_space import ParamSpec # pylint: enable=import-outside-toplevel - # ``predicates_file`` lives at ``/predicates.py``, so its - # parent is the sandbox root — spill oversize output there rather than - # letting the agent SDK dump it outside the sandbox. - _text = _make_spilling_text_result(os.path.dirname(predicates_file)) _snapshotter = _ArtifactSnapshotter( live_file=predicates_file, versions_dir=predicates_versions_dir, @@ -174,51 +164,22 @@ def rec(idx: int, picked: List[Any], used: set) -> None: rec(0, [], set()) return out - @tool( - "evaluate_predicate_quality", - "Load LEARNED_PREDICATES (fresh from `predicates.py`) and " - "report milestone behaviour over demo trajectories. For each " - "predicate × each grounding, evaluates pred.holds(state) at " - "every step and reports: coverage (ever-true / ever-false), " - "transition counts, first-flip step, and monotonicity (ideal " - "milestone flips False->True exactly once and stays true). " - "After loading, the predicate set used by " - "sim.refine is updated — so call this tool any " - "time you edit predicates.py before re-running refinement. " - "Snapshots the predicates file into predicates_versions/; " - "output tagged [cycle_XXX_vers_YYY].", - { - "type": "object", - "properties": { - "max_trajectories": { - "type": "integer", - "description": "Max trajectories to scan " - "(default 10).", - }, - "max_groundings_per_predicate": { - "type": - "integer", - "description": - "Max object groundings to evaluate " - "per predicate (default 4).", - }, - }, - }, - ) - async def evaluate_predicate_quality( - args: Dict[str, Any]) -> Dict[str, Any]: - max_trajs = int(args.get("max_trajectories", 10)) - max_groundings = int(args.get("max_groundings_per_predicate", 4)) + def predicate_quality(max_trajectories: int = 10, + max_groundings_per_predicate: int = 4) -> str: + """Reload predicates.py, install LEARNED_PREDICATES, and report + milestone behaviour over the recorded trajectories.""" + max_trajs = int(max_trajectories) + max_groundings = int(max_groundings_per_predicate) try: preds, version_tag, err, warnings = ( _snapshot_and_load_predicates(predicates_file)) except Exception: # pylint: disable=broad-except - return _text(f"Error loading predicates.py:\n" - f"{_scrub_host_paths(traceback.format_exc())}") + return (f"Error loading predicates.py:\n" + f"{_scrub_host_paths(traceback.format_exc())}") if err is not None: - return _text(err) + return err prefix = f"[{version_tag}]" scanned = trajectories[:max_trajs] @@ -237,7 +198,7 @@ async def evaluate_predicate_quality( lines.append("") lines.append("LEARNED_PREDICATES is empty — add " "Predicate(...) entries to predicates.py.") - return _text("\n".join(lines)) + return "\n".join(lines) # Pre-materialise per-step `latent` per trajectory. For # recurrent approaches this rolls the trajectory through the @@ -320,6 +281,6 @@ async def evaluate_predicate_quality( for el in error_lines[:max_trajs]: lines.append(el) - return _text("\n".join(lines)) + return "\n".join(lines) - return [evaluate_predicate_quality] + return predicate_quality diff --git a/predicators/agent_sdk/tools/proposals.py b/predicators/agent_sdk/tools/proposals.py deleted file mode 100644 index 61f7d7c8c..000000000 --- a/predicators/agent_sdk/tools/proposals.py +++ /dev/null @@ -1,418 +0,0 @@ -"""Proposal and retraction tools.""" -import logging -import os -import traceback -from typing import Any, Callable, Dict, List - -from predicators.agent_sdk.config import ToolSurfaceConfig -from predicators.agent_sdk.proposal_exec import build_exec_context, \ - exec_code_safely, validate_predicate -from predicators.agent_sdk.tools.context import ToolContext -from predicators.agent_sdk.tools.results import _error_result, \ - _save_option_to_sandbox -from predicators.agent_sdk.tools.sandbox_guard import _scrub_host_paths -from predicators.structs import CausalProcess, ParameterizedOption, \ - Predicate, Type - - -def _build_proposal_tools(ctx: ToolContext, _text_result: Callable, - tool: Callable) -> Dict[str, Any]: - """Proposal tools (agent authors new types/predicates/options/etc.).""" - _propose_count = [0] # mutable counter in closure - - def _save_proposal_code(tool_name: str, code: str, names: List[str], - description: str) -> None: - if not ctx.sandbox_dir: - return - _propose_count[0] += 1 - subdir = os.path.join(ctx.sandbox_dir, "proposed_code") - os.makedirs(subdir, exist_ok=True) - names_slug = "_".join(names)[:80] - filename = f"{_propose_count[0]:03d}_{tool_name}_{names_slug}.py" - filepath = os.path.join(subdir, filename) - header = f'"""{tool_name}: {description}"""\n\n' - with open(filepath, "w", encoding="utf-8") as f: - f.write(header + code) - logging.info(f"Saved proposal code to {filepath}") - - @tool( - "propose_types", - "Propose new types. Code must define `proposed_types` (a list of " - "Type objects).", - { - "type": "object", - "properties": { - "code": { - "type": "string", - "description": "Python code defining proposed_types" - }, - "description": { - "type": "string", - "description": "Why these types are needed" - }, - }, - "required": ["code", "description"], - }, - ) - async def propose_types(args: Dict[str, Any]) -> Dict[str, Any]: - if not ToolSurfaceConfig.from_cfg().propose_types: - return _error_result("Type proposals are disabled.") - code = args["code"] - exec_ctx = build_exec_context(ctx.types, ctx.predicates, ctx.options) - result, error = exec_code_safely(code, exec_ctx, "proposed_types") - if error: - return _error_result(f"Code execution failed:\n{error}") - if not isinstance(result, (list, set)): - return _error_result( - f"proposed_types must be a list/set, got {type(result)}") - for t in result: - if not isinstance(t, Type): - return _error_result( - f"Each item must be a Type, got {type(t)}: {t}") - proposed = set(result) - ctx.iteration_proposals.proposed_types |= proposed - names = [t.name for t in proposed] - logging.info(f"Agent proposed types: {names}") - _save_proposal_code("propose_types", code, names, - args.get("description", "")) - return _text_result( - f"Successfully proposed {len(proposed)} types: {names}") - - @tool( - "propose_predicates", - "Propose new predicates. Code must define `proposed_predicates` " - "(a list of Predicate objects).", - { - "type": "object", - "properties": { - "code": { - "type": "string", - "description": "Python code defining proposed_predicates" - }, - "description": { - "type": "string", - "description": "What these predicates capture" - }, - }, - "required": ["code", "description"], - }, - ) - async def propose_predicates(args: Dict[str, Any]) -> Dict[str, Any]: - if not ToolSurfaceConfig.from_cfg().propose_predicates: - return _error_result("Predicate proposals are disabled.") - code = args["code"] - exec_ctx = build_exec_context(ctx.types, ctx.predicates, ctx.options) - result, error = exec_code_safely(code, exec_ctx, "proposed_predicates") - if error: - return _error_result(f"Code execution failed:\n{error}") - if not isinstance(result, (list, set)): - return _error_result( - f"proposed_predicates must be a list/set, got {type(result)}") - - validated = [] - errors = [] - for pred in result: - if not isinstance(pred, Predicate): - errors.append(f"Not a Predicate: {type(pred)}: {pred}") - continue - if ctx.example_state is not None: - err = validate_predicate(pred, ctx.types, ctx.example_state) - if err: - errors.append(f"{pred.name}: {err}") - continue - validated.append(pred) - - proposed = set(validated) - ctx.iteration_proposals.proposed_predicates |= proposed - names = [p.name for p in proposed] - logging.info(f"Agent proposed predicates: {names}") - _save_proposal_code("propose_predicates", code, names, - args.get("description", "")) - - msg = f"Successfully proposed {len(proposed)} predicates: {names}" - if errors: - msg += f"\n\nValidation errors ({len(errors)}):\n" + \ - "\n".join(errors) - return _text_result(msg) - - @tool( - "propose_task_augmentor", - "Propose a task augmentation function. Code must define " - "`augment_task(task) -> Task`.", - { - "type": "object", - "properties": { - "code": { - "type": - "string", - "description": - "Python code defining augment_task(task) -> Task" - }, - "description": { - "type": "string", - "description": "What the augmentor does" - }, - }, - "required": ["code", "description"], - }, - ) - async def propose_task_augmentor(args: Dict[str, Any]) -> Dict[str, Any]: - if not ToolSurfaceConfig.from_cfg().propose_objects: - return _error_result("Object augmentor proposals are disabled.") - code = args["code"] - exec_ctx = build_exec_context(ctx.types, ctx.predicates, ctx.options) - result, error = exec_code_safely(code, exec_ctx, "augment_task") - if error: - return _error_result(f"Code execution failed:\n{error}") - if not callable(result): - return _error_result( - f"augment_task must be callable, got {type(result)}") - - # Test on first train task if available - if ctx.train_tasks: - try: - test_task = ctx.train_tasks[0] - augmented = result(test_task) - orig_objs = set(test_task.init) - new_objs = set(augmented.init) - orig_objs - obj_names = [str(o) for o in sorted(new_objs, key=str)] - except Exception: # pylint: disable=broad-except - return _error_result( - f"augment_task failed on test task:\n" - f"{_scrub_host_paths(traceback.format_exc())}") - else: - obj_names = ["(no tasks to test on)"] - - ctx.iteration_proposals.augment_task_fn = result - ctx.iteration_proposals.augment_task_code = code - logging.info(f"Agent proposed augmentor adding objects: {obj_names}") - _save_proposal_code("propose_task_augmentor", code, obj_names, - args.get("description", "")) - return _text_result( - f"Successfully proposed augmentor. Test added objects: {obj_names}" - ) - - @tool( - "propose_processes", - "Propose new causal processes. Code must define " - "`proposed_processes` (a list of CausalProcess objects).", - { - "type": "object", - "properties": { - "code": { - "type": "string", - "description": "Python code defining proposed_processes" - }, - "description": { - "type": "string", - "description": "What these processes model" - }, - }, - "required": ["code", "description"], - }, - ) - async def propose_processes(args: Dict[str, Any]) -> Dict[str, Any]: - if not ToolSurfaceConfig.from_cfg().propose_processes: - return _error_result("Process proposals are disabled.") - code = args["code"] - exec_ctx = build_exec_context(ctx.types, ctx.predicates, ctx.options) - result, error = exec_code_safely(code, exec_ctx, "proposed_processes") - if error: - return _error_result(f"Code execution failed:\n{error}") - if not isinstance(result, (list, set)): - return _error_result( - f"proposed_processes must be a list/set, got {type(result)}") - for proc in result: - if not isinstance(proc, CausalProcess): - return _error_result( - f"Each item must be a CausalProcess, got {type(proc)}") - proposed = set(result) - ctx.iteration_proposals.proposed_processes |= proposed - names = [p.name for p in proposed] - logging.info(f"Agent proposed processes: {names}") - _save_proposal_code("propose_processes", code, names, - args.get("description", "")) - return _text_result( - f"Successfully proposed {len(proposed)} processes: {names}") - - @tool( - "propose_options", - "Propose new parameterized options. Code must define " - "`proposed_options` (a list of ParameterizedOption objects).", - { - "type": "object", - "properties": { - "code": { - "type": "string", - "description": "Python code defining proposed_options" - }, - "description": { - "type": "string", - "description": "What these options do" - }, - }, - "required": ["code", "description"], - }, - ) - async def propose_options(args: Dict[str, Any]) -> Dict[str, Any]: - if not ToolSurfaceConfig.from_cfg().propose_options: - return _error_result("Option proposals are disabled.") - if ctx.proposals_disabled: - return _error_result( - "Proposals are disabled during test-time solving. " - "Options can only be proposed during learning.") - code = args["code"] - exec_ctx = build_exec_context(ctx.types, - ctx.predicates, - ctx.options, - extra_context=ctx.skill_factory_context) - result, error = exec_code_safely(code, exec_ctx, "proposed_options") - if error: - return _error_result(f"Code execution failed:\n{error}") - if not isinstance(result, (list, set)): - return _error_result( - f"proposed_options must be a list/set, got {type(result)}") - for opt in result: - if not isinstance(opt, ParameterizedOption): - return _error_result( - f"Each item must be a ParameterizedOption, " - f"got {type(opt)}") - proposed = set(result) - ctx.iteration_proposals.proposed_options |= proposed - ctx.options |= proposed - names = [o.name for o in proposed] - # Save proposal code to sandbox for each option - for opt in proposed: - _save_option_to_sandbox(ctx, opt.name, code) - logging.info(f"Agent proposed options: {names}") - _save_proposal_code("propose_options", code, names, - args.get("description", "")) - return _text_result( - f"Successfully proposed {len(proposed)} options: {names}") - - return { - "propose_types": propose_types, - "propose_predicates": propose_predicates, - "propose_task_augmentor": propose_task_augmentor, - "propose_processes": propose_processes, - "propose_options": propose_options, - } - - -def _build_retraction_tools(ctx: ToolContext, _text_result: Callable, - tool: Callable) -> Dict[str, Any]: - """Retraction tools (remove agent-proposed abstractions).""" - - @tool( - "retract_abstractions", - "Remove previously proposed abstractions that are no longer needed. " - "Specify names of predicates, processes, options, or helper types to " - "remove, and/or set clear_task_augmentor to remove the augmentor.", - { - "type": "object", - "properties": { - "predicate_names": { - "type": "array", - "items": { - "type": "string" - }, - "description": "Names of predicates to remove", - }, - "process_names": { - "type": "array", - "items": { - "type": "string" - }, - "description": "Names of processes to remove", - }, - "option_names": { - "type": "array", - "items": { - "type": "string" - }, - "description": "Names of options to remove", - }, - "type_names": { - "type": "array", - "items": { - "type": "string" - }, - "description": "Names of helper types to remove", - }, - "clear_task_augmentor": { - "type": "boolean", - "description": - "Set to true to remove the object augmentor", - }, - "reason": { - "type": "string", - "description": "Why these abstractions are being removed", - }, - }, - "required": ["reason"], - }, - ) - async def retract_abstractions(args: Dict[str, Any]) -> Dict[str, Any]: - if ctx.proposals_disabled: - return _error_result( - "Retractions are disabled during test-time solving. " - "Abstractions can only be retracted during learning.") - pred_names = set(args.get("predicate_names") or []) - proc_names = set(args.get("process_names") or []) - opt_names = set(args.get("option_names") or []) - type_names = set(args.get("type_names") or []) - clear_augmentor = bool(args.get("clear_task_augmentor", False)) - - if not any( - [pred_names, proc_names, opt_names, type_names, clear_augmentor]): - return _text_result("Nothing to retract.") - - lines = [f"Retracting abstractions. Reason: {args['reason']}"] - - if pred_names: - existing = {p.name for p in ctx.predicates} - unknown = pred_names - existing - valid = pred_names & existing - ctx.iteration_proposals.retract_predicate_names |= valid - lines.append(f"Predicates to retract: {sorted(valid)}") - if unknown: - lines.append(f" (unknown, ignored: {sorted(unknown)})") - - if proc_names: - existing = {p.name for p in ctx.processes} - unknown = proc_names - existing - valid = proc_names & existing - ctx.iteration_proposals.retract_process_names |= valid - lines.append(f"Processes to retract: {sorted(valid)}") - if unknown: - lines.append(f" (unknown, ignored: {sorted(unknown)})") - - if opt_names: - existing = {o.name for o in ctx.options} - unknown = opt_names - existing - valid = opt_names & existing - ctx.iteration_proposals.retract_option_names |= valid - ctx.options = {o for o in ctx.options if o.name not in valid} - lines.append(f"Options to retract: {sorted(valid)}") - if unknown: - lines.append(f" (unknown, ignored: {sorted(unknown)})") - - if type_names: - existing = {t.name for t in ctx.types} - unknown = type_names - existing - valid = type_names & existing - ctx.iteration_proposals.retract_type_names |= valid - lines.append(f"Helper types to retract: {sorted(valid)}") - if unknown: - lines.append(f" (unknown, ignored: {sorted(unknown)})") - - if clear_augmentor: - ctx.iteration_proposals.retract_task_augmentor = True - lines.append("Object augmentor will be cleared.") - - logging.info(f"Agent retraction request: {args}") - return _text_result("\n".join(lines)) - - return { - "retract_abstractions": retract_abstractions, - } diff --git a/predicators/agent_sdk/tools/python_exec.py b/predicators/agent_sdk/tools/python_exec.py index 024130130..e9be16038 100644 --- a/predicators/agent_sdk/tools/python_exec.py +++ b/predicators/agent_sdk/tools/python_exec.py @@ -1,7 +1,7 @@ -"""Shared python-exec tool core behind run_python and explore_python.""" +"""The ``run_python`` tool core, shared by the solve and synthesis sessions.""" import os import time -from typing import Any, Callable, Dict, Optional +from typing import Any, Callable, Dict, Optional, Tuple from predicators.agent_sdk.config import ToolSurfaceConfig from predicators.agent_sdk.tools.budget import _arm_budget_watchdog, \ @@ -11,6 +11,27 @@ _screen_text_for_sandbox_escape, _scrub_host_paths +def _resolve_sandbox_file( + path: str, sandbox_dir: Optional[str]) -> Tuple[str, Optional[str]]: + """Resolve a ``path`` argument to a readable file inside the sandbox. + + Relative paths resolve against the sandbox (the agent's working + directory); with a sandbox the resolved file must lie inside it (the + PreToolUse hook that confines Read/Write does not see MCP calls). + Returns ``(host_path, None)`` or ``("", error_message)``. + """ + base = sandbox_dir or os.getcwd() + host = path if os.path.isabs(path) else os.path.join(base, path) + resolved = os.path.realpath(host) + if sandbox_dir is not None: + root = os.path.realpath(sandbox_dir) + if resolved != root and not resolved.startswith(root + os.sep): + return "", f"`path` must stay inside the sandbox: {path}" + if not os.path.isfile(resolved): + return "", f"`path` is not a file: {path}" + return resolved, None + + def _make_python_exec_tool( tool: Callable, *, @@ -25,16 +46,19 @@ def _make_python_exec_tool( ) -> Any: """Build a code-execution MCP tool over a persistent namespace. - Shared core behind the synthesis-phase ``run_python`` (namespace = - trajectory data) and the solve-phase ``explore_python`` (namespace = - the ``BeliefProbe`` exploration facade): sandbox-escape screening, in- - process ``exec`` with stdout capture, and oversize-output spill to + One tool, two namespaces: the solve-phase instance binds the + ``BeliefProbe`` facade over the deployed belief model, the synthesis + instance binds the fit data plus the probe over the candidate + simulator. Shared here: sandbox-escape screening, in-process ``exec`` + with stdout capture, and oversize-output spill to ``/tool_outputs//``. The namespace persists - across calls, so agents can define helpers once and reuse them. + across calls, so agents can define helpers once and reuse them; + ``path`` runs a ``.py`` file from the sandbox in that same namespace, + so helpers and sweeps can be developed as files with Write/Edit. ``budget_ctx`` (the solve session's ToolContext) opts the tool into wall-clock budgeting: each call arms the per-call deadline - (``agent_sdk_explore_python_call_timeout``) that probe sim calls + (``agent_sdk_python_call_timeout``) that probe sim calls enforce cooperatively, a call arriving after the attempt deadline is refused with a submit-now message, and every result carries a ``[budget]`` footer (attempt time + rollout counts) so sweeps have a @@ -83,15 +107,34 @@ def _make_python_exec_tool( "code": { "type": "string", "description": "Python code to execute.", - } + }, + "path": { + "type": + "string", + "description": + ("Path of a .py file inside the sandbox to execute in " + "the same persistent namespace (instead of `code`)."), + }, }, - "required": ["code"], }, ) async def python_exec(args: Dict[str, Any]) -> Dict[str, Any]: # pylint: disable-next=import-outside-toplevel from predicators.agent_sdk.belief_probe import ProbeBudgetExceeded - code = args["code"] + code = args.get("code") or "" + path = args.get("path") or "" + if bool(code) == bool(path): + return text_result( + "Error: pass exactly one of `code` (inline Python) or " + "`path` (a .py file inside the sandbox).") + label = f"<{name}>" + if path: + resolved, err = _resolve_sandbox_file(path, sandbox_dir) + if err is not None: + return text_result(f"Error: {err}") + with open(resolved, encoding="utf-8") as f: + code = f.read() + label = path # The code execs in-process with full filesystem access, and the # sandbox's PreToolUse file-path hook does not cover MCP tools, so # screen the code here for out-of-sandbox reads / source @@ -114,11 +157,10 @@ async def python_exec(args: Dict[str, Any]) -> Dict[str, Any]: return text_result( "The attempt's wall-clock exploration budget is " "exhausted - this call was not run. Submit your single " - "best plan NOW via evaluate_option_plan on the current " + "best plan NOW via submit_plan on the current " "task (omit task_idx)." + _budget_footer(budget_ctx, rollouts_before)) - call_timeout = ToolSurfaceConfig.from_cfg( - ).explore_python_call_timeout + call_timeout = ToolSurfaceConfig.from_cfg().python_call_timeout if budget_ctx.probe_option_model_provider is not None: # Synthesis sessions probe the CANDIDATE simulator, whose # rollouts are far slower than belief-sim ones and whose @@ -126,8 +168,8 @@ async def python_exec(args: Dict[str, Any]) -> Dict[str, Any]: # can exceed the solve-tuned cap, so synthesis is exempt # from the per-call limit. call_timeout = 0.0 - budget_ctx.explore_call_deadline = (time.monotonic() + call_timeout - if call_timeout > 0 else None) + budget_ctx.python_call_deadline = (time.monotonic() + call_timeout + if call_timeout > 0 else None) def _footer() -> str: if budget_ctx is None: @@ -140,8 +182,8 @@ def _footer() -> str: watchdog_disarm: Optional[Callable[[], None]] = None wd_deadlines = [] if budget_ctx is not None: - if budget_ctx.explore_call_deadline is not None: - wd_deadlines.append(budget_ctx.explore_call_deadline) + if budget_ctx.python_call_deadline is not None: + wd_deadlines.append(budget_ctx.python_call_deadline) if (budget_ctx.attempt_deadline is not None and not budget_ctx.capture_best_effort_plan): wd_deadlines.append(budget_ctx.attempt_deadline) @@ -155,7 +197,7 @@ def _footer() -> str: old_stdout = sys.stdout sys.stdout = captured = io.StringIO() try: - exec(code, exec_ns) # pylint: disable=exec-used + exec(compile(code, label, "exec"), exec_ns) # pylint: disable=exec-used except ProbeBudgetExceeded as e: partial = captured.getvalue() prefix = f"{partial}\n" if partial else "" @@ -177,7 +219,7 @@ def _footer() -> str: watchdog_disarm() sys.stdout = old_stdout if budget_ctx is not None: - budget_ctx.explore_call_deadline = None + budget_ctx.python_call_deadline = None output = captured.getvalue() if not output: diff --git a/predicators/agent_sdk/tools/registry.py b/predicators/agent_sdk/tools/registry.py index 41b6c144a..b4ef0ff78 100644 --- a/predicators/agent_sdk/tools/registry.py +++ b/predicators/agent_sdk/tools/registry.py @@ -1,8 +1,6 @@ """Tool-name rosters and the session tool-list surface.""" from typing import Any, Dict, List, Optional, Sequence -from predicators.agent_sdk.config import ToolSurfaceConfig - MCP_SERVER_NAME = "predicator_tools" # Built-in Claude tools available to the sandboxed agent. @@ -22,87 +20,32 @@ "TaskList", ] -INSPECTION_TOOL_NAMES = [ - "inspect_types", - "inspect_predicates", - "inspect_processes", - "inspect_options", - "inspect_trajectories", - "inspect_train_tasks", - "inspect_planning_results", - "inspect_past_proposals", -] -PROPOSAL_TOOL_NAMES = [ - "propose_types", - "propose_predicates", - "propose_task_augmentor", - "propose_processes", - "propose_options", -] -RETRACTION_TOOL_NAMES = [ - "retract_abstractions", -] TESTING_TOOL_NAMES = [ - "evaluate_predicate_on_trajectory", - "evaluate_option_plan", + "submit_plan", # Closed-loop policy mode (agent_solve_policy_mode): validates and # captures the agent-written policy.py. Only offered on solve # rosters when the mode is on. - "evaluate_policy", + "submit_policy", ] -PLANNING_TOOL_NAMES = [ - "generate_bilevel_plan", - "generate_abstract_plan", - "refine_plan_sketch", -] -# Solve-phase exploration: ``explore_python`` over the BeliefProbe facade -# (predicators/agent_sdk/belief_probe.py). Named distinctly from the -# synthesis-phase ``run_python`` (same execution core, different -# namespace) so sessions, transcripts, and log greps never conflate the -# two capabilities. Built only when agent_planner_use_explore_python -# is on - the legacy ``tool_names=None`` surface ("all static MCP -# tools") must not hand baseline arms an ungated code-execution tool. +# The one code-execution tool. Solve sessions get the static instance +# built by ``create_mcp_tools`` (namespace = the BeliefProbe facade over +# the deployed belief model, predicators/agent_sdk/belief_probe.py); +# synthesis sessions attach their own instance under the same name +# (fit data + the probe over the candidate simulator), which replaces +# the static one at assembly. Offered to every session that has a +# simulator to probe (see ``AgentModelFreeApproach._get_solve_tool_names``). EXPLORATION_TOOL_NAMES = [ - "explore_python", -] -# Solve-journal writing (agent_solve_use_journal): agent-authored -# lessons for future fresh-context attempts. Read side is prompt -# injection, so this is the only journal tool. -JOURNAL_TOOL_NAMES = [ - "record_journal", + "run_python", ] - - -def explore_python_replaces_tools() -> bool: - """Whether explore_python replaces the tools it subsumes this session. - - The single definition of the tool-roster policy (refine_plan_sketch - -> ``sim.refine``; inspect_trajectories / inspect_train_tasks -> - ``trajectories`` / ``describe_trajectory`` / ``sim.task()`` in the - probe namespace): the approaches' tool lists and every prompt - surface read this one predicate, so the offered tools and the - guidance that names them cannot drift apart. - """ - surface_cfg = ToolSurfaceConfig.from_cfg() - return (surface_cfg.use_explore_python - and not surface_cfg.explore_python_keep_replaced_tools) - - -ALL_TOOL_NAMES = (INSPECTION_TOOL_NAMES + PROPOSAL_TOOL_NAMES + - RETRACTION_TOOL_NAMES + TESTING_TOOL_NAMES + - PLANNING_TOOL_NAMES + EXPLORATION_TOOL_NAMES + - JOURNAL_TOOL_NAMES) - -# Names of tools returned by ``create_synthesis_tools`` (sim-learning) -# and ``create_predicate_synthesis_tools`` (predicate invention). These -# tools are produced by ``AgentSessionMixin._build_synthesis_mcp_tools`` -# and joined to the static MCP set at session-open time; the constants -# exist so callers / tests can refer to them without typing the strings -# twice. ``tests/agent_sdk/test_tool_registry.py`` asserts that the -# factory outputs match these tuples. +ALL_TOOL_NAMES = TESTING_TOOL_NAMES + EXPLORATION_TOOL_NAMES + +# Name of the tool ``create_synthesis_tools`` builds for a synthesis +# session (the same ``run_python`` name as the solve-phase instance - +# see EXPLORATION_TOOL_NAMES). ``tests/agent_sdk/test_tool_registry.py`` +# asserts that the factory output matches this tuple. Predicate and +# sampler drafts are loaded through the probe (``sim.predicates()`` / +# ``sim.samplers()``), not through tools. SYNTHESIS_TOOL_NAMES = ("run_python", ) -PREDICATE_SYNTHESIS_TOOL_NAMES = ("evaluate_predicate_quality", ) -SAMPLER_SYNTHESIS_TOOL_NAMES = ("evaluate_sampler", ) def get_allowed_tool_list(tool_names: Optional[List[str]] = None) -> List[str]: diff --git a/predicators/agent_sdk/tools/results.py b/predicators/agent_sdk/tools/results.py index caf770c03..13553466c 100644 --- a/predicators/agent_sdk/tools/results.py +++ b/predicators/agent_sdk/tools/results.py @@ -5,7 +5,6 @@ from typing import Any, Callable, Dict, Optional from predicators.agent_sdk.config import RefinementConfig -from predicators.agent_sdk.tools.context import ToolContext def session_log_filename(query_count: int, @@ -53,7 +52,7 @@ def _make_coercing_tool(tool: Callable) -> Callable: Harness-side JSON-schema validation rejects ``"0"`` for an ``integer`` property before the handler ever runs (agents lost - whole tools to it - ``inspect_trajectories`` went 0-for-6 in + whole tools to it - a trajectory-inspection tool went 0-for-6 in run_20260717_154753 seed2), and which tools accept strings was inconsistent. This wrapper loosens every top-level ``integer`` / ``number`` property to also accept a string, then coerces the value @@ -169,21 +168,3 @@ def _text(text: str) -> Dict[str, Any]: return _text_result("\n".join(parts)) return _text - - -def _save_option_to_sandbox(ctx: ToolContext, option_name: str, - code: str) -> Optional[str]: - """Save option source code to sandbox/proposed_code/.py. - - Returns the relative path (e.g. ``./proposed_code/Pick.py``) or None - if the sandbox directory is not set. - """ - if ctx.sandbox_dir is None: - return None - proposed_dir = os.path.join(ctx.sandbox_dir, "proposed_code") - os.makedirs(proposed_dir, exist_ok=True) - filename = f"{option_name}.py" - filepath = os.path.join(proposed_dir, filename) - with open(filepath, "w", encoding="utf-8") as f: - f.write(code) - return f"./proposed_code/{filename}" diff --git a/predicators/agent_sdk/tools/sampler_synthesis.py b/predicators/agent_sdk/tools/sampler_synthesis.py index 1486b3c91..0cbbc231b 100644 --- a/predicators/agent_sdk/tools/sampler_synthesis.py +++ b/predicators/agent_sdk/tools/sampler_synthesis.py @@ -1,5 +1,4 @@ -"""Sampler-synthesis tools (create_sampler_synthesis_tools).""" -import os +"""The ``sim.samplers()`` loader for sampler-synthesis sessions.""" from typing import Any, Callable, Dict, List, Optional, Tuple import numpy as np @@ -8,23 +7,21 @@ load_learned_samplers from predicators.agent_sdk.synthesis_backend import SamplerSynthesisBackend from predicators.agent_sdk.tools.params_view import _ParamsView -from predicators.agent_sdk.tools.results import _make_coercing_tool, \ - _make_spilling_text_result from predicators.agent_sdk.tools.sandbox_guard import _scrub_host_paths from predicators.agent_sdk.tools.snapshots import _ArtifactSnapshotter from predicators.settings import CFG from predicators.structs import Object -def create_sampler_synthesis_tools( +def make_sampler_loader( samplers_file: str, samplers_versions_dir: str, approach: SamplerSynthesisBackend, cycle_index_provider: Optional[Callable[[], int]] = None, -) -> list: - """Create the per-skill sampler-synthesis tool. +) -> Callable[[], str]: + """Build the ``sim.samplers()`` loader for one synthesis session. - Returns ``[evaluate_sampler]``. On each call the tool loads + On each call the loader loads ``samplers.py`` fresh (snapshotting into ``samplers_versions_dir``), validates the ``LEARNED_SAMPLERS`` dict (option name -> callable), installs it into ``approach._synthesized_samplers`` so refinement @@ -40,13 +37,9 @@ def create_sampler_synthesis_tools( # pylint: disable=import-outside-toplevel import traceback # pylint: disable=redefined-outer-name,reimported - from claude_agent_sdk import tool as _sdk_tool - tool = _make_coercing_tool(_sdk_tool) - from predicators.code_sim_learning.fit_space import ParamSpec # pylint: enable=import-outside-toplevel - _text = _make_spilling_text_result(os.path.dirname(samplers_file)) _snapshotter = _ArtifactSnapshotter( live_file=samplers_file, versions_dir=samplers_versions_dir, @@ -136,43 +129,18 @@ def _sanity_check(name: str, fn: Any) -> str: return (f" {name}: OK — {n_draws} draws, {in_box}/{n_draws} " f"within the params box.") - @tool( - "evaluate_sampler", - "Load LEARNED_SAMPLERS (fresh from `samplers.py`) and install " - "them as the per-skill samplers used by refinement. Each entry " - "maps an option name to a function " - "(state, subgoal_atoms, rng, objects) -> params array (the same " - "signature as the env's NSRT samplers); refinement calls it " - "instead of drawing uniformly so the sampler can aim continuous " - "params at the step's subgoal, then clips the result to the box. " - "At steps with no subgoal annotation, subgoal_atoms is the empty " - "set - the sampler must handle that without crashing. A sketch " - "step carrying a `~ [widths]` region annotation bypasses the " - "sampler (precedence: per-step region > per-skill sampler > " - "uniform); samplers are the reusable cross-task prior, regions a " - "per-call override. " - "Reports a per-option sanity check (return shape + within-box) " - "over a representative train-task state. After loading, the " - "samplers used by sim.refine are updated — so call " - "this any time you edit samplers.py before re-running " - "refinement. Snapshots samplers.py into samplers_versions/; " - "output tagged [cycle_XXX_vers_YYY].", - { - "type": "object", - "properties": {}, - }, - ) - async def evaluate_sampler(args: Dict[str, Any]) -> Dict[str, Any]: - del args + def sampler_report() -> str: + """Reload samplers.py, install LEARNED_SAMPLERS, and report the per- + option sanity check.""" try: samplers, version_tag, err, warnings = ( _snapshot_and_load_samplers(samplers_file)) except Exception: # pylint: disable=broad-except - return _text(f"Error loading samplers.py:\n" - f"{_scrub_host_paths(traceback.format_exc())}") + return (f"Error loading samplers.py:\n" + f"{_scrub_host_paths(traceback.format_exc())}") if err is not None: - return _text(err) + return err prefix = f"[{version_tag}]" lines = [ @@ -189,7 +157,7 @@ async def evaluate_sampler(args: Dict[str, Any]) -> Dict[str, Any]: lines.append("") lines.append("LEARNED_SAMPLERS is empty — add " "{\"OptionName\": fn} entries to samplers.py.") - return _text("\n".join(lines)) + return "\n".join(lines) lines.append("") lines.append("Sanity check (representative train-task state):") @@ -199,6 +167,6 @@ async def evaluate_sampler(args: Dict[str, Any]) -> Dict[str, Any]: lines.append("Now call sim.refine with a sketch that " "uses these options to measure the samples-to-refine " "improvement.") - return _text("\n".join(lines)) + return "\n".join(lines) - return [evaluate_sampler] + return sampler_report diff --git a/predicators/agent_sdk/tools/synthesis.py b/predicators/agent_sdk/tools/synthesis.py index 766e44b8a..cf1b34984 100644 --- a/predicators/agent_sdk/tools/synthesis.py +++ b/predicators/agent_sdk/tools/synthesis.py @@ -399,7 +399,12 @@ def _evaluate_rollout_fit(rules: list, n: v for n, v in outcome.fit_result.point_estimate.items() if n in rule_names - }, version_tag, simulator_file) + }, + version_tag, + simulator_file, + fit_result=outcome.fit_result, + sse=post_sse, + applied_physical=dict(applied)) if hasattr(approach, "_record_sysid_diagnostics"): approach._record_sysid_diagnostics( # pylint: disable=protected-access ident_report, physical_names, outcome.num_survivors, @@ -492,7 +497,7 @@ def _evaluate_rollout_fit(rules: list, # visible to probe sweeps. Since the fit/refine/forward-validate # surfaces all live on `sim` now, the probe is unconditional in # synthesis (there is no other validation surface). Shared blurb, - # so the wording cannot drift from the solve-phase explore_python + # so the wording cannot drift from the solve-phase run_python # surface. # pylint: disable-next=import-outside-toplevel from predicators.agent_sdk.tools.exploration import belief_probe_blurb @@ -511,7 +516,8 @@ def _evaluate_rollout_fit(rules: list, tool, name="run_python", description=( - "Execute Python code for ad-hoc data exploration. Available " + "Execute Python code (`code`, or `path` to a .py file you wrote " + "in the sandbox) for ad-hoc data exploration. Available " "variables: trajectories (List[LowLevelTrajectory]; each has " "`is_demo`, `train_task_idx`, `states`, `actions`), train_tasks " "(List[Task]; each has `init`, `goal`, `goal_holds(state)`), " @@ -705,7 +711,11 @@ def run_fit(path: Optional[str] = None, if canonical and approach is not None: # Deploy: the probe runs at exactly these values from now on. approach._publish_probe_fit( # pylint: disable=protected-access - dict(fitted_params), version_tag, p) + dict(fitted_params), + version_tag, + p, + fit_result=fit_result, + sse=post_sse) if pre_sse > 0: pct = (pre_sse - post_sse) / pre_sse * 100 pct_str = f"({pct:+.1f}% vs init)" diff --git a/predicators/agent_sdk/tools/testing.py b/predicators/agent_sdk/tools/testing.py index 20d7fa752..5739e0c59 100644 --- a/predicators/agent_sdk/tools/testing.py +++ b/predicators/agent_sdk/tools/testing.py @@ -1,4 +1,4 @@ -"""Testing tools, including the evaluate_option_plan capture surface.""" +"""Testing tools, including the submit_plan capture surface.""" import contextlib import functools import logging @@ -9,15 +9,14 @@ from predicators import utils from predicators.agent_sdk import bilevel_sketch -from predicators.agent_sdk.config import RefinementConfig, ToolSurfaceConfig, \ - ValidationConfig +from predicators.agent_sdk.config import RefinementConfig, ValidationConfig from predicators.agent_sdk.parallel_rollouts import \ prefetch_parallel as _prefetch_parallel from predicators.agent_sdk.tools.budget import _budget_footer from predicators.agent_sdk.tools.capture import BestEffortReason, \ CaptureDecision, _decide_capture from predicators.agent_sdk.tools.context import ToolContext, \ - _capture_task_key, absolute_rollout_seed, decorrelated_rollout_seed + _capture_task_key, decorrelated_rollout_seed from predicators.agent_sdk.tools.results import _error_result from predicators.agent_sdk.tools.scene import format_object_poses, \ render_scene_image @@ -65,8 +64,8 @@ def _parameter_margin_sweep( """Margin sweep over BOTH parameter-uncertainty sources of one. capture-eligible submission - the single code path behind the - physics-margin and rule-parameter gates of ``evaluate_option_plan`` - and ``evaluate_policy``. + physics-margin and rule-parameter gates of ``submit_plan`` + and ``submit_policy``. The execution repeats before this all run AT the fitted parameters, so they cannot see a submission whose success band excludes the @@ -163,127 +162,40 @@ def _member_rollout(point: Dict[str, float]) -> Tuple[bool, str]: def _build_testing_tools(ctx: ToolContext, _text_result: Callable, tool: Callable) -> Dict[str, Any]: - """Evaluation tools (run predicates / option plans against tasks).""" - - @tool( - "evaluate_predicate_on_trajectory", - "Evaluate a predicate's truth value across timesteps in a trajectory", - { - "type": "object", - "properties": { - "predicate_name": { - "type": "string", - "description": "Name of the predicate to test" - }, - "traj_idx": { - "type": "integer", - "description": "Trajectory index" - }, - "object_names": { - "type": "array", - "items": { - "type": "string" - }, - "description": "Object names to ground the predicate on" - }, - }, - "required": ["predicate_name", "traj_idx", "object_names"], - }, - ) - async def evaluate_predicate_on_trajectory( - args: Dict[str, Any]) -> Dict[str, Any]: - pred_name = args["predicate_name"] - traj_idx = args["traj_idx"] - object_names = args["object_names"] - - # Find the predicate - all_preds = ctx.predicates | ctx.iteration_proposals.proposed_predicates - pred = None - for p in all_preds: - if p.name == pred_name: - pred = p - break - if pred is None: - return _error_result(f"Predicate '{pred_name}' not found.") - - all_trajs = ctx.offline_trajectories + ctx.online_trajectories - if not all_trajs: - return _error_result("No trajectories available yet.") - if traj_idx < 0 or traj_idx >= len(all_trajs): - return _error_result(f"Invalid traj_idx {traj_idx}. " - f"Available: 0-{len(all_trajs)-1}") - - traj = all_trajs[traj_idx] - - # Find objects by name - objects = [] - for name in object_names: - found = None - for obj in traj.states[0]: - if obj.name == name: - found = obj - break - if found is None: - avail = [o.name for o in sorted(traj.states[0], key=str)] - return _error_result(f"Object '{name}' not found in " - f"trajectory {traj_idx}. " - f"Available: {avail}") - objects.append(found) - - results = [] - for t_step, state in enumerate(traj.states): - try: - val = pred.holds(state, objects) - results.append(f"t={t_step}: {val}") - except Exception as e: # pylint: disable=broad-except - results.append(f"t={t_step}: ERROR ({e})") - - return _text_result( - f"Predicate {pred_name}({', '.join(object_names)}) " - f"over trajectory {traj_idx}:\n" + "\n".join(results)) + """Evaluation tools (option plans / policies against tasks).""" # Tool descriptions bake config values at BUILD time (session open); # the handlers below re-read config at CALL time. _gs_eval_doc = ( "Runs your exact params with NO sampling (a `~` ground-sampler " "annotation - `~ [w1, w2]` region or `~ my_sampler` - is accepted " - "but IGNORED here; only refine_plan_sketch uses it). " + "but IGNORED here; only `sim.refine` uses it). " if RefinementConfig.from_cfg().ground_samplers else "Runs your exact params with NO sampling. ") - # When the session carries explore_python, the two surfaces divide - # cleanly: exploration (modified states, partial plans, sweeps) - # belongs in the probe, and this tool is the SUBMISSION path. - _probe_split_doc = ( - " This tool always runs from the task's TRUE initial state and is " - "the ONLY path that captures an answer: do your exploration " - "(modified states, partial plans, parameter sweeps) in " - "explore_python, then validate and SUBMIT the final plan here." - if ToolSurfaceConfig.from_cfg().use_explore_python else "") - @tool( - "evaluate_option_plan", - "Execute a fully-specified plan on a task via the option model and " - "report the result at each step. `plan` is text — one option per " - "line, same grammar as refine_plan_sketch: " - "`Option(obj1:type1, obj2:type2)[param1, param2] -> {Atom(obj:type), " - "...}` (typed object refs; EXACT continuous params in `[]`, `[]` for " - "none; optional `-> {atoms}` subgoals, prefix NOT to require false). " - + _gs_eval_doc + "Use include_states/" - "include_atoms to control output. If the plan reaches the goal on the " - "CURRENT task (omit task_idx), it is captured as your answer, and the " - "per-step subgoals make it execute closed-loop (monitored, with " - "replan-on-divergence). Capture is gated: a goal-reaching plan is " - "re-run several times (simulation varies across runs; each rollout " - "reports the motion-planner seed it ran at) and a FLAKY plan is " - "reported instead of captured - add margin and resubmit. " - "`validation_rollouts` requests a STRICTER gate for this " - "submission (more rollouts; never fewer than configured). " - "`rollout_seed` re-runs the plan at that exact planner seed " - "with full per-step reporting - use it to reproduce and debug a " - "reported failed rollout; combine with validation_rollouts=N for " - "N seeded trials at consecutive seeds. A seeded run is diagnostic " - "only and is never captured. " + "submit_plan", + "SUBMIT a fully-specified plan as your answer for the CURRENT task. " + "`plan` is text - one option per line, same grammar as `sim.run` / " + "`sim.refine`: `Option(obj1:type1, obj2:type2)[param1, param2] -> " + "{Atom(obj:type), ...}` (typed object refs; EXACT continuous params " + "in `[]`, `[]` for none; optional `-> {atoms}` subgoals, prefix NOT " + "to require false). " + _gs_eval_doc + + "The plan is rolled out from the task's TRUE initial state through " + "the belief model and reported step by step (include_states/" + "include_atoms control the report). If it reaches the goal it is " + "captured as your answer, and the per-step subgoals make it execute " + "closed-loop (monitored, with replan-on-divergence). Capture is " + "gated: a goal-reaching plan is re-run several times (simulation " + "varies across runs; each rollout reports the motion-planner seed " + "it ran at) and a FLAKY plan is reported instead of captured - " + "reproduce a failed rollout with `sim.run(plan, seed=S)` in " + "run_python, add margin, and resubmit. `validation_rollouts` " + "requests a STRICTER gate for this submission (more rollouts; never " + "fewer than configured). This is the ONLY path that captures an " + "answer: explore (other tasks, modified states, partial plans, " + "parameter sweeps, seeded reproductions) with `sim` in run_python, " + "then submit the final plan here. " "When identified physical parameters are active, it is also re-run " "at a grid of perturbations spanning +-1 sigma of those parameters " "(the physics fit's own uncertainty); a PARAM-SENSITIVE plan is " @@ -292,7 +204,7 @@ async def evaluate_predicate_on_trajectory( "When the task has an evaluator, a goal-reaching plan the evaluator " "still scores as a non-solve (no success credit in its reward) is " "NOT captured (the real env applies the same scoring, so it could " - "never count as a solve)." + _probe_split_doc, + "never count as a solve).", { "type": "object", "properties": { @@ -321,13 +233,6 @@ async def evaluate_predicate_on_trajectory( "Include atoms added/deleted after each step", "default": True }, - "task_idx": { - "type": - "integer", - "description": - "Train task index to test on. Omit to use " - "the current solve-time task." - }, "validation_rollouts": { "type": "integer", @@ -336,27 +241,13 @@ async def evaluate_predicate_on_trajectory( "rollouts a goal-reaching submission must pass. The " "effective count is max(configured gate, this) - it can " "raise the gate but never lower it. Use before " - "committing a plan you suspect is marginal. Combined " - "with rollout_seed=S it instead runs exactly this many " - "DIAGNOSTIC trials at planner seeds S, S+1, ... (each " - "outcome reported with its seed; never captured).", - }, - "rollout_seed": { - "type": - "integer", - "description": - "Diagnostic: run the plan at this exact motion-planner " - "seed (as reported per rollout in validation output) " - "with full per-step reporting, to reproduce a failed " - "validation rollout. Add validation_rollouts=N to run " - "N trials at seeds S, S+1, ..., like sim.run(plan, " - "trials=N, seed=S). A seeded run is never captured.", + "committing a plan you suspect is marginal.", }, }, "required": ["plan"], }, ) - async def evaluate_option_plan(args: Dict[str, Any]) -> Dict[str, Any]: + async def submit_plan(args: Dict[str, Any]) -> Dict[str, Any]: refine_cfg = RefinementConfig.from_cfg() validation_cfg = ValidationConfig.from_cfg() ctx.test_call_id += 1 @@ -369,46 +260,40 @@ async def evaluate_option_plan(args: Dict[str, Any]) -> Dict[str, Any]: if ctx.option_model is None: return _error_result("No option model available in ToolContext.") - # Sync the option model's option map with all current options - # (GT + proposed) so it stays in sync after propose/retract. - all_options = ctx.options | ctx.iteration_proposals.proposed_options + all_options = ctx.options opt_map = {o.name: o for o in all_options} model = ctx.option_model model._name_to_parameterized_option = ( # type: ignore[attr-defined] # pylint: disable=protected-access opt_map) - task_idx = args.get("task_idx") plan_text = (args.get("plan") or "").strip() include_states = args.get("include_states", False) include_atoms = args.get("include_atoms", True) requested_rollouts = args.get("validation_rollouts") - diagnostic_seed = args.get("rollout_seed") if requested_rollouts is not None and (not isinstance( requested_rollouts, int) or requested_rollouts < 1): return _error_result( "validation_rollouts must be a positive integer.") - if diagnostic_seed is not None and not isinstance( - diagnostic_seed, int): - return _error_result("rollout_seed must be an integer.") - resolved, task_err = _resolve_task(ctx, task_idx) + # Always the CURRENT task from its true initial state: this is + # the submission path, and exploration on other tasks or from + # modified states lives on the probe (sim.run). + resolved, task_err = _resolve_task(ctx, None) if task_err is not None: return task_err assert resolved is not None task = resolved.task task_label = resolved.label - is_current = resolved.is_current lines = [f"Testing option plan on task {task_label}:"] saved_image_paths: List[str] = [] - all_predicates = (ctx.predicates - | ctx.iteration_proposals.proposed_predicates) + all_predicates = ctx.predicates if not plan_text: return _error_result("`plan` is required (option plan text).") # Parse the text plan into a sketch (options + objects + exact params + - # subgoals) using the SAME grammar/parser as refine_plan_sketch. + # subgoals) using the SAME grammar/parser as sim.refine. types = set(ctx.types) for opt in all_options: types.update(opt.types) @@ -490,11 +375,6 @@ def _report_step(i: int, outcome: Any) -> None: step_line += ("\n State:\n" + post.dict_str(indent=4, num_decimal_points=4)) lines.append(step_line) - # A seeded diagnostic rollout runs on a FRESH env (when the - # session provides one), but the renderer draws the shared - # session env - its stale scene would be misleading. - if diagnostic_seed is not None and diag_fresh_scope is not None: - return img_block = render_scene_image(ctx, f"step_{i}_{opt.name}") if img_block and img_block.get("saved_path"): saved_image_paths.append(img_block["saved_path"]) @@ -505,37 +385,13 @@ def _report_step(i: int, outcome: Any) -> None: # continue past a collision and report a goal that the real rollout — # which ends the episode at that failed option — never reaches. ctx.attempt_rollout_count += 1 - # A seeded diagnostic rollout reproduces a validation repeat - # faithfully: fresh env (when the session provides one) plus the - # requested planner seed. diag_fresh_scope is also read by - # _report_step to skip stale-env renders. - diag_fresh_scope = (ctx.validation_env_scope - if diagnostic_seed is not None - and validation_cfg.fresh_env else None) - if diagnostic_seed is not None: - trials_note = ( - f"; running {min(requested_rollouts, _MAX_REQUESTED_ROLLOUTS)}" - " diagnostic trials at consecutive seeds" - if requested_rollouts is not None and requested_rollouts > 1 - else "; validation repeats skipped") - lines.append( - f"DIAGNOSTIC rollout at planner seed {diagnostic_seed}" + - (" on a fresh simulator env" - if diag_fresh_scope is not None else "") + - f" - never captured{trials_note}. Resubmit without " - "rollout_seed to capture.") - with (diag_fresh_scope() if diag_fresh_scope is not None else - contextlib.nullcontext()), \ - absolute_rollout_seed(diagnostic_seed): - result = bilevel_sketch.execute_plan_forward( - task, - grounded_plan, - ctx.option_model, - predicates=all_predicates, - sketch=sketch_steps, - on_step=_report_step, - stop_on_failure=True) - + result = bilevel_sketch.execute_plan_forward(task, + grounded_plan, + ctx.option_model, + predicates=all_predicates, + sketch=sketch_steps, + on_step=_report_step, + stop_on_failure=True) final_atoms = utils.abstract(result.final_state, ctx.predicates) # Use the env's goal-check (its own classifiers); robust to invented # predicates that don't reuse env names. @@ -669,12 +525,7 @@ def _validation_rollout() -> Tuple[bool, str, List[Optional[State]]]: lines.append( f"NOTE: validation_rollouts={requested_rollouts} capped " f"at {_MAX_REQUESTED_ROLLOUTS}.") - # With rollout_seed the request means "this many diagnostic - # trials", exactly as asked - there is no capture gate to - # protect, so neither the configured gate nor the flaky - # escalation inflates it. - if diagnostic_seed is None: - n_rollouts = max(n_rollouts, capped_request) + n_rollouts = max(n_rollouts, capped_request) # Fresh env per validation rollout when the approach provides one: # repeats on the shared env are correlated (its reset cannot # reconstruct state exactly), so only fresh envs sample the same @@ -691,9 +542,9 @@ def _validation_rollout() -> Tuple[bool, str, List[Optional[State]]]: # excluded: they run under deliberately perturbed physics. passing_validation_posts: List[List[Optional[State]]] = [] base_planner_seed = CFG.seed - if (ctx.capture_goal_reaching_plans and is_current and goal_achieved + if (ctx.capture_goal_reaching_plans and goal_achieved and not evaluator_rejected and grounded_plan - and diagnostic_seed is None and n_rollouts > 1): + and n_rollouts > 1): # Run ALL validation rollouts even after a failure: the # per-rollout outcome list distinguishes failure modes (a # physics-tail fizzle vs. an IK stall vs. a certificate @@ -748,45 +599,6 @@ def _repeat_rollout( "across runs; repeats sample that execution " f"variability{fresh_note}).") - # Diagnostic trials: rollout_seed combined with - # validation_rollouts=N runs N rollouts at planner seeds - # S, S+1, ..., S+N-1 (rollout 1, reported step by step above, - # ran at S) - the same contract as explore_python's - # ``sim.run(plan, trials=N, seed=S)``. Reported only: a seeded - # run never captures and never arms the flaky escalation. - if (diagnostic_seed is not None and grounded_plan - and capped_request is not None and capped_request > 1): - r1_ok = (result.first_failure_idx is None and result.goal_reached) - if r1_ok: - r1_line = "goal reached" - elif result.first_failure_idx is not None: - r1_line = "FAILED - see the step report above" - else: - r1_line = "goal NOT reached" - diag_outcomes = [ - f"rollout 1 (planner seed {diagnostic_seed}): {r1_line}" - ] - for repeat_idx in range(2, capped_request + 1): - ctx.attempt_rollout_count += 1 - repeat_seed = diagnostic_seed + repeat_idx - 1 - with (fresh_scope() if fresh_scope is not None else - contextlib.nullcontext()), \ - absolute_rollout_seed(repeat_seed): - ok, why, _ = _validation_rollout() - if ok: - diag_outcomes.append(f"rollout {repeat_idx} (planner seed " - f"{repeat_seed}): goal reached") - else: - diag_outcomes.append(f"rollout {repeat_idx} (planner seed " - f"{repeat_seed}): FAILED - {why}") - n_ok_diag = sum(1 for o in diag_outcomes - if o.endswith("goal reached")) - per_diag = "\n".join(f" {o}" for o in diag_outcomes) - lines.append( - f"Diagnostic trials: {n_ok_diag}/{capped_request} reached " - f"the goal (planner seeds {diagnostic_seed}-" - f"{diagnostic_seed + capped_request - 1}):\n{per_diag}") - # Parameter-margin gates (see _parameter_margin_sweep): the # execution repeats above all run AT the fitted parameters, so # they cannot see a plan whose success band excludes the fit's @@ -795,8 +607,7 @@ def _repeat_rollout( param_sensitive_detail: Optional[str] = None margin_outcomes: List[str] = [] if (fresh_scope is not None and ctx.capture_goal_reaching_plans - and is_current and goal_achieved and not evaluator_rejected - and grounded_plan and diagnostic_seed is None + and goal_achieved and not evaluator_rejected and grounded_plan and flaky_detail is None): margin_outcomes, param_sensitive_detail, margin_note = \ _parameter_margin_sweep( @@ -825,16 +636,12 @@ def _stash_uncaptured_submission() -> None: # also documents the best-effort-mode semantics); the branches # below apply its ctx mutations and format its messages. capture_outcome = _decide_capture( - # A seeded diagnostic rollout is never captured: letting the - # agent choose the planner seed of a capturing rollout would - # let a cherry-picked known-good seed bypass the gate. # In policy mode the deliverable is policy.py (via - # evaluate_policy); this tool remains a probe but can no + # submit_policy); this tool remains a probe but can no # longer capture the answer. capture_enabled=(ctx.capture_goal_reaching_plans - and not ctx.policy_capture_mode - and diagnostic_seed is None), - is_current_task=is_current, + and not ctx.policy_capture_mode), + is_current_task=True, have_plan=bool(grounded_plan), goal_achieved=goal_achieved, evaluator_rejected=evaluator_rejected, @@ -1023,8 +830,8 @@ def _probe_without_latent(atom: Any, "real environment samples the same variability - a plan " "that only sometimes succeeds in simulation will likely " "fail for real. To debug a failed rollout first, re-run " - "it exactly: call this tool with rollout_seed= for full per-step reporting at " + "it exactly in run_python: sim.run(plan, seed=) gives full per-step reporting at " "that seed. Then add margin (e.g. tighter spacing, aim " "impacts closer to the middle of the fall path) and " "resubmit. Because this task has now produced a flaky " @@ -1059,11 +866,6 @@ def _probe_without_latent(atom: Any, "same scoring, so executing this plan cannot count as a " "solve. Find a plan whose rollout the evaluator scores " "solved=True.") - elif decision is CaptureDecision.WRONG_TASK_NOTE: - lines.append( - f"NOTE: this ran on train task {task_label}, NOT the current " - "task, so it is NOT captured as your answer. To submit, " - "re-run the plan on the current task (omit task_idx).") if result.first_failure_idx is not None: fr = result.steps[result.first_failure_idx].failure_reason lines.append( @@ -1125,7 +927,7 @@ def _probe_without_latent(atom: Any, _budget_footer(ctx, rollouts_before)) @tool( - "evaluate_policy", + "submit_policy", "Validate ./policy.py - your closed-loop `get_option(state, memory)` " "program - on the CURRENT task and capture it as your answer. The " "policy source is SNAPSHOTTED at call time (later edits need a new " @@ -1140,14 +942,13 @@ def _probe_without_latent(atom: Any, "one identical line that keeps completing with no state change " "(its no-op livelock twin) DO end it. Capture " "is gated like " - "evaluate_option_plan: the goal-reaching rollout is repeated " + "submit_plan: the goal-reaching rollout is repeated " "several times (fresh simulator env + varied planner seed per " "repeat, fresh memory per episode) and a FLAKY policy is reported " "instead of captured; physics-margin perturbations apply too. " - "`validation_rollouts` requests a stricter gate; `rollout_seed` " - "runs one diagnostic rollout at that planner seed (never " - "captured). Test recovery behavior first with sim.run_policy() in " - "explore_python, which runs ./policy.py from the CURRENT probe " + "`validation_rollouts` requests a stricter gate. Test recovery " + "behavior first with sim.run_policy() in " + "run_python, which runs ./policy.py from the CURRENT probe " "state (including perturbed or mid-plan states).", { "type": "object", @@ -1160,15 +961,6 @@ def _probe_without_latent(atom: Any, "rollouts a goal-reaching policy must pass (effective " "count is max(configured, this); never fewer).", }, - "rollout_seed": { - "type": - "integer", - "description": - "Diagnostic: run ONE rollout at this exact motion-" - "planner seed with full per-step reporting, to " - "reproduce a reported failed validation rollout. Never " - "captured.", - }, "include_atoms": { "type": "boolean", "description": @@ -1178,7 +970,7 @@ def _probe_without_latent(atom: Any, }, }, ) - async def evaluate_policy(args: Dict[str, Any]) -> Dict[str, Any]: + async def submit_policy(args: Dict[str, Any]) -> Dict[str, Any]: # pylint: disable-next=import-outside-toplevel from predicators.agent_sdk.policy_execution import \ build_policy_option_fn, execute_policy_forward @@ -1187,26 +979,22 @@ async def evaluate_policy(args: Dict[str, Any]) -> Dict[str, Any]: rollouts_before = ctx.attempt_rollout_count if not ctx.policy_capture_mode: return _error_result( - "evaluate_policy is only available in policy mode " + "submit_policy is only available in policy mode " "(agent_solve_policy_mode); submit plans via " - "evaluate_option_plan instead.") + "submit_plan instead.") if ctx.option_model is None: return _error_result("No option model available in ToolContext.") - all_options = ctx.options | ctx.iteration_proposals.proposed_options + all_options = ctx.options model = ctx.option_model model._name_to_parameterized_option = ( # type: ignore[attr-defined] # pylint: disable=protected-access {o.name: o for o in all_options}) requested_rollouts = args.get("validation_rollouts") - diagnostic_seed = args.get("rollout_seed") include_atoms = args.get("include_atoms", True) if requested_rollouts is not None and (not isinstance( requested_rollouts, int) or requested_rollouts < 1): return _error_result( "validation_rollouts must be a positive integer.") - if diagnostic_seed is not None and not isinstance( - diagnostic_seed, int): - return _error_result("rollout_seed must be an integer.") resolved, task_err = _resolve_task(ctx, None) if task_err is not None: @@ -1224,8 +1012,7 @@ async def evaluate_policy(args: Dict[str, Any]) -> Dict[str, Any]: with open(policy_path, "r", encoding="utf-8") as f: policy_source = f.read() - all_predicates = (ctx.predicates - | ctx.iteration_proposals.proposed_predicates) + all_predicates = ctx.predicates types = set(ctx.types) for opt in all_options: types.update(opt.types) @@ -1269,30 +1056,17 @@ def _report_step(i: int, outcome: Any) -> None: step_line += (f"\n Added: {{{added_s}}}" f"\n Deleted: {{{del_s}}}") lines.append(step_line) - if diagnostic_seed is not None and diag_fresh_scope is not None: - return img_block = render_scene_image(ctx, f"policy_step_{i}_{opt.name}") if img_block and img_block.get("saved_path"): saved_image_paths.append(img_block["saved_path"]) ctx.attempt_rollout_count += 1 - diag_fresh_scope = (ctx.validation_env_scope - if diagnostic_seed is not None - and validation_cfg.fresh_env else None) - if diagnostic_seed is not None: - lines.append( - f"DIAGNOSTIC rollout at planner seed {diagnostic_seed} - " - "never captured. Call again without rollout_seed to " - "capture.") - with (diag_fresh_scope() if diag_fresh_scope is not None else - contextlib.nullcontext()), \ - absolute_rollout_seed(diagnostic_seed): - result = execute_policy_forward(task, - option_fn, - model, - predicates=all_predicates, - max_policy_options=max_opts, - on_step=_report_step) + result = execute_policy_forward(task, + option_fn, + model, + predicates=all_predicates, + max_policy_options=max_opts, + on_step=_report_step) goal_reached = result.goal_reached within_horizon = (result.actions_to_goal is not None @@ -1376,7 +1150,7 @@ def _policy_validation_rollout() -> Tuple[bool, str]: capture_task_key = _capture_task_key(ctx) if capture_task_key in ctx.flaky_capture_task_keys: n_rollouts = max(n_rollouts, validation_cfg.rollouts_after_flaky) - if requested_rollouts is not None and diagnostic_seed is None: + if requested_rollouts is not None: n_rollouts = max(n_rollouts, min(requested_rollouts, _MAX_REQUESTED_ROLLOUTS)) fresh_scope = (ctx.validation_env_scope @@ -1384,8 +1158,7 @@ def _policy_validation_rollout() -> Tuple[bool, str]: rollout_outcomes: List[str] = [] base_planner_seed = CFG.seed if (ctx.capture_goal_reaching_plans and goal_achieved - and not evaluator_rejected and diagnostic_seed is None - and n_rollouts > 1): + and not evaluator_rejected and n_rollouts > 1): def _policy_repeat_rollout(repeat_idx: int) -> Tuple[bool, str]: with (fresh_scope() if fresh_scope is not None else @@ -1423,13 +1196,13 @@ def _policy_repeat_rollout(repeat_idx: int) -> Tuple[bool, str]: f"{base_planner_seed + n_rollouts - 1}; fresh env and " "fresh policy memory per rollout).") - # Parameter-margin gates, mirroring evaluate_option_plan (one + # Parameter-margin gates, mirroring submit_plan (one # shared code path: see _parameter_margin_sweep). param_sensitive_detail: Optional[str] = None margin_outcomes: List[str] = [] if (fresh_scope is not None and ctx.capture_goal_reaching_plans and goal_achieved and not evaluator_rejected - and diagnostic_seed is None and flaky_detail is None): + and flaky_detail is None): margin_outcomes, param_sensitive_detail, margin_note = \ _parameter_margin_sweep(ctx, validation_cfg, fresh_scope, _policy_validation_rollout, "policy") @@ -1437,8 +1210,7 @@ def _policy_repeat_rollout(repeat_idx: int) -> Tuple[bool, str]: capture_outcome = _decide_capture( capture_enabled=(ctx.capture_goal_reaching_plans - and ctx.policy_capture_mode - and diagnostic_seed is None), + and ctx.policy_capture_mode), is_current_task=True, have_plan=True, goal_achieved=goal_achieved, @@ -1526,7 +1298,6 @@ def _policy_repeat_rollout(repeat_idx: int) -> Tuple[bool, str]: _budget_footer(ctx, rollouts_before)) return { - "evaluate_predicate_on_trajectory": evaluate_predicate_on_trajectory, - "evaluate_option_plan": evaluate_option_plan, - "evaluate_policy": evaluate_policy, + "submit_plan": submit_plan, + "submit_policy": submit_policy, } diff --git a/predicators/agent_sdk/tools/verdicts.py b/predicators/agent_sdk/tools/verdicts.py index 95170a1cd..babc63d04 100644 --- a/predicators/agent_sdk/tools/verdicts.py +++ b/predicators/agent_sdk/tools/verdicts.py @@ -16,7 +16,7 @@ class _EvalStateCollector: """Per-step states + option labels of one rollout, for evaluator verdicts. The single collector behind every surface that scores a belief-sim - rollout (``evaluate_option_plan``'s first and validation rollouts, + rollout (``submit_plan``'s first and validation rollouts, ``_belief_rollout_verdict``). The cascade certificate needs per-step states (topple-onset analysis); option-boundary states give garbage verdicts, so prefer the option model's ``last_trajectory`` and flag @@ -86,8 +86,8 @@ def make_solved_check( ) -> Callable[[List[State], List[Any], bool], Tuple[bool, str]]: """Build the evaluator gate used inside refinement searches. - One policy for every surface (the MCP ``refine_plan_sketch`` and - ``BeliefProbe.refine``), so identical parameters can never get + One policy for every surface (``BeliefProbe.refine`` and the + explorer's refinement), so identical parameters can never get contradictory verdicts across tools: - a coarse rollout (option-boundary states only) never blocks, the same rule the capture path applies (a coarse certificate can @@ -194,11 +194,9 @@ def load_ground_sampler_fns( return {}, None with open(path, "r", encoding="utf-8") as f: code = f.read() - exec_ctx = build_exec_context( - types=ctx.types, - predicates=ctx.predicates - | ctx.iteration_proposals.proposed_predicates, - options=ctx.options | ctx.iteration_proposals.proposed_options) + exec_ctx = build_exec_context(types=ctx.types, + predicates=ctx.predicates, + options=ctx.options) fns, warnings, err = load_ground_samplers(code, exec_ctx) if err is not None: return {}, f"Error loading {path}:\n{err}" @@ -214,7 +212,7 @@ def _belief_rollout_verdict( """Execute ``grounded_plan`` in the belief sim and score it with the task's evaluator, returning ``(verdict, coarse)`` or None. - Used by ``refine_plan_sketch``, whose internal refinement rollouts + Used by the refinement search, whose internal rollouts don't expose per-step states; costs one extra plan rollout. Fully failure-tolerant: any problem returns None. """ diff --git a/predicators/approaches/agent_model_based_approach.py b/predicators/approaches/agent_model_based_approach.py index dcf005484..015a222ac 100644 --- a/predicators/approaches/agent_model_based_approach.py +++ b/predicators/approaches/agent_model_based_approach.py @@ -2,10 +2,10 @@ The agent plans a sequence of parameterized skills with object bindings, subgoal atoms after each step, and continuous parameters, and must -DELIVER it as an ``evaluate_option_plan`` capture on the current task - +DELIVER it as an ``submit_plan`` capture on the current task - nothing it did not validate in the simulator (the model) is ever executed. A backtracking parameter search remains available to the agent -as a tool (``refine_plan_sketch`` / ``sim.refine``) and to mid-episode +as a probe method (``sim.refine``) and to mid-episode suffix replans, but there is no approach-side refinement of unvalidated sketches. @@ -33,8 +33,7 @@ from predicators.agent_sdk.session_base import AgentSessionFatalError, \ query_fatal_error from predicators.agent_sdk.sketch_types import SketchStep as _SketchStep -from predicators.agent_sdk.tools import BUILTIN_TOOLS, \ - explore_python_replaces_tools, load_ground_sampler_fns +from predicators.agent_sdk.tools import BUILTIN_TOOLS, load_ground_sampler_fns from predicators.approaches import ApproachFailure from predicators.approaches.agent_model_free_approach import \ AgentModelFreeApproach @@ -63,7 +62,7 @@ _FINAL_SUBMIT_NUDGE = ( "You are out of exploration budget for this attempt. Do NOT explore " "further. In as few tool calls as possible, submit your single best " - "plan NOW via evaluate_option_plan on the current task (omit " + "plan NOW via submit_plan on the current task (omit " "task_idx), using the best parameters you have already validated. " "It is captured as your answer even if it does not fully reach the " "goal or does not score as a solve; then finish.") @@ -74,7 +73,7 @@ _FINAL_SUBMIT_POLICY_NUDGE = ( "You are out of exploration budget for this attempt. Do NOT explore " "further. In as few tool calls as possible, submit your current best " - "./policy.py NOW via evaluate_policy on the current task. It is " + "./policy.py NOW via submit_policy on the current task. It is " "captured as your answer even if it does not fully reach the goal or " "does not score as a solve; then finish.") @@ -206,31 +205,13 @@ def _get_synthesis_tool_names(self) -> Optional[List[str]]: """No synthesis phase in this approach - declare an empty set.""" return [] - def _get_solve_tool_names(self) -> Optional[List[str]]: - # Bilevel solving hands continuous refinement to a search, so the - # agent also gets refine_plan_sketch (backtracking refinement + - # forward validation on a param-free sketch). Needs a simulator. - # explore_python's sim.refine subsumes it (same search core, from - # any state); when explore_python is on, the standalone tool is - # offered only if the keep-replaced-tools flag asks for both. - tools = list(super()._get_solve_tool_names() or []) - if CFG.agent_planner_use_simulator and \ - not explore_python_replaces_tools(): - tools.append("refine_plan_sketch") - return tools - # ------------------------------------------------------------------ # # System prompt (simplified - no parameter tuning workflow) # ------------------------------------------------------------------ # def _get_agent_system_prompt(self) -> str: propose = CFG.agent_bilevel_use_llm_initial_params - # When explore_python replaces the standalone refine tool, every - # guidance mention must point at the probe equivalent instead of - # a tool the session lacks. - probe_replaces = explore_python_replaces_tools() - refine_ref = ("sim.refine (in explore_python)" - if probe_replaces else "refine_plan_sketch") + refine_ref = "sim.refine (in run_python)" # What a sketch step consists of (shared between modes). if propose: sketch_desc = ( @@ -264,7 +245,7 @@ def _get_agent_system_prompt(self) -> str: "write ./policy.py with `get_option(state, memory)` " "returning ONE plan line (the same sketch grammar) from " "the actual current state, or None when finished, then " - "run evaluate_policy on the current task until it " + "run submit_policy on the current task until it " "reaches the goal - that validated policy.py snapshot is " "your ONLY accepted output. Option failures do NOT end " "an episode: they arrive in memory['last_failure'] and " @@ -281,18 +262,18 @@ def _get_agent_system_prompt(self) -> str: "(sim.run / sim.refine) but are not the deliverable.") else: contract = ( - "You DELIVER by running evaluate_option_plan with " + "You DELIVER by running submit_plan with " "per-step subgoals on the current task until it reaches " "the goal - that captured plan is your ONLY accepted " - "output, so do not finish until evaluate_option_plan " + "output, so do not finish until submit_plan " "reaches the goal.") job = ("Your job is to produce a plan - " + sketch_desc + " - that reaches the goal. " + contract + "\n" - "evaluate_option_plan runs your EXACT parameters with no " + "submit_plan runs your EXACT parameters with no " "sampling, so every parameter must be right. To find working " f"values you MAY use {refine_ref} while reasoning (it " "searches for parameters but is slower); read the parameters " - "it reports and submit them via evaluate_option_plan. Use " + "it reports and submit them via submit_plan. Use " "whatever tools help.") # Parameter-effort and ground-sampler guidance live in the query's # Instructions (sketch_prompts.build_solve_prompt) - single source. @@ -301,7 +282,7 @@ def _get_agent_system_prompt(self) -> str: # the output-token overflow, and testing is often faster than deriving. brevity = ( " Keep your reasoning concise: prefer making a concrete attempt " - f"and testing it with {refine_ref} / evaluate_option_plan to " + f"and testing it with {refine_ref} / submit_plan to " "let the simulator tell you what's wrong.") params_clause = job + brevity + "\n\n" # Keep the subgoal-annotation template's option format consistent with @@ -357,11 +338,15 @@ def _build_solve_prompt(self, task: Task) -> str: """Build prompt asking for a plan sketch without continuous params.""" journal_text = "" strategy_text = "" + attempts_text = "" if CFG.agent_solve_use_journal: # pylint: disable-next=import-outside-toplevel from predicators.agent_sdk import journal as journal_mod journal_text = journal_mod.read_journal( self._tool_context.sandbox_dir) + attempts_text = journal_mod.read_journal( + self._tool_context.sandbox_dir, + filename=journal_mod.ATTEMPTS_FILENAME) strategy_text = journal_mod.read_strategy( self._tool_context.sandbox_dir) return bilevel_sketch.build_solve_prompt( @@ -376,6 +361,7 @@ def _build_solve_prompt(self, task: Task) -> str: ground_samplers=CFG.agent_bilevel_ground_samplers, journal=journal_text, strategy=strategy_text, + attempts=attempts_text, physics_margin=CFG.agent_plan_validation_physics_margin, policy_mode=CFG.agent_solve_policy_mode, ) @@ -455,7 +441,7 @@ def _solve(self, task: Task, timeout: int) -> Callable[[State], Action]: # entries in later sessions sharing this ToolContext. ctx.attempt_deadline = None # Policy mode is scoped to solve attempts: left armed, it - # would silently disable evaluate_option_plan's capture + # would silently disable submit_plan's capture # gate for the EXPLORER's queries, which deliver sketches # even in policy-mode configs. ctx.policy_capture_mode = False @@ -558,7 +544,8 @@ def _append_journal_auto_entry(self, header: str, sandbox_dir, header, "\n".join(body_lines), - max_chars=journal_mod.MAX_AUTO_ENTRY_CHARS) + max_chars=journal_mod.MAX_AUTO_ENTRY_CHARS, + filename=journal_mod.ATTEMPTS_FILENAME) except OSError as e: logging.warning("Journal entry %r failed: %s", header, e) return False @@ -600,12 +587,12 @@ def _record_task_context_in_journal(self, task: Task) -> None: def _record_attempt_in_journal(self, attempt: int, max_attempts: int, policy: Optional[Any], info: Optional[_CaptureInfo]) -> None: - """Auto-append this attempt's factual record to the solve journal. + """Auto-append this attempt's factual record to the attempt log. The harness-written record (outcome, budget spent, captured or - best refused plan) guarantees the journal's essentials even when - the agent records nothing; agent-authored lessons arrive - separately via the record_journal tool. + best refused plan) guarantees the essentials of every attempt + are on record even when the agent writes nothing; the agent's + own lessons live in journal.md, which it edits directly. """ if not self._journal_active(): return @@ -656,7 +643,7 @@ def _solve_attempt(self, task: Task) -> Callable[[State], Action]: The attempt's budgets are the wall clock (``agent_solve_attempt_wall_clock``) and the query's turn cap; - the only deliverable is an ``evaluate_option_plan`` capture + the only deliverable is an ``submit_plan`` capture (consumed via :meth:`_consume_validated_plan`). However that query ends - a spent budget, an unparseable sketch, @@ -670,11 +657,11 @@ def _solve_attempt(self, task: Task) -> Callable[[State], Action]: """ self._sync_tool_context() self._tool_context.current_task = task - # Let evaluate_option_plan record a goal-reaching + # Let submit_plan record a goal-reaching # plan on this task into solved_plan/solved_sketch (consumed below). self._tool_context.capture_goal_reaching_plans = True - # Policy mode: the deliverable is policy.py via evaluate_policy; - # evaluate_option_plan stays available for probing but cannot + # Policy mode: the deliverable is policy.py via submit_policy; + # submit_plan stays available for probing but cannot # capture. self._tool_context.policy_capture_mode = CFG.agent_solve_policy_mode # LLM-free bypass: a prewritten policy.py as the captured @@ -704,21 +691,21 @@ def _solve_attempt(self, task: Task) -> Callable[[State], Action]: raise except Exception as e: # pylint: disable=broad-except # The agent may have validated a working plan via - # refine_plan_sketch even if its final text didn't parse. + # submit_plan even if its final text didn't parse. policy = self._consume_validated_plan() if policy is not None: return policy logging.warning("[%s] Solve query failed: %s", self._run_id, e) else: # Fast path: the agent already refined + forward-validated - # a plan on this task via refine_plan_sketch - return it + # a plan on this task via submit_plan - return it # directly instead of re-refining the (possibly different) # final-text sketch. policy = self._consume_validated_plan() if policy is not None: return policy # The agent must itself reach a confirmed - # evaluate_option_plan capture (consumed above) so we + # submit_plan capture (consumed above) so we # never execute a plan it didn't verify. logging.info("[%s] Query ended without a validated plan.", self._run_id) @@ -1001,15 +988,14 @@ def _nudge_final_submission(self) -> Optional[Callable[[State], Action]]: nudge = (_FINAL_SUBMIT_POLICY_NUDGE if CFG.agent_solve_policy_mode else _FINAL_SUBMIT_NUDGE) if CFG.agent_solve_use_journal: - nudge += ( - " If an earlier attempt's entry in the Solve Journal " - "records a better plan (captured or refused) than " - "anything from this attempt, resubmit that plan instead." - " After the submission, call record_journal ONCE with a " - "short factual entry for later fresh-context attempts and " - "tasks: what you tried (exact parameters), the key " - "measurements, and what to try differently - facts and " - "measurements only, no verdicts like 'impossible'.") + nudge += (" If an earlier attempt's entry in the Attempt Log " + "records a better plan (captured or refused) than " + "anything from this attempt, resubmit that plan instead." + " After the submission, append ONE short factual entry " + "to ./journal.md for later fresh-context attempts and " + "tasks: what you tried (exact parameters), the key " + "measurements, and what to try differently - facts and " + "measurements only, no verdicts like 'impossible'.") # SUSPEND (not clear) the attempt deadline for the nudge query: # its cooperative refusals and the sandbox interrupt backstop # must not block the submission (or the journal entry) itself. @@ -1045,12 +1031,11 @@ def _nudge_final_submission(self) -> Optional[Callable[[State], Action]]: def _consume_validated_plan(self) -> Optional[Callable[[State], Action]]: """Return a policy from an agent-validated plan, or None. - ``evaluate_option_plan`` records a captured (goal-reaching, - validated) plan on the current solve task into the tool context. - Returning that exact simulator-verified plan guarantees the - agent's tool-validated answer is what executes, and avoids a - fresh refinement that with a different seed might not reproduce - it. + ``submit_plan`` records a captured (goal-reaching, validated) + plan on the current solve task into the tool context. Returning + that exact simulator-verified plan guarantees the agent's tool- + validated answer is what executes, and avoids a fresh refinement + that with a different seed might not reproduce it. """ capture = self._tool_context.take_plan_capture() if capture.policy_source: diff --git a/predicators/approaches/agent_model_free_approach.py b/predicators/approaches/agent_model_free_approach.py index d7039071a..1ac4771e3 100644 --- a/predicators/approaches/agent_model_free_approach.py +++ b/predicators/approaches/agent_model_free_approach.py @@ -31,9 +31,8 @@ from predicators.agent_sdk.rendering import save_task_state_image from predicators.agent_sdk.session_base import AgentSessionFatalError, \ query_fatal_error -from predicators.agent_sdk.tools import agent_render_resolution, \ - explore_python_replaces_tools -from predicators.agent_sdk.tools.inspection import render_options_digest, \ +from predicators.agent_sdk.tools import agent_render_resolution +from predicators.agent_sdk.tools.digests import render_options_digest, \ render_types_digest from predicators.approaches import ApproachFailure from predicators.approaches.agent_session_mixin import AgentSessionMixin @@ -118,6 +117,7 @@ def __init__(self, # entries persist across cycles while one evaluation's test-task # entries never leak into the next evaluation. self._pre_test_journal: Optional[str] = None + self._pre_test_attempts: Optional[str] = None self._pre_test_journal_valid = False # Scene renders attempted this episode. The first is the true initial # state; later ones come from mid-episode replans and get distinct @@ -215,7 +215,7 @@ def _create_planner_option_model(self) -> Optional[_OptionModelBase]: Honors two CFG knobs: * ``agent_planner_use_simulator`` -- when False, returns ``None`` - so the agent gets no ``evaluate_option_plan`` rollouts and must + so the agent gets no ``submit_plan`` rollouts and must plan open-loop from data + LLM reasoning (the model-free baseline). * ``agent_planner_use_base_simulator`` -- when True (and a @@ -258,9 +258,9 @@ def _create_planner_option_model(self) -> Optional[_OptionModelBase]: ## Scratchpad - CRITICAL You MUST maintain `./notes.md` as your working memory. \ **Read it at the very start of the session** and **read it \ -again before every evaluate_option_plan call** to remind yourself \ +again before every submit_plan call** to remind yourself \ what you already tried. **Update it immediately after every \ -evaluate_option_plan call** - no exceptions. +submit_plan call** - no exceptions. Use this exact format for each option you are tuning: @@ -302,7 +302,7 @@ def _get_agent_system_prompt(self) -> str: if use_scratchpad: steps.append( "**Read `./notes.md` before every test**, then **update it " - "immediately after every evaluate_option_plan call**. Record " + "immediately after every submit_plan call**. Record " "what you tried, what happened, and what you learned. " "This is your memory - without it you will repeat failures.") steps += [ @@ -361,37 +361,23 @@ def _get_sandbox_reference_files(self) -> Dict[str, str]: return files def _get_solve_tool_names(self) -> Optional[List[str]]: - # inspect_types / inspect_options are never offered: their - # digests are static per session, so the solve prompt injects - # them directly (same renderers - see _build_solve_prompt); - # a zero-turn prompt section beats a one-turn tool call that - # every fresh-context attempt would re-pay. + # Type / option digests are static per session, so the solve + # prompt injects them directly (see _build_solve_prompt); the + # trajectory and task digests live in run_python's namespace + # (`trajectories` / `describe_trajectory` / `sim.task()`). + # Every remaining tool needs a simulator: submit_plan + # rolls fully-specified plans out through the option model and + # run_python probes it, so a planner without a simulator + # gets neither. tools = [] - # When the probe is present it subsumes the remaining inspect - # tools too: `trajectories` / `describe_trajectory` in - # explore_python's namespace and `sim.task()`. The extra - # use_simulator guard keeps them for (hypothetical) sim-free - # configs where explore_python itself is never offered below. - probe_subsumes = (CFG.agent_planner_use_simulator - and explore_python_replaces_tools()) - if not probe_subsumes: - tools += ["inspect_trajectories", "inspect_train_tasks"] - # The remaining tools require a simulator: evaluate_option_plan - # rolls fully-specified plans out through the option model. - # None are offered when the planner has no simulator. - # (refine_plan_sketch, which backtracking-refines a param-free sketch, - # is exposed only by AgentModelBasedApproach.) if CFG.agent_planner_use_simulator: - tools.append("evaluate_option_plan") + tools.append("submit_plan") # Closed-loop policy mode: the delivery gate for the - # agent-written policy.py (evaluate_option_plan stays as a + # agent-written policy.py (submit_plan stays as a # probe but no longer captures). if CFG.agent_solve_policy_mode: - tools.append("evaluate_policy") - if CFG.agent_planner_use_explore_python: - tools.append("explore_python") - if CFG.agent_solve_use_journal: - tools.append("record_journal") + tools.append("submit_policy") + tools.append("run_python") return tools # ------------------------------------------------------------------ # @@ -659,21 +645,24 @@ def _journal_active(self) -> bool: and self._tool_context.sandbox_dir) def _snapshot_journal_for_test_phase(self) -> None: - """Capture the learning-only journal content at test-phase entry. + """Capture the learning-only journal and attempt log at test start. - The snapshot is what ``end_test_phase`` rolls the journal back + The snapshots are what ``end_test_phase`` rolls both files back to. A failed capture leaves ``_pre_test_journal_valid`` False so the rollback is skipped rather than destroying learning entries. """ self._pre_test_journal = None + self._pre_test_attempts = None self._pre_test_journal_valid = False if not self._journal_active(): return # pylint: disable-next=import-outside-toplevel from predicators.agent_sdk import journal as journal_mod + sandbox_dir = self._tool_context.sandbox_dir try: - self._pre_test_journal = journal_mod.read_raw( - self._tool_context.sandbox_dir) + self._pre_test_journal = journal_mod.read_raw(sandbox_dir) + self._pre_test_attempts = journal_mod.read_raw( + sandbox_dir, filename=journal_mod.ATTEMPTS_FILENAME) self._pre_test_journal_valid = True except OSError as e: logging.warning( @@ -682,21 +671,25 @@ def _snapshot_journal_for_test_phase(self) -> None: self._run_id, e) def _archive_and_rollback_test_journal(self) -> None: - """Archive the test-phase journal, then roll it back. + """Archive the test-phase journal and attempt log, then roll back. Each evaluation must be independent of previous evaluations: - entries recorded while solving test tasks (harness auto-entries - and agent notes) would otherwise leak this evaluation's test - tasks into the next one. Learning entries - the pre-test - snapshot - persist across cycles. Before the rollback, the full - journal (learning + this evaluation's additions) is copied to - the run's log dir, which lives outside the sandbox so the agent - cannot read it, for later inspection. + content written while solving test tasks (harness attempt-log + entries and the agent's own journal notes) would otherwise leak + this evaluation's test tasks into the next one. Learning content + - the pre-test snapshots - persists across cycles. Before the + rollback, both files (learning + this evaluation's additions) + are copied to the run's log dir, which lives outside the sandbox + so the agent cannot read them, for later inspection. """ if not self._pre_test_journal_valid: return - snapshot = self._pre_test_journal + snapshots = { + "journal": self._pre_test_journal, + "attempts": self._pre_test_attempts, + } self._pre_test_journal = None + self._pre_test_attempts = None self._pre_test_journal_valid = False sandbox_dir = self._tool_context.sandbox_dir if not self._journal_active(): @@ -704,25 +697,31 @@ def _archive_and_rollback_test_journal(self) -> None: assert sandbox_dir is not None # pylint: disable-next=import-outside-toplevel from predicators.agent_sdk import journal as journal_mod + filenames = { + "journal": journal_mod.JOURNAL_FILENAME, + "attempts": journal_mod.ATTEMPTS_FILENAME, + } + # One archive per evaluation phase, named by the 0-based cycle + # whose learning it evaluates (matching main.py's "ONLINE + # LEARNING CYCLE i"). The counter has already advanced past that + # cycle's learn, so subtract 1; the pre-learning initial test + # archives as "initial". A same-cycle re-eval overwrites its own + # file. + eval_cycle = self._online_learning_cycle - 1 + label = "initial" if eval_cycle < 0 else f"cycle{eval_cycle}" try: - content = journal_mod.read_raw(sandbox_dir) - if content is not None: - # One archive per evaluation phase, named by the 0-based - # cycle whose learning it evaluates (matching main.py's - # "ONLINE LEARNING CYCLE i"). The counter has already - # advanced past that cycle's learn, so subtract 1; the - # pre-learning initial test archives as "initial". A - # same-cycle re-eval overwrites its own file. - eval_cycle = self._online_learning_cycle - 1 - label = "initial" if eval_cycle < 0 else f"cycle{eval_cycle}" - archive_path = os.path.join(self._get_log_dir(), - f"journal_eval_{label}.md") - with open(archive_path, "w", encoding="utf-8") as f: - f.write(content) - logging.info( - "[%s] Archived the test-phase solve journal to %s", - self._run_id, archive_path) - journal_mod.restore(sandbox_dir, snapshot) + for kind, filename in filenames.items(): + content = journal_mod.read_raw(sandbox_dir, filename=filename) + if content is not None: + archive_path = os.path.join(self._get_log_dir(), + f"{kind}_eval_{label}.md") + with open(archive_path, "w", encoding="utf-8") as f: + f.write(content) + logging.info("[%s] Archived the test-phase %s to %s", + self._run_id, filename, archive_path) + journal_mod.restore(sandbox_dir, + snapshots[kind], + filename=filename) except OSError as e: logging.warning( "[%s] Failed to archive/roll back the test-phase solve " @@ -772,12 +771,11 @@ def _query_agent_for_option_plan(self, task: Task) -> list: def _solve_prompt_visualize_line(self) -> str: """The stuck-step visualization bullet: the probe's staging + render is the only visualization surface, so the bullet appears only when - explore_python is offered.""" - if CFG.agent_planner_use_simulator and \ - CFG.agent_planner_use_explore_python: + run_python is offered.""" + if CFG.agent_planner_use_simulator: return ( - "- **Use explore_python when stuck** - after 3+ failures on " - "the same step, STOP testing and use explore_python " + "- **Use run_python when stuck** - after 3+ failures on " + "the same step, STOP testing and use run_python " "(`sim.reset(mods={...})`, then `sim.render(...)`) to move " "the object to several candidate positions and " "orientations. It's free (no physics). Find the right " @@ -789,7 +787,7 @@ def _solve_prompt_scratchpad_line(self) -> str: if CFG.agent_planner_use_scratchpad: return ( "- **Read `./notes.md` before every " - "evaluate_option_plan call** " + "submit_plan call** " "and **update it immediately after each call** - append a " "row to the parameter table and update the explored-ranges " "summary. If you realize you forgot to update, STOP and " @@ -816,9 +814,8 @@ def _build_solve_prompt(self, task: Task) -> str: if a.predicate in visible_preds ] - # Types and options: the same digests the inspect_types / - # inspect_options tools would serve, injected here so those - # tools need not be offered (see _get_solve_tool_names). + # Types and options: static per-session digests, injected here + # instead of costing a tool turn (see _get_solve_tool_names). types_digest = render_types_digest(self._tool_context.types) options_digest = render_options_digest( self._get_all_options(), @@ -1093,10 +1090,10 @@ def _create_explorer(self) -> BaseExplorer: def _sync_tool_context(self) -> None: """Push current approach state into the shared ToolContext. - The MCP tools (inspect_options, evaluate_option_plan, etc.) read - from the ToolContext dataclass, not the approach directly. This - keeps them in sync after mutations (e.g. new trajectories - collected, options added). Called before each solve and learning + The MCP tools (submit_plan, run_python, etc.) read from the + ToolContext dataclass, not the approach directly. This keeps + them in sync after mutations (e.g. new trajectories collected, + options added). Called before each solve and learning interaction. Subclasses should call super() and then set additional fields (e.g. skill_factory_context). """ diff --git a/predicators/approaches/agent_option_learning_approach.py b/predicators/approaches/agent_option_learning_approach.py deleted file mode 100644 index b2fabf3f4..000000000 --- a/predicators/approaches/agent_option_learning_approach.py +++ /dev/null @@ -1,361 +0,0 @@ -"""Agent option learning approach: skill invention + planning via Claude Agent -SDK. - -At solve time, the agent can invent new parameterized options (using skill -factory reference files) and then plan with them. Requires -``agent_sdk_use_docker_sandbox=True`` so the agent can read skill factory -source files in ``/sandbox/reference/``. - -Example command:: - - python predicators/main.py --env pybullet_domino \\ - --approach agent_option_learning --seed 0 \\ - --num_train_tasks 1 --num_test_tasks 1 \\ - --agent_sdk_use_docker_sandbox True -""" -import logging -from functools import lru_cache -from typing import Any, Callable, Dict, List, Optional, Set - -from gym.spaces import Box - -from predicators import utils -from predicators.agent_sdk.proposal_exec import ProposalBundle -from predicators.approaches.agent_model_free_approach import \ - AgentModelFreeApproach -from predicators.settings import CFG -from predicators.structs import Action, ParameterizedOption, Predicate, \ - State, Task, Type - - -class AgentOptionLearningApproach(AgentModelFreeApproach): - """Option-learning planning approach using Claude Agent SDK. - - Extends AgentModelFreeApproach with the ability to invent and - retract parameterized options at solve time. The agent reads skill - factory reference files and writes Python code using skill factory - functions (create_pick_skill, create_place_skill, etc.) to define - new options, then plans with them in the same query. - """ - - _save_suffix = "AgentOptionLearning" - - def __init__(self, initial_predicates: Set[Predicate], - initial_options: Set[ParameterizedOption], types: Set[Type], - action_space: Box, train_tasks: List[Task], *args: Any, - **kwargs: Any) -> None: - # Agent-specific state (before super().__init__). - # (_agent_session_id is initialized by the session mixin.) - self._agent_proposed_options: Set[ParameterizedOption] = set() - - super().__init__(initial_predicates, initial_options, types, - action_space, train_tasks, *args, **kwargs) - - @classmethod - def get_name(cls) -> str: - return "agent_option_learning" - - # ------------------------------------------------------------------ # - # AgentSessionMixin hooks - # ------------------------------------------------------------------ # - - def _get_agent_system_prompt(self) -> str: - ref_root = ("/sandbox" if CFG.agent_sdk_use_docker_sandbox else ".") - return f"""\ -You are a robot planning agent that can also invent new skills. Your -primary goal is to generate an option plan to achieve task goals. If the -existing options are insufficient, you can propose new parameterized -options before planning. - -## Workflow -1. **Inspect** the task, available options, and trajectory data -2. **Invent** new options if needed — either by writing and executing - Python code directly, or by using the `propose_options` tool -3. **Test** — either write and run Python experiments to verify your - options, or use `evaluate_option_plan` to check that a plan achieves - the goal. Use `retract_abstractions` to remove options that don't - work. -4. **Plan** — output the final option plan - -## Skill Factories -Read the reference files in {ref_root}/reference/skill_factories/ for the -full API. Key factory functions available in the exec context for -propose_options: -- `create_pick_skill(name, types, config, get_target_pose_fn)` — \ -pick up an object (move above, descend, grasp, lift). \ -Continuous params: `(grasp_z_offset,)`. -- `create_place_skill(name, types, config, use_move_above=False)` — \ -place a held object (move to release position, open gripper, retreat). \ -No get_target_pose_fn; target comes from continuous params: \ -`(target_x, target_y, release_z, target_yaw)`. Set \ -`use_move_above=True` to add a MoveAbove phase before descending. -- `create_push_skill(name, types, config, get_target_pose_fn)` — \ -push with standard 4-waypoint trajectory. Requires \ -`config.robot_home_pos` to be set. Facing direction is \ -`(sin(yaw), cos(yaw))` from `get_target_pose_fn`. \ -Continuous params: `(approach_distance, contact_z_offset)`. -- `create_pour_skill(name, types, config, get_target_pose_fn)` — pour \ -from a held container. `get_target_pose_fn` returns cup position. \ -The skill computes jug-to-robot displacement internally using fixed \ -constants (y_off=-0.135, pour_z=0.65625, handle_h=0.1). \ -No continuous params. -- `create_move_to_skill(name, types, params_space, config, \ -get_target_pose_fn)` — move end-effector to a target pose -- `create_wait_option(name, config, robot_type)` — hold current pose; \ -annotate with `-> {{atoms}}` in the plan to specify when it should \ -terminate (e.g. `Wait(robot:Robot) -> {{Boiled(water:water_type)}}`). \ -Use `NOT Pred(...)` for atoms that should become false - -All factories (except `create_place_skill` and `create_wait_option`) \ -take a `SkillConfig` (available as `skill_config` in the exec \ -context) and a `get_target_pose_fn` callback with signature \ -`(state, objects, params, config) -> (x, y, z, yaw)`. The callback \ -receives empty params; geometry params are continuous params of the \ -output ParameterizedOption (except pour, which has no continuous \ -params). `config.transport_z` controls the transport height. - -Also available: `Phase`, `PhaseSkill`, `PhaseAction`, -`make_move_to_phase` for building custom multi-phase skills, and -`chain_options(name, children)` for chaining options. - -## Important -- No need to import — all standard imports (np, Box, - ParameterizedOption, State, Type, etc.), current types (e.g. - `robot_type`, `domino_type`), predicates, and options are already - available in the exec context. -- Only propose new options if existing ones cannot achieve the goal -- You can invent and test options in two ways: (a) write and execute - Python code directly in the sandbox, or (b) use the `propose_options`, - `retract_abstractions`, and `evaluate_option_plan` tools -- Always test your plan before committing -- Output the final plan in the standard format at the end - -## Debugging Tips -- Use `inspect_options` with `option_name` to save an option's source - code to ./proposed_code/.py, then Read it to study the implementation -- `evaluate_option_plan` automatically saves scene images to ./test_images/ - after each step — check them to debug spatial issues -- Your session logs are in ./session_logs/ — Glob and Read them to review - past attempts when iterating -- All proposal and option source code is in ./proposed_code/ — Read - files there to understand how existing options work -- When `evaluate_option_plan` fails, check the "Object poses at failure" - and "Missing goal atoms" in the output""" - - def _get_solve_tool_names(self) -> Optional[List[str]]: - return [ - "inspect_types", - "inspect_options", - "inspect_trajectories", - "inspect_train_tasks", - "inspect_past_proposals", - "propose_options", - "retract_abstractions", - "evaluate_option_plan", - ] - - def _get_sandbox_reference_files( # pylint: disable=useless-super-delegation - self) -> Dict[str, str]: - # Inherit skill_factories + options.py from AgentModelFreeApproach - return super()._get_sandbox_reference_files() - - # ------------------------------------------------------------------ # - # Overridable helpers (from AgentModelFreeApproach) - # ------------------------------------------------------------------ # - - def _get_all_options(self) -> Set[ParameterizedOption]: - # Include tool_context.options so options proposed during the query - # (via propose_options) reach the parser before - # _agent_proposed_options is snapshotted. iteration_proposals is a - # fallback for when the Docker sync to tool_context.options was - # incomplete. - proposal_opts = self._tool_context.iteration_proposals.proposed_options - result = (self._initial_options | self._agent_proposed_options - | self._tool_context.options | proposal_opts) - if not result: - logging.warning( - "_get_all_options() returning empty set. " - "initial=%d, agent_proposed=%d, ctx.options=%d, " - "proposal_opts=%d", - len(self._initial_options), - len(self._agent_proposed_options), - len(self._tool_context.options), - len(proposal_opts), - ) - return result - - def _sync_tool_context(self) -> None: - """Synchronize ToolContext with current state.""" - super()._sync_tool_context() - - # Override options to include agent-proposed ones - self._tool_context.options = self._get_all_options() - - # Inject skill factory functions + config into exec context - self._tool_context.skill_factory_context = \ - self._build_skill_factory_context() - - # ------------------------------------------------------------------ # - # Skill factory context - # ------------------------------------------------------------------ # - - def _build_skill_factory_context(self) -> Dict[str, Any]: - """Build exec context with skill factory functions for - propose_options.""" - # pylint: disable=import-outside-toplevel - from predicators.ground_truth_models.skill_factories import Phase, \ - PhaseAction, PhaseSkill, SkillConfig, create_move_to_skill, \ - create_pick_skill, create_place_skill, create_pour_skill, \ - create_push_skill, create_wait_option, make_move_to_phase - - context: Dict[str, Any] = { - # Skill factory functions - "create_pick_skill": create_pick_skill, - "create_place_skill": create_place_skill, - "create_push_skill": create_push_skill, - "create_pour_skill": create_pour_skill, - "create_move_to_skill": create_move_to_skill, - "create_wait_option": create_wait_option, - "make_move_to_phase": make_move_to_phase, - # Building blocks - "Phase": Phase, - "PhaseAction": PhaseAction, - "PhaseSkill": PhaseSkill, - "SkillConfig": SkillConfig, - # Generic helpers - "chain_options": utils.LinearChainParameterizedOption, - } - - # For pybullet envs, provide a pre-built SkillConfig - if CFG.env.startswith("pybullet"): - try: - context["skill_config"] = self._get_skill_config() - except Exception as e: # pylint: disable=broad-except - logging.warning( - f"Failed to build SkillConfig for {CFG.env}: {e}") - - return context - - @staticmethod - @lru_cache(maxsize=1) - def _get_skill_config() -> Any: - """Lazily build a SkillConfig for the current pybullet env.""" - from predicators.ground_truth_models.skill_factories import \ - SkillConfig # pylint: disable=import-outside-toplevel - - env_cls = _get_pybullet_env_cls(CFG.env) - _, robot, _ = env_cls.initialize_pybullet(using_gui=False) - - simulator = env_cls(use_gui=False) \ - if CFG.skill_phase_use_motion_planning else None - - return SkillConfig( - robot=robot, - open_fingers_joint=robot.open_fingers, - closed_fingers_joint=robot.closed_fingers, - fingers_state_to_joint=( # pylint: disable=protected-access - env_cls._fingers_state_to_joint), - max_vel_norm=CFG.pybullet_max_vel_norm, - ik_validate=CFG.pybullet_ik_validate, - robot_init_tilt=getattr(env_cls, 'robot_init_tilt', 0.0), - robot_init_wrist=getattr(env_cls, 'robot_init_wrist', 0.0), - robot_home_pos=(env_cls.robot_init_x, env_cls.robot_init_y, - env_cls.robot_init_z), - simulator=simulator, - ) - - # ------------------------------------------------------------------ # - # Solving (with option invention) - # ------------------------------------------------------------------ # - - def _build_solve_prompt(self, task: Task) -> str: - """Build solve prompt that adds skill invention instructions.""" - base_prompt = super()._build_solve_prompt(task) - - ref_root = ("/sandbox" if CFG.agent_sdk_use_docker_sandbox else ".") - skill_instructions = f""" - -## Skill Invention -You can also invent new options before planning. Follow these steps: - -1. **Analyse** — Determine whether the existing options are sufficient \ -to achieve the goal. -2. **Invent** — If not, read the skill factory reference files in \ -{ref_root}/reference/skill_factories/ to understand how to build new \ -options. You can create options in two ways: - - **Python code**: Write and execute Python scripts that import the \ -skill factories and construct options directly. - - **MCP tools**: Use `propose_options` to create options via the \ -tool interface. Use `retract_abstractions` to remove options that \ -don't work. - A pre-built `skill_config` (SkillConfig) is available in the exec \ -context for pybullet environments. -3. **Test** — Verify your options and plan work correctly: - - **Python code**: Write and run Python experiments to unit-test \ -individual options or full plans. - - **MCP tools**: Use `evaluate_option_plan` to check that a plan \ -(including any new options) achieves the goal. - Iterate until the test passes. -4. **Commit** — Once the test passes, output the final plan. Your \ -proposed options will be added to the option library for future tasks.""" - - return base_prompt + skill_instructions - - def _solve(self, task: Task, timeout: int) -> Callable[[State], Action]: - """Solve with option invention enabled. - - The propose_options and retract_abstractions tools directly - update ctx.options during the agent query. After solving we - snapshot the agent-proposed options for persistence. - """ - self._tool_context.iteration_proposals = ProposalBundle() - - policy = super()._solve(task, timeout) - - # Snapshot agent-proposed options (everything beyond initial) - self._agent_proposed_options = (self._tool_context.options - - self._initial_options) - - # Record iteration summary (options only) - proposals = self._tool_context.iteration_proposals - summary = { - "cycle": self._online_learning_cycle, - "proposed_options": [o.name for o in proposals.proposed_options], - "retracted_options": sorted(proposals.retract_option_names), - } - self._tool_context.iteration_history.append(summary) - - return policy - - # ------------------------------------------------------------------ # - # Save / Load - # ------------------------------------------------------------------ # - - def _extra_save_state(self) -> Dict[str, Any]: - return {"agent_proposed_options": self._agent_proposed_options} - - def _load_extra_save_state(self, save_dict: Dict[str, Any]) -> None: - self._agent_proposed_options = save_dict.get("agent_proposed_options", - set()) - logging.info("[Run %s] Restored %d agent-proposed options.", - self._run_id, len(self._agent_proposed_options)) - - -# --------------------------------------------------------------------------- # -# Lazy pybullet env lookup (module-level, cached) -# --------------------------------------------------------------------------- # - - -@lru_cache(maxsize=1) -def _get_pybullet_env_cls(env_name: str) -> Any: - """Look up the concrete PyBulletEnv subclass by name.""" - # pylint: disable=import-outside-toplevel - import predicators.envs as _envs_pkg # noqa: F401 - from predicators.envs.base_env import BaseEnv - from predicators.envs.pybullet_env import PyBulletEnv - for cls in utils.get_all_subclasses(BaseEnv): - if not cls.__abstractmethods__ and cls.get_name() == env_name: - if issubclass(cls, PyBulletEnv): - return cls - break - raise RuntimeError(f"No PyBulletEnv subclass found for env '{env_name}'") diff --git a/predicators/approaches/agent_session_mixin.py b/predicators/approaches/agent_session_mixin.py index 8780c78b2..b008db164 100644 --- a/predicators/approaches/agent_session_mixin.py +++ b/predicators/approaches/agent_session_mixin.py @@ -199,9 +199,16 @@ def _ensure_agent_session(self) -> None: "[%s] %s session tool surface: ALL static MCP tools " "(no subset declared).", approach_name, phase) else: - static = sorted(n for n in tool_names if n in set(ALL_TOOL_NAMES)) - dynamic = sorted(n for n in tool_names - if n not in set(ALL_TOOL_NAMES)) + attached_names = { + getattr(t, "name", "") + for t in (self._tool_context.extra_mcp_tools or ()) + } + static = sorted( + n for n in tool_names + if n in set(ALL_TOOL_NAMES) and n not in attached_names) + dynamic = sorted( + n for n in tool_names + if n not in set(ALL_TOOL_NAMES) or n in attached_names) lines = [ f"[{approach_name}] {phase} session tool surface " f"({len(tool_names)} total):" diff --git a/predicators/approaches/agent_sim_learning_approach.py b/predicators/approaches/agent_sim_learning_approach.py index c3351b92e..a593dcf4a 100644 --- a/predicators/approaches/agent_sim_learning_approach.py +++ b/predicators/approaches/agent_sim_learning_approach.py @@ -36,11 +36,10 @@ from predicators import utils from predicators.agent_sdk.session_base import AgentSessionFatalError, \ max_session_log_number, query_fatal_error -from predicators.agent_sdk.tools import JOURNAL_TOOL_NAMES, \ - SAMPLER_SYNTHESIS_TOOL_NAMES, SYNTHESIS_TOOL_NAMES, _SnapshotTarget, \ - create_synthesis_tools, evaluate_states_with, \ +from predicators.agent_sdk.tools import SYNTHESIS_TOOL_NAMES, \ + _SnapshotTarget, create_synthesis_tools, evaluate_states_with, \ finalize_versioned_snapshot, make_write_snapshot_hook -from predicators.agent_sdk.tools.inspection import render_options_digest, \ +from predicators.agent_sdk.tools.digests import render_options_digest, \ render_trajectory_digest, render_types_digest from predicators.approaches.agent_model_based_approach import \ AgentModelBasedApproach @@ -975,10 +974,11 @@ def _get_synthesis_tool_names(self) -> Optional[List[str]]: No inspect tools: the type/option digests are injected into the learn message (see :meth:`_build_synthesis_learn_message`) and trajectory access lives in ``run_python`` (``trajectories`` + - ``describe_trajectory``). No ``explore_python`` either: in - synthesis sessions the probe rides inside ``run_python``'s - namespace as ``sim`` (one exec namespace per session - a helper - defined next to the data is visible to probe sweeps). In the + ``describe_trajectory``). The probe rides inside that same + ``run_python`` namespace as ``sim`` (one exec namespace per + session - a helper defined next to the data is visible to probe + sweeps; the solve-phase instance of the tool is not built when + this one is attached). In the agent-synthesis session the probe runs against the CANDIDATE simulator.py via ctx.probe_option_model_provider (installed in _synthesize_with_agent); in the oracle-sim-program sampler @@ -986,18 +986,6 @@ def _get_synthesis_tool_names(self) -> Optional[List[str]]: ctx.option_model, which there IS the deployed belief model. """ names: List[str] = list(SYNTHESIS_TOOL_NAMES) - # When the agent is learning samplers in this session (not using - # ground-truth ones), expose the evaluate_sampler tool. - if self._do_synthesize_samplers: - names += list(SAMPLER_SYNTHESIS_TOOL_NAMES) - # The run's solve journal is also writable from learn sessions: - # what the learn phase discovers about the domain is exactly what - # future fresh-context solve attempts need (agents were already - # appending to journal.md by hand, bypassing the size cap and the - # facts-only guidance). The flag name reads "solve" but gates the - # run's journal channel as a whole. - if CFG.agent_solve_use_journal: - names += list(JOURNAL_TOOL_NAMES) return names # ── Subclass hooks ────────────────────────────────────────── @@ -1022,16 +1010,21 @@ def _compute_extra_synthesis_paths(self, base: str) -> Dict[str, str]: del base return {} - def _extra_synthesis_tools( + def _install_extra_synthesis_surfaces( self, exec_ns: Dict[str, Any], base_pred_triples: List[Tuple[State, Action, State]], inferred_hint: Dict[str, List[str]], extra_paths: Dict[str, str], - ) -> List[Any]: - """Return additional MCP tools to append to the synthesis tool list.""" + ) -> None: + """Install per-arm probe surfaces for the synthesis session. + + Subclasses register loaders in + ``self._tool_context.probe_artifact_loaders`` (the backends of + ``sim.predicates()`` / ``sim.samplers()``); the base arm has + none. + """ del exec_ns, base_pred_triples, inferred_hint, extra_paths - return [] def _extra_synthesis_message(self, extra_paths: Dict[str, str]) -> str: """Return text to append to the agent's first synthesis message. @@ -1161,8 +1154,8 @@ def _checkpoint_after_interaction_results(self, cycle: int) -> None: _CHECKPOINT_SANDBOX_FILES = ("simulator.py", "predicates.py", "samplers.py", "ground_samplers.py", - "notes.md", "journal.md", "strategy.md", - "open_questions.md") + "notes.md", "journal.md", "attempts.md", + "strategy.md", "open_questions.md") _CHECKPOINT_SANDBOX_DIRS = ("simulator_versions", "predicates_versions", "samplers_versions") _CHECKPOINT_MAX_FILE_BYTES = 2 * 1024 * 1024 @@ -1524,14 +1517,26 @@ def _probe_fit_state(self) -> Dict[str, Any]: setattr(self, "_probe_fit_state_store", state) return state - def _publish_probe_fit(self, params: Dict[str, float], version_tag: str, - simulator_file: str) -> None: + def _publish_probe_fit( + self, + params: Dict[str, float], + version_tag: str, + simulator_file: str, + fit_result: Optional[FitResult] = None, + sse: float = float("nan"), + applied_physical: Optional[Dict[str, float]] = None, + ) -> None: """Deploy a canonical ``sim.fit`` result to the candidate probe. Publishes the fitted values in place (invented predicates hold a live view over ``_fitted_params``), records the fitted file content, and drops the cached probe model so the next probe - rebuilds at these values without fitting again. + rebuilds at these values without fitting again. The full + ``fit_result`` (point estimate plus the Laplace bundle the + exploration ensemble is calibrated from), its ``sse``, and the + physical values actually applied to the planning env are kept + so the cycle's deployed model can be exactly this fit (see + :meth:`_published_fit_for_file`). """ self._fitted_params.clear() self._fitted_params.update(params) @@ -1542,11 +1547,41 @@ def _publish_probe_fit(self, params: Dict[str, float], version_tag: str, state = self._probe_fit_state() state["digest"] = digest state["version"] = version_tag + state["fit_result"] = fit_result + state["sse"] = sse + state["applied_physical"] = dict(applied_physical or {}) self._probe_model_cache().clear() self._tool_context.probe_param_status = f"fitted ({version_tag})" logger.info("Synthesis probe: sim.fit deployed %d params (%s).", len(params), version_tag) + def _published_fit_for_file( + self, + simulator_file: str, + expected_names: Collection[str], + ) -> Optional[Tuple[FitResult, float, str]]: + """The agent's last canonical ``sim.fit`` of exactly this file. + + Returns ``(fit_result, sse, version_tag)`` when the last + published fit ran on the current content of ``simulator_file`` + and over exactly ``expected_names``; ``None`` when nothing was + published, the file changed after the fit (an UNFITTED edit), or + the parameter set differs (a spec added or dropped after the + fit). + """ + state = self._probe_fit_state() + fit = state.get("fit_result") + if fit is None or not os.path.isfile(simulator_file): + return None + with open(simulator_file, "rb") as f: + digest = hashlib.sha256(f.read()).hexdigest() + if state.get("digest") != digest: + return None + if set(fit.names) != set(expected_names): + return None + return fit, float(state.get("sse", + float("nan"))), str(state.get("version")) + def _make_candidate_probe_model_provider( self, simulator_file: str, @@ -1554,7 +1589,7 @@ def _make_candidate_probe_model_provider( base_pred_triples: List[Tuple[State, Action, State]], inferred_hint: Dict[str, List[str]], ) -> Callable[[], _OracleOptionModel]: - """Lazy option-model builder behind the synthesis explore_python. + """Lazy option-model builder behind the synthesis run_python. The returned callable is installed as ``ctx.probe_option_model_provider`` for the synthesis session: @@ -1578,7 +1613,7 @@ def _make_candidate_probe_model_provider( def _provider() -> _OracleOptionModel: if not os.path.isfile(simulator_file): raise RuntimeError( - "explore_python probe: no candidate simulator yet - " + "run_python probe: no candidate simulator yet - " "write ./simulator.py (RESIDUAL_RULES / PARAM_SPECS / " "RESIDUAL_FEATURES) first; the probe runs against it.") with open(simulator_file, "rb") as f: @@ -1591,7 +1626,7 @@ def _provider() -> _OracleOptionModel: simulator_file, trajectories) if rules is None or specs is None: raise RuntimeError( - "explore_python probe: ./simulator.py failed to load " + "run_python probe: ./simulator.py failed to load " "(exec error, or RESIDUAL_RULES / PARAM_SPECS missing) - " "fix the file and probe again.") residual_features = (features @@ -1932,6 +1967,7 @@ def _run_agent_synthesis_session( finally: self._tool_context.extra_session_hooks = {} self._tool_context.extra_mcp_tools = [] + self._tool_context.probe_artifact_loaders.clear() self._tool_context.probe_option_model_provider = None self._tool_context.probe_fit_provider = None self._tool_context.probe_param_status = None @@ -1995,8 +2031,7 @@ def _build_synthesis_exec_ns( "ParamSpec": ParamSpec, } - # Curated per-trajectory digest (same renderer the old - # inspect_trajectories tool used), for a first look before + # Curated per-trajectory digest, for a first look before # ad-hoc exploration over the raw ``trajectories`` objects. all_predicates = self._get_all_predicates() @@ -2038,7 +2073,7 @@ def _attach_synthesis_session_state( Everything installed here is cleared by the caller's ``finally`` once the session query returns. """ - # Label tool output (e.g. record_journal headers) with the + # Label tool output (e.g. attempt-log headers) with the # learning cycle for the duration of this session. self._tool_context.learn_cycle_index = self._learning_cycle_index() # Build dynamic synthesis tools and attach them to the tool @@ -2062,11 +2097,10 @@ def _attach_synthesis_session_state( budget_check=lambda: _check_time_budget(self._tool_context), ) tools = list(toolkit.tools) - tools.extend( - self._extra_synthesis_tools(exec_ns, base_pred_triples, - inferred_hint, extra_paths)) + self._install_extra_synthesis_surfaces(exec_ns, base_pred_triples, + inferred_hint, extra_paths) if self._do_synthesize_samplers: - tools.extend(self._make_sampler_tools(sampler_paths)) + self._install_sampler_surface(sampler_paths) declared = set(self._get_synthesis_tool_names() or ()) self._tool_context.extra_mcp_tools = [ t for t in tools if getattr(t, "name", "") in declared @@ -2076,8 +2110,8 @@ def _attach_synthesis_session_state( # wraps the real env), then merge the probe facade into # run_python's namespace: synthesis sessions offer ONE exec # namespace, so helpers defined next to the data are visible to - # probe sweeps (no explore_python tool here - the roster method - # documents the policy). Unconditional: with fit / refine / + # probe sweeps (create_mcp_tools skips the solve-phase instance + # when this one is attached). Unconditional: with fit / refine / # forward-validation all living on ``sim``, the probe IS the # validation surface, so a synthesis session without it would # have no way to test what it writes. Only ``sim``/``BeliefProbe`` @@ -2130,9 +2164,8 @@ def _build_synthesis_learn_message( n_interaction = n_trajs - n_demos predicate_listing = self._format_predicate_signatures( self._get_all_predicates()) - # Static per-session digests, injected instead of offering the - # inspect_types / inspect_options tools (same renderers, zero - # turns; see the roster note in _get_synthesis_tool_names). + # Static per-session digests, injected instead of costing a tool + # turn (see the roster note in _get_synthesis_tool_names). types_digest = render_types_digest(self._tool_context.types) options_digest = render_options_digest( self._tool_context.options, @@ -2186,19 +2219,17 @@ def _build_synthesis_learn_message( # `sim` forward-rolls the CANDIDATE simulator. One sentence so # the two are not conflated; details live in the tool # description. - probe_note = "" - if CFG.agent_planner_use_explore_python: - probe_note = ( - "\n\nThe `sim` probe inside `run_python` forward-rolls " - "the CANDIDATE simulator you are editing at the params " - "of your last `sim.fit()` - it never fits on its own, so " - "after a structural edit its results are marked UNFITTED " - "until you run `sim.fit()` on the current file; pass " - "task_idx explicitly to `sim.reset`, `sim.task(task_idx)` " - "for a task digest). Its rollouts " - "are candidate predictions - do not mix them up with the " - "recorded real `trajectories`. Usage and the validation " - "protocol are in the system prompt's Tools section.") + probe_note = ( + "\n\nThe `sim` probe inside `run_python` forward-rolls " + "the CANDIDATE simulator you are editing at the params " + "of your last `sim.fit()` - it never fits on its own, so " + "after a structural edit its results are marked UNFITTED " + "until you run `sim.fit()` on the current file; pass " + "task_idx explicitly to `sim.reset`, `sim.task(task_idx)` " + "for a task digest). Its rollouts " + "are candidate predictions - do not mix them up with the " + "recorded real `trajectories`. Usage and the validation " + "protocol are in the system prompt's Tools section.") # Tool surface of the (just-opened) synthesis session, rendered # the same way the solve/explore prompts list theirs. # ``tool_names`` already merges the sandbox built-ins with the @@ -2319,8 +2350,9 @@ def _build_synthesis_learn_message( real test episode. A literal is earned only once the data brackets \ the constant from both sides with margin to spare. -Before ending the session, run `sim.fit()` on the final file (the \ -deployed model is fit from exactly that file; a GO verdict on \ +Before ending the session, run `sim.fit()` on the final file (that \ +fit IS the model deployed for this cycle - end without one and the \ +harness fits on its own and logs the deviation; a GO verdict on \ UNFITTED values is worthless) and then a GO/NO-GO check: refine a \ full solve of the train task in your candidate simulator and validate \ it with several trials (`sim.refine` / `sim.run(plan, trials=5)`). \ @@ -2350,7 +2382,9 @@ def _build_synthesis_learn_message( (never hard-coded to one task's coordinates), and known pitfalls. \ Future solve sessions read it as advisory reference (clearly framed \ as possibly wrong), so state uncertainty honestly. Unlike the \ -append-only journal, strategy.md is a LIVING document: REWRITE it \ +journal (`./journal.md`, an append-only log of facts and \ +measurements that you may also add to), strategy.md is a LIVING \ +document: REWRITE it \ freely this cycle wherever new evidence corrects or supersedes \ earlier advice, rather than appending contradictions.""" @@ -2459,25 +2493,51 @@ def _fit_params_after_synthesis( self._last_fit_result = None self._fit_sse = float("inf") else: - # This is the solver/test-time fit. It deliberately follows - # CFG.code_sim_learning_num_mcmc_steps; any extra - # info-seeking MCMC is run below and is not published into - # _fitted_params. - if self._physical_param_specs or has_physics_rules(rules): - # System ID: physical + rule params fit jointly against - # free-running rollouts (teacher-forced triples cannot - # see physical params - no velocities in State - and - # cannot see physics-command rules either, whose effects - # only exist through engine stepping). - fit_result, self._fit_sse = ( - self._fit_parameters_joint_rollout(rules, specs, - residual_features)) - elif has_latent_rules(rules): - fit_result, self._fit_sse = self._fit_parameters_recurrent( - rules, specs, base_pred_triples, residual_features) + # The deployed model is the agent's own canonical sim.fit of + # the final simulator.py: the values its GO/NO-GO check + # validated, with the Laplace bundle the exploration ensemble + # is calibrated from. The harness fits only when no such fit + # exists (session ended UNFITTED, ran out of turns, or an + # oracle sim program with no session at all), and says so. + expected = [s.name for s in self._physical_param_specs + ] + [s.name for s in specs] + published = None + if self._probe_fit_state().get("fit_result") is not None: + published = self._published_fit_for_file( + self._resolve_synthesis_paths().simulator_file, expected) + if published is not None: + fit_result, self._fit_sse, version = published + logger.info( + "Deploying the agent's published sim.fit (%s) of the " + "final simulator.py: %d params, SSE %.6f.", version, + len(expected), self._fit_sse) + applied = self._probe_fit_state().get("applied_physical") + if self._physical_param_specs and applied: + self._apply_identified_physical_params(dict(applied)) else: - fit_result, self._fit_sse = fit_rule_parameters( - rules, specs, base_pred_triples, residual_features) + if CFG.agent_sim_learn_oracle_sim_program: + logger.info("Oracle sim program: fitting its " + "parameters on the harness side.") + else: + logger.warning( + "FIT FALLBACK: the learn session ended without a " + "canonical sim.fit() of the final simulator.py " + "(last published fit: %s). Fitting on the " + "harness side - the deployed parameters were " + "never validated by the agent's GO check.", + self._probe_fit_state().get("version") or "none") + if self._physical_param_specs or has_physics_rules(rules): + fit_result, self._fit_sse = ( + self._fit_parameters_joint_rollout( + rules, specs, residual_features)) + elif has_latent_rules(rules): + fit_result, self._fit_sse = \ + self._fit_parameters_recurrent( + rules, specs, base_pred_triples, + residual_features) + else: + fit_result, self._fit_sse = fit_rule_parameters( + rules, specs, base_pred_triples, residual_features) self._last_fit_result = fit_result self._fitted_params.clear() self._fitted_params.update(fit_result.point_estimate) @@ -3302,12 +3362,12 @@ def materialise_latent( ) -> List[Optional[Dict[str, Any]]]: """Roll a trajectory through the rules; return per-step latent. - Used by :func:`evaluate_predicate_quality` so latent-aware - predicates can be scored against meaningful latent values. - Returned list aligns with ``traj.states``; entry ``i`` is the - latent *before* predicates are evaluated at state ``i``. If no - rules are loaded, every entry is ``None`` so latent-aware - classifiers fall back to their default branch. + Used by ``sim.predicates()`` so latent-aware predicates can be + scored against meaningful latent values. Returned list aligns + with ``traj.states``; entry ``i`` is the latent *before* + predicates are evaluated at state ``i``. If no rules are loaded, + every entry is ``None`` so latent-aware classifiers fall back to + their default branch. """ if not self._residual_rules: return [None] * len(traj.states) @@ -3806,7 +3866,7 @@ def _fresh_validation_env_scope( session env is never touched. Installed as ``ToolContext.validation_env_scope`` so - ``evaluate_option_plan``'s capture-validation rollouts each sample + ``submit_plan``'s capture-validation rollouts each sample a fresh physics world. The shared ``_base_env``'s reset cannot reconstruct state exactly (solver warm-start state, velocity residuals, near-matching bodies skipped by the reconstruction diff diff --git a/predicators/approaches/agent_sim_predicate_invention_approach.py b/predicators/approaches/agent_sim_predicate_invention_approach.py index eb4360625..c3b2d647b 100644 --- a/predicators/approaches/agent_sim_predicate_invention_approach.py +++ b/predicators/approaches/agent_sim_predicate_invention_approach.py @@ -39,11 +39,10 @@ import logging import os -from typing import Any, Dict, FrozenSet, List, Optional, Set, Tuple +from typing import Any, Dict, FrozenSet, List, Set, Tuple -from predicators.agent_sdk.tools import PREDICATE_SYNTHESIS_TOOL_NAMES, \ - _SnapshotTarget, create_predicate_synthesis_tools, \ - finalize_versioned_snapshot +from predicators.agent_sdk.tools import _SnapshotTarget, \ + finalize_versioned_snapshot, make_predicate_quality_loader from predicators.approaches.agent_sim_learning_approach import \ AgentSimLearningApproach from predicators.settings import CFG @@ -108,22 +107,6 @@ def _get_all_predicates(self) -> Set[Predicate]: # ── Agent session hooks ───────────────────────────────────── - def _get_synthesis_tool_names(self) -> Optional[List[str]]: - """Add the predicate-synthesis callable to the synthesis surface. - - Adds ``evaluate_predicate_quality`` (built by - :meth:`_extra_synthesis_tools`). Scene work (staging states, - rendering with overlays to verify geometric thresholds) lives on - the ``sim`` probe inside ``run_python``. - """ - names = super()._get_synthesis_tool_names() - if names is None: - return None - for extra in PREDICATE_SYNTHESIS_TOOL_NAMES: - if extra not in names: - names.append(extra) - return names - # ── Synthesis hooks ────────────────────────────────────────── def _compute_extra_synthesis_paths(self, base: str) -> Dict[str, str]: @@ -143,22 +126,23 @@ def _compute_extra_synthesis_paths(self, base: str) -> Dict[str, str]: "predicates_file_for_agent": predicates_file_for_agent, } - def _extra_synthesis_tools( + def _install_extra_synthesis_surfaces( self, exec_ns: Dict[str, Any], base_pred_triples: List[Tuple[State, Action, State]], inferred_hint: Dict[str, List[str]], extra_paths: Dict[str, str], - ) -> List[Any]: + ) -> None: del exec_ns, base_pred_triples, inferred_hint - trajectories = self._get_all_trajectories() - return create_predicate_synthesis_tools( - predicates_file=extra_paths["predicates_file"], - predicates_versions_dir=extra_paths["predicates_versions_dir"], - approach=self, - trajectories=trajectories, - cycle_index_provider=self._learning_cycle_index, - ) + self._tool_context.probe_artifact_loaders["predicates"] = \ + make_predicate_quality_loader( + predicates_file=extra_paths["predicates_file"], + predicates_versions_dir=extra_paths[ + "predicates_versions_dir"], + approach=self, + trajectories=self._get_all_trajectories(), + cycle_index_provider=self._learning_cycle_index, + ) def _build_write_snapshot_targets( self, @@ -203,7 +187,7 @@ def _extra_synthesis_message(self, extra_paths: Dict[str, str]) -> str: is too loose; tighten it or share the gating parameter with the rule \ via `params[...]` so MCMC can fit them jointly. -Workflow: edit `predicates.py`, call `evaluate_predicate_quality` \ +Workflow: edit `predicates.py`, call `sim.predicates()` in `run_python` \ (fast, also reloads predicates into the live set), then run \ `sim.refine` / `sim.run` with sketches that reference your invented \ names. Any predicate you reference in a sketch must exist in \ @@ -237,11 +221,10 @@ def _format_goal_nl_block(self) -> str: def _synthesis_workflow_extra(self) -> str: # The base workflow's step 4 depends on invented predicates: # sketches can only reference predicates that already exist. - return ( - "\nStep 4's sketches need subgoal predicates that do not " - "exist until you invent them: before validating, write them " - "to `predicates.py` and load with `evaluate_predicate_quality` " - "(see \"Predicate Invention\").") + return ("\nStep 4's sketches need subgoal predicates that do not " + "exist until you invent them: before validating, write them " + "to `predicates.py` and load with `sim.predicates()` " + "(see \"Predicate Invention\").") def _extra_synthesis_system_prompt(self) -> str: # The scene workbench is the sim probe inside run_python (the @@ -487,7 +470,7 @@ def _widget_at_fixture(s, objs): __SCENE_RENDER_REF__ render to confirm what's actually where - and \ `run_python` for the numeric sweep over trajectory states. -Validate with `evaluate_predicate_quality` (cheap; reports first-flip step, \ +Validate with `sim.predicates()` (cheap; reports first-flip step, \ monotonicity, coverage across all available trajectories). On goal-reaching \ trajectories (`reached_goal=True` in `describe_trajectory`) a milestone \ predicate should flip False→True exactly once and stay true; on failed \ @@ -497,7 +480,7 @@ def _widget_at_fixture(s, objs): physics doesn't follow). A placement predicate should be true exactly \ when an object is at its intended location and false otherwise. -`evaluate_predicate_quality` is also the loader: it updates the predicate \ +`sim.predicates()` is also the loader: it updates the predicate \ set used by `sim.refine`. Call it after every edit to \ `predicates.py` before re-running plan refinement. @@ -513,7 +496,7 @@ def _widget_at_fixture(s, objs): # Predicate-side latent guidance appended (after the base class's # simulator-side recurrent tutorial) under ``CFG.partially_observable``. # Invention-only: it teaches the optional ``latent`` classifier kwarg -# and the latent materialisation in ``evaluate_predicate_quality``, +# and the latent materialisation in ``sim.predicates()``, # which non-invention arms have no use for. _RECURRENT_PREDICATE_SECTION = """\ ### Predicate signature @@ -549,7 +532,7 @@ def _widget_at_fixture(s, objs): ### Diagnostics -`evaluate_predicate_quality` rolls each trajectory through your +`sim.predicates()` rolls each trajectory through your simulator to materialise the latent before scoring classifiers, so latent-aware predicates get a real block there. Use the eval report to localise failures (bad rule chain vs. bad threshold). diff --git a/predicators/approaches/sampler_learning_mixin.py b/predicators/approaches/sampler_learning_mixin.py index f54b91cf8..d8ed78b71 100644 --- a/predicators/approaches/sampler_learning_mixin.py +++ b/predicators/approaches/sampler_learning_mixin.py @@ -28,9 +28,8 @@ from predicators.agent_sdk.session_base import AgentSessionFatalError, \ query_fatal_error from predicators.agent_sdk.tools import _SnapshotTarget, \ - create_sampler_synthesis_tools, create_synthesis_tools, \ - finalize_versioned_snapshot -from predicators.agent_sdk.tools.inspection import render_options_digest + create_synthesis_tools, finalize_versioned_snapshot, make_sampler_loader +from predicators.agent_sdk.tools.digests import render_options_digest from predicators.code_sim_learning.fit_space import ParamSpec from predicators.ground_truth_models import get_gt_samplers from predicators.settings import CFG @@ -164,14 +163,15 @@ def _sampler_paths(self, base: str) -> Dict[str, str]: "samplers_file_for_agent": samplers_file_for_agent, } - def _make_sampler_tools(self, paths: Dict[str, str]) -> List[Any]: - """Build the evaluate_sampler MCP tool for a synthesis session.""" - return create_sampler_synthesis_tools( - samplers_file=paths["samplers_file"], - samplers_versions_dir=paths["samplers_versions_dir"], - approach=self, - cycle_index_provider=self._learning_cycle_index, - ) + def _install_sampler_surface(self, paths: Dict[str, str]) -> None: + """Register the ``sim.samplers()`` loader for a synthesis session.""" + self._tool_context.probe_artifact_loaders["samplers"] = \ + make_sampler_loader( + samplers_file=paths["samplers_file"], + samplers_versions_dir=paths["samplers_versions_dir"], + approach=self, + cycle_index_provider=self._learning_cycle_index, + ) def _sampler_snapshot_target(self, paths: Dict[str, str]) -> _SnapshotTarget: @@ -231,7 +231,7 @@ def _sampler_synthesis_message(self, paths: Dict[str, str]) -> str: from the Options digest in your prompt and the predicate classifiers \ (for the subgoal geometry) with the predicate listing above. -Workflow: write `{path}`, call `evaluate_sampler` (snapshots + installs \ +Workflow: write `{path}`, call `sim.samplers()` (snapshots + installs \ them and sanity-checks shape/box), then call `sim.refine` \ with a sketch using those options — the samples-to-refine count should \ drop sharply versus uniform. Iterate with `Edit` and re-run. Every \ @@ -263,7 +263,7 @@ def _load_samplers_from_module_file( Mirrors ``_load_predicates_from_module_file``. Returns an empty dict on missing file or exec failure (samplers are optional). Validation (unknown option names, non-callables) is shared with - the ``evaluate_sampler`` tool via ``load_learned_samplers``. + ``sim.samplers()`` via ``load_learned_samplers``. """ # pylint: disable=import-outside-toplevel from predicators.agent_sdk.proposal_exec import build_exec_context, \ @@ -354,7 +354,7 @@ def _synthesize_samplers_standalone( budget_check=lambda: _check_time_budget(self._tool_context), ) tools = list(toolkit.tools) - tools.extend(self._make_sampler_tools(paths)) + self._install_sampler_surface(paths) # Use the same declared surface as the mixin will assert against # (_get_synthesis_tool_names already includes the sampler tool since # _do_synthesize_samplers is True here). The rule-fitting surface is @@ -420,6 +420,7 @@ def _synthesize_samplers_standalone( finally: self._tool_context.extra_session_hooks = {} self._tool_context.extra_mcp_tools = [] + self._tool_context.probe_artifact_loaders.clear() self._tool_context.probe_fit_provider = None self._tool_context.probe_residuals_provider = None self._learning_mode = False diff --git a/predicators/explorers/agent_bilevel_explorer.py b/predicators/explorers/agent_bilevel_explorer.py index a50e15706..f9352b2a6 100644 --- a/predicators/explorers/agent_bilevel_explorer.py +++ b/predicators/explorers/agent_bilevel_explorer.py @@ -1,14 +1,19 @@ -"""Agent bilevel explorer: sketch, refine against mental model, execute real. - -Produces a plan *sketch* via a Claude agent, runs backtracking refinement -against the approach's currently-learned option model (from -``tool_context.option_model``), then rolls the refined plan out for real. -When the mental model disagrees with reality (e.g. a subgoal atom it -expected after a Wait doesn't actually hold), the trajectory is a targeted -learning signal for online simulator synthesis. +"""Agent bilevel explorer: the agent sketches an experiment, it runs as +written. + +Queries a Claude agent for a fully parameterized plan sketch and rolls +it out for real exactly as written. The agent refines and validates +in-session against the currently-learned belief model (``sim.refine``, +``sim.run``, ``submit_plan``); a plan that passed the capture +gate executes as a belief-certified solve attempt, anything else +executes as an experiment with no harness-side parameter search or +substitution. When the belief model disagrees with reality (e.g. a +subgoal atom it expected after a Wait doesn't actually hold), the +trajectory is a targeted learning signal for online simulator +synthesis. Parallels ``AgentPlanExplorer`` for session plumbing and -``AgentModelBasedApproach`` for the sketch/refine workflow. +``AgentModelBasedApproach`` for the sketch workflow. """ import logging @@ -34,7 +39,7 @@ class AgentBilevelExplorer(BaseExplorer): - """Queries a Claude agent for a plan sketch, refines it, and executes.""" + """Queries a Claude agent for a plan sketch and executes it as written.""" def __init__(self, predicates: Set[Predicate], options: Set[ParameterizedOption], types: Set[Type], @@ -64,7 +69,7 @@ def _get_exploration_strategy(self, train_task_idx: int, "agent_bilevel explorer needs a synced option_model" # Reset the per-request mental-model verdict so a stale value can't - # leak if refinement below throws or falls back to random before + # leak if the query below throws or falls back to random before # producing one. self._tool_context.last_mental_model_solved = None @@ -84,8 +89,8 @@ def _get_exploration_strategy(self, train_task_idx: int, self._tool_context.last_mental_model_solved = True return self._certified_plan_strategy(certified) - # Point the agent's interactive tools (refine_plan_sketch, - # evaluate_option_plan, the sim probe) at the EXPLORE task. They + # Point the agent's interactive tools (submit_plan, the + # sim probe) at the EXPLORE task. They # default to ctx.current_task when the agent omits task_idx, and # test-time _solve leaves current_task on the last TEST task. # Without this the agent tunes/validates its exploration plan against @@ -95,11 +100,11 @@ def _get_exploration_strategy(self, train_task_idx: int, # # Enable the capture path too (keyed to current_task == this explore # task): the agent often submits + simulator-validates a goal-reaching - # plan via evaluate_option_plan / refine_plan_sketch but ends with a + # plan via submit_plan but ends with a # prose summary whose final text doesn't parse into a sketch. Without # capture that productive solve is lost to the random-options fallback; # with it we recover the captured plan below (see _sketch_from_capture) - # and feed its continuous params into the info-gain search. Clear any + # and execute it at its captured params. Clear any # stale capture first; the next test _solve re-points current_task and # clears capture again, so an exploration plan can't leak into a test # solve. @@ -125,8 +130,8 @@ def _get_exploration_strategy(self, train_task_idx: int, initial_image_section=self._initial_image_section( task, train_task_idx), propose_params=CFG.agent_bilevel_use_llm_initial_params, - # The explorer refines its own sketch for exploration; it does - # not use the approach's tool-validated capture path. + # The sketch is the experiment; a capture-gate-validated + # plan is welcome (it counts as a solve) but not required. require_tool_validation=False, # Explore contract: the sketch is a real-env experiment, and # the belief model may lack goal-critical dynamics, so a @@ -150,7 +155,7 @@ def _get_exploration_strategy(self, train_task_idx: int, plan_text = self._extract_option_plan_text(responses) # The session's tool capture: a goal-reaching plan the agent # validated in the belief through the capture gate - # (evaluate_option_plan / refine_plan_sketch, N fresh + # (submit_plan, N fresh # rollouts). ``reached_goal`` is the gate's verdict. capture = self._tool_context.take_plan_capture() if CFG.agent_explorer_replay_certified_plan and capture.plan \ @@ -201,166 +206,36 @@ def _get_exploration_strategy(self, train_task_idx: int, ground_sampler_fns=gs_fns or None, ) if plan_text else [] if not sketch: - # Final message didn't parse into a sketch, but the agent may - # have submitted + simulator-validated a goal-reaching plan via - # evaluate_option_plan / refine_plan_sketch (captured into - # solved_plan / solved_sketch). Recover it as the sketch, - # carrying its continuous params as initial_params so they seed - # the info-gain search below rather than replaying verbatim. - # Mirrors the test solver's preference for the tool-validated - # capture over the final text. sketch = self._sketch_from_capture(capture) or [] if not sketch: raise ValueError("parsed empty plan sketch") - self._tool_context.last_sketch_subgoals = [ (s.subgoal_atoms, s.subgoal_neg_atoms) for s in sketch ] self._tool_context.last_sketch_options = [ (s.option.name, [o.name for o in s.objects]) for s in sketch ] - - # Log the sketch + subgoal annotations the learner will refine - # (mirrors the solver's sketch log). Subgoal-annotated steps are - # the ones info-seeking can turn into boundary probes. - sketch_lines = [] - for i, s in enumerate(sketch): - objs = ", ".join(o.name for o in s.objects) - line = f" {i}: {s.option.name}({objs})" - if s.initial_params is not None and len(s.initial_params): - par = ", ".join(f"{p:.4f}" for p in s.initial_params) - line += f"[{par}]" - if s.subgoal_atoms: - atoms = ", ".join(str(a) for a in s.subgoal_atoms) - line += f" -> {{{atoms}}}" - sketch_lines.append(line) + # The agent's sketch IS the experiment: it runs in the real + # environment exactly as written. The harness does no belief + # refinement of it - the agent refines and validates + # in-session (sim.refine / sim.run / submit_plan), + # and only a capture-gate-certified plan (handled above) + # counts as a belief-validated solve for early stopping. + plan = self._ground_sketch_verbatim(sketch) + self._tool_context.last_mental_model_solved = False + record = self._format_sketch(sketch, plan) logging.info( - "agent_bilevel explorer: refining sketch for train task %d " - "(%d steps):\n%s", train_task_idx, len(sketch), - "\n".join(sketch_lines)) - # Record this request's plan so the cycle's NEXT explore query - # (generated before anything executes) can differ from it. + "agent_bilevel explorer: executing the agent's sketch " + "verbatim for train task %d (%d steps; not " + "belief-certified):\n%s", train_task_idx, len(plan), record) self._tool_context.cycle_scheduled_plans.append( - "\n".join(sketch_lines)) - - # Explorer mode: keep BOTH subgoal and final-goal validation ON so - # the mental model reports the deepest step it cannot predict - a - # per-step subgoal it can't establish, or (at the final step) the - # task goal it predicts won't hold. On failure the returned plan - # keeps the searched prefix (the failing step runs with the exact - # params the model rejected) and CONTINUES with the sketch's - # seeded params for the suffix: exploration exists because the - # belief model is known-wrong, so its inability to certify later - # steps is a reason to collect the data, not to drop the tail of - # the designed experiment (a truncation here once discarded the - # blocking bond test of a bridge episode and cost the next learn - # session hours of belief-sim guesswork). `success` honestly - # reflects whether the mental model could reach the goal, so a - # model that merely executes-but-mispredicts is distinguishable - # from one that truly solves the task. - # Active-experiment design: when info-seeking is on, hand - # refinement the ensemble-disagreement scorer so it picks the most - # *informative* feasible continuous parameters (those straddling - # the learned model's decision boundaries) rather than the first - # feasible sample. Sampling pools feasible candidates within the - # step's per-node rollout budget (max_samples_per_step) and - # proposes them best-first across backtracking retries (the ranked - # remainder is replayed with no new rollouts), so hard-to-satisfy - # subgoals yield a real argmax without multiplying the budget. Off - # -> info_scorer is None and refinement behaves as before. - info_scorer = None - info_n_feasible_target = 1 - if CFG.agent_explorer_info_seeking: - info_scorer = self._tool_context.atom_disagreement_fn - info_n_feasible_target = \ - CFG.agent_explorer_info_n_feasible_target - n_annotated = sum(1 for s in sketch - if s.subgoal_atoms is not None) - logging.info( - "agent_bilevel explorer: info-seeking ON " - "(pool %d feasible candidates/step within the " - "%d-rollout step budget, ensemble size %d) — %d/%d " - "steps are subgoal-annotated and eligible for boundary " - "probing.%s", info_n_feasible_target, - CFG.agent_bilevel_explorer_max_samples_per_step, - CFG.agent_explorer_info_ensemble_size, n_annotated, - len(sketch), "" if info_scorer is not None else - " WARNING: no ensemble scorer wired (atom_disagreement_fn " - "is None) — probing disabled.") - - outcome = bilevel_sketch.refine_sketch( - task, - sketch, - option_model, - predicates=self._predicates, - timeout=float(timeout), - rng=self._rng, - max_samples_per_step=CFG. - agent_bilevel_explorer_max_samples_per_step, - check_subgoals=True, - check_final_goal=True, - truncate_on_subgoal_fail=True, - strip_latent_wait_targets=( - not self._tool_context.latent_tracking_available), - log_state=CFG.agent_bilevel_log_state, - run_id="agent_bilevel_explorer", - info_scorer=info_scorer, - info_n_feasible_target=info_n_feasible_target, - parameterized_samplers=self._tool_context. - parameterized_samplers, - pin_proposed_params=CFG.agent_explorer_pin_proposed_params, - pinned_step_retries=CFG.agent_explorer_pinned_step_retries, - ) - plan, success = outcome.plan, outcome.success - # Record the honest verdict so get_interaction_requests can stamp - # it onto this request: early stopping must not treat a task as - # solved when the mental model couldn't reach its goal, even if - # real-env execution of the experiment happens to. - self._tool_context.last_mental_model_solved = success - mm_status = ("solved the goal" if success else - "did NOT reach the goal — running as experiment") - logging.info( - f"agent_bilevel explorer: sketch has {len(sketch)} steps, " - f"refined {len(plan)} (mental model {mm_status}).") - seeded_from = outcome.seeded_only_from - if plan: - plan_strs = [] - for i, opt in enumerate(plan): - obj_s = ", ".join(o.name for o in opt.objects) - par_s = ", ".join(f"{p:.4f}" for p in opt.params) - mark = (" [seeded-only]" if seeded_from is not None - and i >= seeded_from else "") - plan_strs.append( - f" {i}: {opt.name}({obj_s})[{par_s}]{mark}") - logging.info("agent_bilevel explorer: experiment plan:\n%s", - "\n".join(plan_strs)) - # Keep the scheduled-plan record honest for the cycle's next - # explore query: say which steps run without belief-model - # certification, and whether any sketch tail was dropped for - # lack of seeds. - record_notes = [] - if seeded_from is not None: - record_notes.append( - f"steps {seeded_from}..{len(plan) - 1} execute on the " - "sketch's seeded params without belief-model " - "certification") - if len(plan) < len(sketch): - record_notes.append( - f"only the first {len(plan)}/{len(sketch)} sketch steps " - "execute (later steps lacked seeded params)") - if record_notes: - self._tool_context.cycle_scheduled_plans[-1] += ( - "\n NOTE: " + "; ".join(record_notes) + ".") - - if plan: - policy = utils.option_plan_to_policy( - plan, - abstract_function=lambda s: utils.abstract( - s, self._predicates)) - return self._wrap_policy(policy), lambda _: False - - logging.info("agent_bilevel explorer: refinement produced zero " - "steps, falling back to random.") + record + "\n NOTE: executes as written, without " + "belief-model certification.") + policy = utils.option_plan_to_policy( + plan, + abstract_function=lambda s: utils.abstract( + s, self._predicates)) + return self._wrap_policy(policy), lambda _: False except AgentSessionFatalError: # A random fallback would hide the broken session backend; # re-raise so the run terminates. @@ -383,16 +258,14 @@ def _sketch_from_capture( capture: PlanCapture) -> Optional[List[bilevel_sketch.SketchStep]]: """Rebuild a sketch from a captured, tool-validated plan, or None. - ``evaluate_option_plan`` / ``refine_plan_sketch`` stash a - forward-validated, goal-reaching plan on the explore task into - ``solved_plan`` (grounded options with continuous params) and - ``solved_sketch`` (the option skeleton plus the subgoals that - actually held). We reconstruct a sketch from that skeleton and - graft each captured option's continuous params onto the step's - ``initial_params``, so the info-gain refinement below seeds them - as the first candidate in each step's pool (see - ``_sample_info_seeking``) rather than replaying them verbatim. - The capture was already taken (consumed) by the caller. + ``submit_plan`` stashes a forward-validated plan on the explore + task into ``solved_plan`` (grounded options with continuous + params) and ``solved_sketch`` (the option skeleton plus the + subgoals that actually held). We reconstruct a sketch from that + skeleton and graft each captured option's continuous params onto + the step's ``initial_params``, so the plan executes at exactly + the values the agent validated. The capture was already taken + (consumed) by the caller. """ plan = capture.plan captured_sketch = capture.sketch @@ -412,10 +285,57 @@ def _sketch_from_capture( initial_params=params)) logging.info( "agent_bilevel explorer: final text didn't parse, recovered the " - "agent's tool-validated plan from capture (%d steps); seeding its " - "continuous params into the info-gain search.", len(seeded)) + "agent's tool-validated plan from capture (%d steps); executing " + "it at the captured params.", len(seeded)) return seeded + def _ground_sketch_verbatim( + self, sketch: Sequence[bilevel_sketch.SketchStep]) -> List[Any]: + """Ground each sketch step at the agent's proposed parameters. + + Nothing is searched or substituted: the executed plan is the + agent's. A step left without parameters (or with the wrong + arity) gets ONE uniform draw from the option's box and a + warning. Wait steps carry their annotated subgoals as + ``wait_target_atoms`` so the option terminates on the intended + atom change. + """ + plan: List[Any] = [] + for i, step in enumerate(sketch): + dim = step.option.params_space.shape[0] + params = step.initial_params + if params is None or len(params) != dim: + if dim > 0: + logging.warning( + "agent_bilevel explorer: step %d (%s) has no " + "usable proposed params (%s); drawing one sample " + "from the option's box - propose every " + "parameter explicitly.", i, step.option.name, + None if params is None else list(params)) + params = bilevel_sketch.sample_params(step.option, self._rng) + plan.append( + bilevel_sketch.ground_step( + step, np.asarray(params, dtype=np.float32))) + return plan + + @staticmethod + def _format_sketch(sketch: Sequence[bilevel_sketch.SketchStep], + plan: Sequence[Any]) -> str: + """One indented ``i: Option(objs)[params] -> {atoms}`` line per + grounded step (params as executed, atoms as annotated).""" + lines = [] + for i, (step, opt) in enumerate(zip(sketch, plan)): + objs = ", ".join(o.name for o in opt.objects) + par = ", ".join(f"{p:.4f}" for p in opt.params) + line = f" {i}: {opt.name}({objs})[{par}]" + atoms = sorted(str(a) for a in (step.subgoal_atoms or set())) + atoms += sorted(f"NOT {a}" + for a in (step.subgoal_neg_atoms or set())) + if atoms: + line += f" -> {{{', '.join(atoms)}}}" + lines.append(line) + return "\n".join(lines) + @staticmethod def _format_plan(plan: Sequence[Any]) -> str: """One indented ``i: Option(objs)[params]`` line per grounded @@ -501,10 +421,9 @@ def _build_experiment_guidance(self) -> str: Always injects the learn phase's open-questions ledger (the ranked experiment specs it wrote for exploration to run) when one exists in the sandbox. When info-seeking is on, additionally - tell the agent that refinement will turn each annotated step - into a boundary-probing experiment, and - when an ensemble - scorer is wired - point it at the predicates the learned model - is currently most internally uncertain about. + point the agent at ``sim.suggest_probes`` and - when an ensemble + scorer is wired - at the predicates the learned model is + currently most internally uncertain about. """ parts = [] ledger = self._read_open_questions() diff --git a/predicators/explorers/agent_plan_explorer.py b/predicators/explorers/agent_plan_explorer.py index 62a01d805..fe49b9a5d 100644 --- a/predicators/explorers/agent_plan_explorer.py +++ b/predicators/explorers/agent_plan_explorer.py @@ -102,10 +102,8 @@ def _build_exploration_prompt(self, train_task_idx: int) -> str: # Goal atoms goal_strs = [str(a) for a in sorted(task.goal, key=str)] - # Available options with signatures, including just-proposed ones. - all_options = (self._options - | - self._tool_context.iteration_proposals.proposed_options) + # Available options with signatures. + all_options = self._options option_strs = [] for opt in sorted(all_options, key=lambda o: o.name): type_sig = ", ".join(t.name for t in opt.types) @@ -126,16 +124,6 @@ def _build_exploration_prompt(self, train_task_idx: int) -> str: # Trajectory summary traj_summary = self._build_trajectory_summary() - # Planning results - planning_info = "" - if self._tool_context.planning_results: - pr = self._tool_context.planning_results - planning_info = ( - f"\n## Recent Planning Results\n" - f"Success rate: {pr.get('success_str', 'N/A')}\n" - f"Avg nodes expanded: {pr.get('avg_nodes_expanded', 'N/A')}\n" - f"Failures: {pr.get('failure_summaries', 'None')}\n") - # Available tools tools_str = "" if self._agent_session.tool_names: @@ -159,7 +147,7 @@ def _build_exploration_prompt(self, train_task_idx: int) -> str: ## Available Options {chr(10).join(option_strs)} -{traj_summary}{planning_info}{tools_str} +{traj_summary}{tools_str} ## Instructions Use your available tools to inspect the environment and test your plan before committing to it. @@ -226,9 +214,7 @@ def _extract_option_plan_text(self, responses: List[Dict[str, def _parse_and_ground_plan(self, plan_text: str, task: Task) -> list: """Parse option plan text and ground into executable options.""" objects = list(task.init) - all_options = (self._options - | - self._tool_context.iteration_proposals.proposed_options) + all_options = self._options parsed = utils.parse_model_output_into_option_plan( plan_text, objects, diff --git a/predicators/settings.py b/predicators/settings.py index ddeada785..f94fbf981 100644 --- a/predicators/settings.py +++ b/predicators/settings.py @@ -1512,17 +1512,11 @@ class GlobalSettings: agent_sdk_image_max_px = 512 # Max size (bytes) of a single newline-delimited JSON message the agent SDK # subprocess transport will buffer. The SDK default is 1 MB, which a tool - # result embedding a base64 scene image (e.g. inspect_train_tasks with - # include_image=True at 900x900) can exceed -> "JSON message exceeded + # result embedding a base64 scene image (e.g. a sim.render() at + # 900x900) can exceed -> "JSON message exceeded # maximum buffer size". 20 MB comfortably fits full-res scene images. agent_sdk_max_buffer_size = 20 * 1024 * 1024 agent_sdk_resume_session = True # resume previous session if available - agent_sdk_propose_types = True - agent_sdk_propose_predicates = True - agent_sdk_propose_objects = True - agent_sdk_propose_processes = True - agent_sdk_propose_options = True - agent_sdk_auto_select_predicates = True # run hill-climbing after proposals agent_sdk_max_trajectories_in_context = 3 agent_sdk_log_agent_responses = True @@ -1538,23 +1532,8 @@ class GlobalSettings: # Agent planner approach settings agent_planner_use_scratchpad = False # include notes.md scratchpad - # Include the solve-phase explore_python tool: a persistent Python - # namespace over the BeliefProbe exploration facade (set the sim to any task - # state or a modified copy, run option plans from it, read full-precision - # features, render, snapshot/restore) so the agent writes sweep loops in - # one call instead of one evaluate_option_plan round-trip per probe. - # Exploratory only: nothing run through it can be captured as the answer. - agent_planner_use_explore_python = False - # When explore_python is on, whether the tools it subsumes - # (refine_plan_sketch -> sim.refine; inspect_trajectories / - # inspect_train_tasks -> trajectories / describe_trajectory / - # sim.task() in the probe namespace) are STILL offered alongside it. - # Default False: one surface per capability, so the agent's habits - # don't split across redundant tools. No effect when - # agent_planner_use_explore_python is False. - agent_planner_explore_python_keep_replaced_tools = False # Whether the planner is given a simulator to test candidate plans with - # (the evaluate_option_plan tool / option-model rollouts). When False, the + # (the submit_plan tool / option-model rollouts). When False, the # agent must plan open-loop from trajectory data and LLM reasoning alone # -- the genuinely model-free baseline. agent_planner_use_simulator = True @@ -1568,10 +1547,6 @@ class GlobalSettings: # Agent bilevel approach settings agent_bilevel_max_samples_per_step = 50 # param samples per step - # Total refine_plan_sketch attempts (fresh rng each) when a refined - # plan reaches the goal atoms but the task evaluator scores it as a - # non-solve; all attempts share the one tool-call timeout budget. - agent_bilevel_refine_evaluator_attempts = 3 agent_bilevel_check_subgoals = True # check subgoal atoms after each step # When True, the agent proposes per-step continuous parameters inside the # plan sketch (`Option(obj:type)[p1, p2] -> {subgoals}`). Refinement tries @@ -1614,7 +1589,7 @@ class GlobalSettings: # submission nudge. agent_solve_max_attempts = 1 # Wall-clock budget per solve attempt, in seconds (0 disables). The - # turn cap bounds turns, not compute - one explore_python sweep hid + # turn cap bounds turns, not compute - one run_python sweep hid # 47k rollouts (~7 h) inside a single turn. On expiry, exploration # tools refuse with a submit-now message and the approach runs the # same best-effort submission flow as turn-cap exhaustion. @@ -1625,10 +1600,10 @@ class GlobalSettings: # raw transcript history, which also carries the *wrong* conclusions # of failed attempts. agent_solve_fresh_context = False - # Persistent per-run solve journal (/journal.md): the harness - # auto-records each attempt's outcome + captured plan, the agent adds - # lessons via the record_journal tool, and the journal is injected - # into every solve prompt. Entries are capped and guided to record + # Persistent per-run solve journal: the harness logs each attempt's + # outcome + captured plan to /attempts.md, the agent keeps + # its own lessons in /journal.md with the file tools, and + # both are injected into every solve prompt. The prompts ask for # facts/measurements rather than verdicts, so failed attempts steer # later ones away from repeated sweeps without re-importing their # anchoring mistakes. @@ -1636,7 +1611,7 @@ class GlobalSettings: # Closed-loop policy mode: the solve agent's deliverable is a per-task # PROGRAM (/policy.py with get_option(state, memory) -> next # plan line or None) validated in the belief model via the - # evaluate_policy tool and executed at test time WITHOUT an LLM in + # submit_policy tool and executed at test time WITHOUT an LLM in # the loop. Option failures are surfaced to the policy (via # memory["last_failure"]) instead of ending the episode, so recovery # (re-place a drifted block, re-aim after a BiRRT refusal) is the @@ -1674,7 +1649,7 @@ class GlobalSettings: # would otherwise silently continue the old run; a Slurm requeue or # a prompt resubmission of a live run is always recent. auto_resume_max_age_hours = 36.0 - # Per-call wall-clock limit for explore_python code execution, in + # Per-call wall-clock limit for solve-session run_python calls, in # seconds (0 disables). Enforced cooperatively at every probe sim # call, plus a hard async-exception watchdog for sim-free code (a # pure-Python loop blocks the event loop, so nothing else can stop @@ -1683,7 +1658,7 @@ class GlobalSettings: # session for hours. Synthesis sessions (candidate-simulator probes, # whose rollouts are far slower and whose reset can trigger a # refit) are exempt from THIS cap and get the generous one below. - agent_sdk_explore_python_call_timeout = 600.0 + agent_sdk_python_call_timeout = 600.0 # Standalone hard cap on one synthesis-session run_python call. # Sized for legitimate slow work (candidate-sim rollouts, refits) # while still killing runaway in-call sweeps: run_20260826_151728's @@ -1726,14 +1701,14 @@ class GlobalSettings: # under scripts/; the file may be a bare name or an absolute path. agent_bilevel_plan_sketch_dir = "plan_sketches" agent_bilevel_plan_sketch_file = "" - # When refine_plan_sketch is called without an explicit timeout, - # the tool computes + # When a sketch refinement runs without an explicit timeout, the + # caller computes # max(_min, _per_step * len(sketch)) # so plans with more steps automatically get more wall-clock budget. agent_bilevel_refinement_timeout_per_step = 30.0 # seconds per step agent_bilevel_refinement_timeout_min = 30.0 # floor on auto-scaled timeout # Total number of belief-sim rollouts a goal-reaching plan must pass in - # evaluate_option_plan before it is captured as the agent's answer. The + # submit_plan before it is captured as the agent's answer. The # shared sim env is nondeterministic across repeats (motion-planner # sampling, physics-solver state), so repeats sample the same execution # variability the real rollout will - a flaky plan is reported to the @@ -1828,35 +1803,19 @@ class GlobalSettings: # option-model rollouts per search node: plain steps spend one per # backtracking attempt (classic semantics); info-seeking steps spend # the same budget pooling candidates (see refine_sketch). - agent_bilevel_explorer_max_samples_per_step = 50 - # Active-experiment-design exploration: refinement picks the feasible - # continuous parameters the learned model is most *uncertain* about - # (ensemble disagreement on the step's subgoal atoms) instead of the - # first feasible sample, pushing probes toward learned decision - # boundaries. Off ⇒ identical to plain feasibility search. + # Active-experiment-design exploration: build the learned model's + # parameter ensemble so the agent can rank candidate probes by the + # ensemble's disagreement on a step's subgoal atoms + # (sim.suggest_probes) and the capture gate can sweep the rule-param + # margin. The agent decides what to run; the harness never moves + # its parameters. Off => no ensemble is built. agent_explorer_info_seeking = False - # Feasible candidates pooled per step before proposing the most - # informative; the pool doubles as the node's ranked retry stock and - # attempt cap (see bilevel_sketch.refine_sketch). 1 disables. - agent_explorer_info_n_feasible_target = 8 # Ensemble size used to estimate disagreement. 1 disables scoring # (every candidate scores 0) and reduces to first-feasible. agent_explorer_info_ensemble_size = 6 - # Exploration keeps the agent's explicit continuous parameters: a - # proposed value is a decision, not a seed. Refinement re-proposes a - # pinned step's own params on each attempt (the belief's motion - # planning and physics vary per rollout) and never samples a - # replacement for it; info-seeking boundary probing is limited to - # steps the agent left unspecified. Off restores seed-then-search - # (run_20260828_173502 traj8: a proposed [0.827, 1.148, 0.44, 0] - # butt Place executed as a sampled [1.081, 1.218, 0.484, -1.59]; - # the agents then stripped their SeatedOn/LegAtSite annotations to - # keep refinement from wandering, blinding the divergence monitor). - agent_explorer_pin_proposed_params = True - agent_explorer_pinned_step_retries = 3 # A plan the explore session validated through the capture gate - # (evaluate_option_plan / refine_plan_sketch: goal reached in + # (submit_plan: goal reached in # agent_plan_validation_rollouts fresh belief rollouts) is executed # verbatim as the episode's solve attempt with mental_model_solved= # True, and the cycle's remaining requests on that task replay it @@ -2169,7 +2128,7 @@ class GlobalSettings: # sketch step's subgoal, instead of bilevel refinement drawing them # uniformly from the option's box. The agent authors a versioned # ``samplers.py`` (LEARNED_SAMPLERS keyed by option name) and tunes it - # with the ``evaluate_sampler`` tool. Sampler learning rides along in + # with ``sim.samplers()``. Sampler learning rides along in # the sim/predicate synthesis session when one runs # (oracle_sim_program=False); when no synthesis session runs # (oracle_sim_program=True) it gets a dedicated session of its own. diff --git a/scripts/configs/predicatorv3/approaches/all.yaml b/scripts/configs/predicatorv3/approaches/all.yaml index 9f811b485..16896f796 100644 --- a/scripts/configs/predicatorv3/approaches/all.yaml +++ b/scripts/configs/predicatorv3/approaches/all.yaml @@ -52,7 +52,6 @@ APPROACHES: agent_bilevel_use_llm_initial_params: True # LLM proposes params agent_sdk_max_agent_turns_per_iteration: 200 agent_sdk_image_max_px: 900 - agent_planner_use_explore_python: True agent_solve_max_attempts: 5 agent_solve_attempt_wall_clock: 2700 agent_solve_fresh_context: True @@ -83,7 +82,6 @@ APPROACHES: agent_bilevel_use_llm_initial_params: True # LLM proposes params agent_sdk_max_agent_turns_per_iteration: 200 agent_sdk_image_max_px: 900 - agent_planner_use_explore_python: True agent_solve_max_attempts: 5 agent_solve_attempt_wall_clock: 2700 agent_solve_fresh_context: True @@ -99,9 +97,9 @@ APPROACHES: # Oracle: GT monolithic sim as world model + GT predicates. Even with # use_llm_initial_params False the LLM delivers a full, concrete, # exact-parameter plan and validates it in the sim via - # evaluate_option_plan -- there is no approach-side backtracking - # fallback. On top of plain agent_planner it has the refine_plan_sketch - # tool (an agent-invokable per-step backtracking parameter search) plus + # submit_plan -- there is no approach-side backtracking + # fallback. On top of plain agent_planner it has sim.refine (an + # agent-invokable per-step backtracking parameter search) plus # the sketch / per-step-subgoal scaffolding. agent_oracle_mono_sim: NAME: "agent_bilevel" @@ -118,7 +116,7 @@ APPROACHES: # Oracle (upper bound): GT hybrid sim + GT base-sim physical params # (true friction), as if all learning had already succeeded. # Includes the 200-turn sandbox budget (2026-07-15 sweeps: every failed - # test episode exhausted the default 50-turn budget), explore_python, + # test episode exhausted the default 50-turn budget), run_python, # and the solve restart loop: up to 5 time-boxed (45 min) attempts, # each on a fresh conversation, with cross-attempt knowledge travelling # through the solve journal (run_20260717_23xx family split: the same @@ -144,7 +142,6 @@ APPROACHES: agent_bilevel_max_execution_replans: 2 agent_sdk_max_agent_turns_per_iteration: 200 agent_sdk_image_max_px: 900 - agent_planner_use_explore_python: True agent_solve_max_attempts: 5 agent_solve_attempt_wall_clock: 2700 agent_solve_fresh_context: True @@ -195,7 +192,7 @@ APPROACHES: # # per-step sampling prior to any sketch step - a uniform window # # `~ [w1, w2]` around its proposed params, or `~ name` referencing a # # function it wrote in ground_samplers.py (reloaded on every - # # refine_plan_sketch call). No learning session; the channel is pure + # # sim.refine call). No learning session; the channel is pure # # sketch grammar, enabled by agent_bilevel_ground_samplers. Compare # # against agent_oracle_hybrid_sim (which never sees the grammar) to # # measure what per-call ground sampling adds. @@ -288,16 +285,6 @@ APPROACHES: num_online_learning_cycles: 0 execution_monitor: "subgoal_annotations" agent_bilevel_max_execution_replans: 2 - # # Baseline: learn options from demonstrations. - # agent_option_learning: - # NAME: "agent_option_learning" - # SKIP: True - # FLAGS: - # explorer: "agent_plan" - # option_learner: "agent" - # demonstrator: "oracle_process_planning" - # terminate_on_goal_reached_and_option_terminated: True - # agent_sdk_use_local_sandbox: True # ====================================================================== # DEMONSTRATOR / SCRIPTED (demo source used by every agent arm above, diff --git a/scripts/log_viewer.py b/scripts/log_viewer.py index 17ffe8307..692cb544b 100644 --- a/scripts/log_viewer.py +++ b/scripts/log_viewer.py @@ -26,7 +26,8 @@ from format_conversation_markdown() in agent_sdk/log_formatter.py * "Captured as the current answer"/"NOT CAPTURED" capture verdicts and "Goal achieved: True|False" rollout lines inside tool-result blocks, - from evaluate_option_plan in agent_sdk/tools/testing.py + from submit_plan (formerly evaluate_option_plan) in + agent_sdk/tools/testing.py * "Test results: defaultdict(..., {...})" lines in info.log Format contracts, continued: @@ -81,7 +82,8 @@ RESULT_RE = re.compile( r"\*\*Result:\*\* (\d+) turns, \$([\d.]+) this solve, \$([\d.]+) total") GOAL_RE = re.compile(r"Goal achieved: (True|False)") -# Session-level capture verdicts from evaluate_option_plan (agent_sdk/ +# Session-level capture verdicts from submit_plan (formerly +# evaluate_option_plan; agent_sdk/ # tools/testing.py). A "Goal achieved" line only says one sim rollout # reached the goal atoms; the capture verdict is what decides whether the # session actually produced an answer. A best-effort capture (budget @@ -809,7 +811,8 @@ def _parse_episode(path: str) -> Dict[str, Any]: info: Dict[str, Any] = {} captures = CAPTURED_RE.findall(text) if captures or NOT_CAPTURED_RE.search(text): - # The session used evaluate_option_plan: its capture verdicts are + # The session used submit_plan (formerly evaluate_option_plan): + # its capture verdicts are # the agent-side outcome. "Goal achieved" lines are per-rollout # goal-atom checks that stay True even when the evaluator rejects # the plan (solved=False), so they must not decide the session. diff --git a/tests/agent_sdk/test_bilevel_sketch_near_miss.py b/tests/agent_sdk/test_bilevel_sketch_near_miss.py index e8c8b4c16..eeca4148b 100644 --- a/tests/agent_sdk/test_bilevel_sketch_near_miss.py +++ b/tests/agent_sdk/test_bilevel_sketch_near_miss.py @@ -4,7 +4,7 @@ executed but failed validation - the failing step's exact params, the missing atoms, and that rollout's post-state - and ``refine_and_validate_report`` surfaces it, so a failed -refine_plan_sketch call returns a gradient instead of only the stuck +sim.refine call returns a gradient instead of only the stuck step's name. """ diff --git a/tests/agent_sdk/test_bilevel_sketch_regions.py b/tests/agent_sdk/test_bilevel_sketch_regions.py index 8d8938312..8322ef920 100644 --- a/tests/agent_sdk/test_bilevel_sketch_regions.py +++ b/tests/agent_sdk/test_bilevel_sketch_regions.py @@ -16,6 +16,7 @@ from gym.spaces import Box from predicators import utils +from predicators.agent_sdk.belief_probe import BeliefProbe from predicators.agent_sdk.sketch_parsing import format_step_line, \ parse_sketch_from_text, strip_region_annotations from predicators.agent_sdk.sketch_refinement import refine_sketch @@ -452,14 +453,14 @@ def fn(*_args): # --------------------------------------------------------------------------- # -def _run_tool(tool_name, args, ground_samplers=True, sandbox_dir=None): +def _tool_ctx(ground_samplers=True, sandbox_dir=None): utils.reset_config({ "agent_bilevel_use_llm_initial_params": True, "agent_bilevel_max_samples_per_step": 200, "agent_bilevel_ground_samplers": ground_samplers, }) task = _task_hi() - ctx = ToolContext( + return ToolContext( types={_block_type}, predicates={_ReachedHi}, processes=set(), @@ -470,6 +471,10 @@ def _run_tool(tool_name, args, ground_samplers=True, sandbox_dir=None): current_task=task, sandbox_dir=sandbox_dir, ) + + +def _run_tool(tool_name, args, ground_samplers=True, sandbox_dir=None): + ctx = _tool_ctx(ground_samplers=ground_samplers, sandbox_dir=sandbox_dir) tools = { t.name: t.handler for t in create_mcp_tools(ctx, tool_names=[tool_name]) @@ -483,36 +488,37 @@ def _run_tool(tool_name, args, ground_samplers=True, sandbox_dir=None): return result["content"][0]["text"] -def test_refine_plan_sketch_tool_accepts_region_grammar(): - """The MCP handler parses the region and searches inside its window.""" - text = _run_tool( - "refine_plan_sketch", { - "plan": ("Move(block0:block)[0.85] ~ [0.1] -> " - "{ReachedHi(block0:block)}"), - "timeout": - 10, - }) - assert "SUCCESS" in text - assert "Parameters found" in text +def _probe_refine(plan, ground_samplers=True, sandbox_dir=None): + """``sim.refine`` on the fake model - the agent-facing refinement + surface (same parser and search core as the explorer's refinement).""" + ctx = _tool_ctx(ground_samplers=ground_samplers, sandbox_dir=sandbox_dir) + return BeliefProbe(ctx).reset(task_idx=0).refine(plan, timeout=10) + + +def _first_param(result): + return float(result.plan_lines[0].split("[")[1].split("]")[0]) + + +def test_probe_refine_accepts_region_grammar(): + """The probe parses the region and searches inside its window.""" + res = _probe_refine("Move(block0:block)[0.85] ~ [0.1] -> " + "{ReachedHi(block0:block)}") + assert res.success + assert "SUCCESS" in str(res) # The reported parameter came from the window's passing band. - param = float(text.split("Move(block0)[")[1].split("]")[0]) - assert 0.9 <= param <= 0.95 + assert 0.9 <= _first_param(res) <= 0.95 -def test_refine_plan_sketch_tool_rejects_bad_region(): - """Strict tool parsing surfaces a malformed region as a clear error.""" - text = _run_tool("refine_plan_sketch", { - "plan": "Move(block0:block)[0.85] ~ [0.1, 0.2]", - "timeout": 10, - }) - assert "Could not parse plan sketch" in text - assert "expects 1" in text +def test_probe_refine_rejects_bad_region(): + """Strict parsing surfaces a malformed region as a clear error.""" + with pytest.raises(ValueError, match="expects 1"): + _probe_refine("Move(block0:block)[0.85] ~ [0.1, 0.2]") -def test_evaluate_option_plan_ignores_region(): - """evaluate_option_plan runs the exact center; the region is inert.""" +def test_submit_plan_ignores_region(): + """submit_plan runs the exact center; the region is inert.""" text = _run_tool( - "evaluate_option_plan", { + "submit_plan", { "plan": ("Move(block0:block)[0.95] ~ [0.05] -> " "{ReachedHi(block0:block)}"), "include_states": @@ -526,26 +532,22 @@ def test_evaluate_option_plan_ignores_region(): assert "Goal achieved: True" in text -def test_refine_plan_sketch_tool_ignores_region_when_disabled(): +def test_probe_refine_ignores_region_when_disabled(): """With agent_bilevel_ground_samplers off, the annotation is a no-op. The params still seed the search but sampling stays uniform, and the report says so - baseline arms still cannot use the channel, but agents no longer burn turns on an error. """ - text = _run_tool("refine_plan_sketch", { - "plan": ("Move(block0:block)[0.85] ~ [0.1] -> " - "{ReachedHi(block0:block)}"), - "timeout": - 10, - }, - ground_samplers=False) - assert "Could not parse plan sketch" not in text - assert "IGNORED" in text - assert "uniform" in text - - -def test_refine_plan_sketch_tool_named_ground_sampler(tmp_path): + res = _probe_refine( + "Move(block0:block)[0.85] ~ [0.1] -> " + "{ReachedHi(block0:block)}", + ground_samplers=False) + assert "IGNORED" in res.note + assert "uniform" in res.note + + +def test_probe_refine_named_ground_sampler(tmp_path): """A `~ name` reference loads GROUND_SAMPLERS from the sandbox and confines the step's draws to the function's distribution.""" (tmp_path / "ground_samplers.py").write_text("""\ @@ -556,42 +558,28 @@ def _hi_band(state, subgoal_atoms, rng, objects): GROUND_SAMPLERS = {"hi_band": _hi_band} """, encoding="utf-8") - text = _run_tool("refine_plan_sketch", { - "plan": ("Move(block0:block)[0.1] ~ hi_band -> " - "{ReachedHi(block0:block)}"), - "timeout": - 10, - }, - sandbox_dir=str(tmp_path)) - assert "SUCCESS" in text - param = float(text.split("Move(block0)[")[1].split("]")[0]) + res = _probe_refine( + "Move(block0:block)[0.1] ~ hi_band -> " + "{ReachedHi(block0:block)}", + sandbox_dir=str(tmp_path)) + assert res.success # The failing center 0.1 was tried once; the named sampler landed a # value inside its own band. - assert 0.9 <= param <= 0.95 + assert 0.9 <= _first_param(res) <= 0.95 -def test_refine_plan_sketch_tool_unknown_named_sampler(tmp_path): - """An unresolvable `~ name` is a clear strict error, listing what is - loaded.""" - text = _run_tool("refine_plan_sketch", { - "plan": "Move(block0:block)[0.1] ~ nope", - "timeout": 10, - }, - sandbox_dir=str(tmp_path)) - assert "Could not parse plan sketch" in text - assert "unknown ground sampler 'nope'" in text +def test_probe_refine_unknown_named_sampler(tmp_path): + """An unresolvable `~ name` is a clear strict error.""" + with pytest.raises(ValueError, match="unknown ground sampler 'nope'"): + _probe_refine("Move(block0:block)[0.1] ~ nope", + sandbox_dir=str(tmp_path)) -def test_refine_plan_sketch_tool_broken_ground_samplers_file(tmp_path): +def test_probe_refine_broken_ground_samplers_file(tmp_path): """A ground_samplers.py that fails to exec is surfaced as an error the agent can fix, not silently ignored.""" (tmp_path / "ground_samplers.py").write_text("raise RuntimeError('bad')\n", encoding="utf-8") - text = _run_tool("refine_plan_sketch", { - "plan": "Move(block0:block)[0.95]", - "timeout": 10, - }, - sandbox_dir=str(tmp_path)) - assert "Error loading" in text - assert "bad" in text - assert "Parameters found" not in text + with pytest.raises(ValueError, match="Error loading") as excinfo: + _probe_refine("Move(block0:block)[0.95]", sandbox_dir=str(tmp_path)) + assert "bad" in str(excinfo.value) diff --git a/tests/agent_sdk/test_bilevel_sketch_samplers.py b/tests/agent_sdk/test_bilevel_sketch_samplers.py index cee192236..93c58a859 100644 --- a/tests/agent_sdk/test_bilevel_sketch_samplers.py +++ b/tests/agent_sdk/test_bilevel_sketch_samplers.py @@ -638,9 +638,9 @@ def test_execute_plan_forward_continues_past_zero_action_failure(): def test_execute_plan_forward_stop_on_failure_aborts(): - """With stop_on_failure (the evaluate_option_plan path), a 0-action step - aborts execution like the real executor: later steps don't run and the goal - is not reached.""" + """With stop_on_failure (the submit_plan path), a 0-action step aborts + execution like the real executor: later steps don't run and the goal is not + reached.""" plan = [ _Stuck.ground([_block], np.array([0.5], dtype=np.float32)), _Move.ground([_block], np.array([0.95], dtype=np.float32)), @@ -728,8 +728,8 @@ def test_execute_plan_forward_not_initiable_stops(): def test_refine_and_validate_report_returns_plan(): """refine_and_validate_report yields (success, report, plan). - The grounded plan is what refine_plan_sketch captures so the - approach can return the simulator-verified answer directly. + The grounded plan is what the refinement captures so the approach + can return the simulator-verified answer directly. """ step = SketchStep(option=_Move, objects=[_block], diff --git a/tests/agent_sdk/test_capture_decision.py b/tests/agent_sdk/test_capture_decision.py index 7e2c43354..21850fdbe 100644 --- a/tests/agent_sdk/test_capture_decision.py +++ b/tests/agent_sdk/test_capture_decision.py @@ -1,6 +1,6 @@ """Direct unit tests for the pure ``_decide_capture`` function. -The e2e harness (``test_evaluate_option_plan_capture.py``) drives the +The e2e harness (``test_submit_plan_capture.py``) drives the same policy through the real tool handler; these tests pin the decision table itself, one test per :class:`CaptureDecision` case, including guard combinations the e2e tests do not reach: diff --git a/tests/agent_sdk/test_log_formatter.py b/tests/agent_sdk/test_log_formatter.py index c15eb2c31..16e6b5474 100644 --- a/tests/agent_sdk/test_log_formatter.py +++ b/tests/agent_sdk/test_log_formatter.py @@ -87,7 +87,7 @@ def test_tool_use_multiline_code_gets_python_fence(): "assistant", "content": [{ "type": "tool_use", - "name": "explore_python", + "name": "run_python", "id": "toolu_9", "input": { "code": "x = 1\nprint(x)", @@ -96,7 +96,7 @@ def test_tool_use_multiline_code_gets_python_fence(): }], }] md = format_conversation_markdown(collected) - assert "**Tool Call:** `explore_python` (id: `toolu_9`)" in md + assert "**Tool Call:** `run_python` (id: `toolu_9`)" in md assert "*code:*" in md assert "```python\nx = 1\nprint(x)\n```" in md # The remaining scalar goes into a compact JSON block. diff --git a/tests/agent_sdk/test_predicate_quality_loader.py b/tests/agent_sdk/test_predicate_quality_loader.py new file mode 100644 index 000000000..1515e2209 --- /dev/null +++ b/tests/agent_sdk/test_predicate_quality_loader.py @@ -0,0 +1,109 @@ +"""Tests for the ``sim.predicates()`` loader (make_predicate_quality_loader). + +Drives the loader through the probe with a stub approach and a fake +trajectory (no PyBullet): the report scores milestone behaviour and the +loaded draft replaces the approach's learned predicate set. +""" +# pylint: disable=protected-access +from typing import Any, Dict, Set, cast + +import numpy as np +from gym.spaces import Box + +from predicators.agent_sdk.belief_probe import BeliefProbe +from predicators.agent_sdk.tools import ToolContext, \ + make_predicate_quality_loader +from predicators.structs import Action, GroundAtom, LowLevelTrajectory, \ + Object, ParameterizedOption, Predicate, State, Task, Type + +_block_type = Type("block", ["x"]) +_block = Object("block0", _block_type) +_Kept = Predicate("Kept", [_block_type], lambda s, o: True) +_Move = ParameterizedOption( + "Move", + types=[_block_type], + params_space=Box(low=np.array([0.0], dtype=np.float32), + high=np.array([1.0], dtype=np.float32)), + policy=lambda _s, _m, _o, _p: Action(np.zeros(1, dtype=np.float32)), + initiable=lambda _s, _m, _o, _p: True, + terminal=lambda _s, _m, _o, _p: False, +) + + +def _state(x: float) -> State: + return State({_block: np.array([x], dtype=np.float32)}) + + +class _StubApproach: + """The minimal approach surface make_predicate_quality_loader uses.""" + + def __init__(self) -> None: + self._types = {_block_type} + self._kept_initial_predicates = {_Kept} + self._learned_predicates: Set[Predicate] = set() + self._train_tasks = [Task(_state(0.0), {GroundAtom(_Kept, [_block])})] + self._fitted_params: Dict[str, float] = {} + + def _get_all_options(self) -> Set[ParameterizedOption]: + return {_Move} + + +def _trajectory() -> LowLevelTrajectory: + states = [_state(0.0), _state(0.4), _state(0.8), _state(1.0)] + actions = [Action(np.zeros(1, dtype=np.float32)) for _ in states[:-1]] + return LowLevelTrajectory(states, actions) + + +_PREDICATES = """\ +LEARNED_PREDICATES = [ + Predicate("Hi", [block_type], lambda s, o: s.get(o[0], "x") >= 0.5), + Predicate("Kept", [block_type], lambda s, o: True), +] +""" + + +def _probe(tmp_path: Any, approach: _StubApproach) -> BeliefProbe: + loader = make_predicate_quality_loader( + predicates_file=str(tmp_path / "predicates.py"), + predicates_versions_dir=str(tmp_path / "predicates_versions"), + approach=cast(Any, approach), + trajectories=[_trajectory()], + cycle_index_provider=lambda: 1, + ) + ctx = ToolContext() + ctx.probe_artifact_loaders["predicates"] = loader + return BeliefProbe(ctx) + + +def test_probe_predicates_loads_scores_and_installs(tmp_path: Any) -> None: + """The report tags the snapshot, scores the milestone, skips the kept-name + collision, and the validated draft becomes the approach's learned set.""" + (tmp_path / "predicates.py").write_text(_PREDICATES, encoding="utf-8") + approach = _StubApproach() + text = _probe(tmp_path, approach).predicates() + assert text.startswith("[cycle_001_vers_001] Predicate quality report") + assert "Hi(block)" in text + assert "coverage: ever-T + ever-F" in text + assert "monotone (1↑ 0↓): 1" in text + assert "Skipped 'Kept' (collides with a kept env predicate)" in text + assert {p.name for p in approach._learned_predicates} == {"Hi"} + + +def test_probe_predicates_reports_a_missing_file(tmp_path: Any) -> None: + """A missing predicates.py is an actionable error, not a crash.""" + approach = _StubApproach() + text = _probe(tmp_path, approach).predicates() + assert "LEARNED_PREDICATES = [...]" in text + assert not approach._learned_predicates + + +def test_probe_predicates_unavailable_without_a_loader() -> None: + """Outside a predicate-invention session the probe has no predicates.py + surface and says so.""" + probe = BeliefProbe(ToolContext()) + try: + probe.predicates() + except RuntimeError as e: + assert "sim.predicates is unavailable" in str(e) + else: + raise AssertionError("sim.predicates() must raise without a loader") diff --git a/tests/agent_sdk/test_probe_synthesis.py b/tests/agent_sdk/test_probe_synthesis.py index 31b4ae068..be19b8abe 100644 --- a/tests/agent_sdk/test_probe_synthesis.py +++ b/tests/agent_sdk/test_probe_synthesis.py @@ -1,4 +1,4 @@ -"""Tests for the synthesis-phase behavior of the explore_python probe. +"""Tests for the synthesis-phase behavior of the ``sim`` probe. Covers the ``ctx.probe_option_model_provider`` hook: model resolution (candidate provider vs. the solve-phase ``ctx.option_model`` fallback), @@ -177,22 +177,22 @@ def _fake_fit(rules, specs, triples, features): def test_probe_descriptions_follow_phase() -> None: - """The probe surface follows the session: explore_python (solve-only) - carries the belief-simulator + evaluate_option_plan wording, while in - synthesis the probe rides inside run_python, whose description carries the - candidate-simulator + evaluate_plan_refinement wording.""" - utils.reset_config({"agent_planner_use_explore_python": True}) + """The probe surface follows the session: the solve-phase run_python + carries the belief-simulator + submit_plan wording, while the synthesis + run_python's description carries the candidate-simulator + + evaluate_plan_refinement wording.""" + utils.reset_config({}) def _desc(ctx: ToolContext) -> str: - (tool, ) = (t for t in create_mcp_tools(ctx, ["explore_python"]) - if getattr(t, "name", "") == "explore_python") + (tool, ) = (t for t in create_mcp_tools(ctx, ["run_python"]) + if getattr(t, "name", "") == "run_python") description = getattr(tool, "description", "") assert description return description solve_desc = _desc(ToolContext()) assert "belief simulator" in solve_desc - assert "evaluate_option_plan" in solve_desc + assert "submit_plan" in solve_desc # The solve namespace also carries the recorded real trajectories. assert "trajectories" in solve_desc assert "describe_trajectory" in solve_desc @@ -219,8 +219,6 @@ def _run_python_desc() -> str: # The fit/refine/forward-run protocol replaced the old validation # tool, and the probe is unconditional in synthesis sessions. assert "evaluate_plan_refinement" not in synth_desc - utils.reset_config({"agent_planner_use_explore_python": False}) - assert "CANDIDATE simulator" in _run_python_desc() def test_probe_namespace_contract() -> None: diff --git a/tests/agent_sdk/test_refine_evaluator_gate.py b/tests/agent_sdk/test_refine_evaluator_gate.py index ef7351e6c..221de3c46 100644 --- a/tests/agent_sdk/test_refine_evaluator_gate.py +++ b/tests/agent_sdk/test_refine_evaluator_gate.py @@ -1,22 +1,19 @@ -"""Evaluator gating tests for the ``refine_plan_sketch`` tool. - -Drives the real MCP tool handler with a fake option model (no -PyBullet). When the task has an evaluator, refinement success is gated -on its scoring: a parameterization that reaches the goal atoms but -scores as a non-solve is discarded and the search resamples with a -fresh rng; if every attempt scores as a non-solve the report is demoted -to FAILURE: SCORED_NON_SOLVE. The report speaks only in (terminated, -reward) terms - the certificate's reason strings never reach the agent. +"""Evaluator gating tests for ``sim.refine(require_solved=True)``. + +Drives the probe with a fake option model (no PyBullet). When the +task has an evaluator, refinement success is gated on its scoring: a +parameterization that reaches the goal atoms but scores as a non-solve +is discarded and the search keeps sampling; if no candidate is ever +certified the result is a FAILURE. The report speaks only in verdict +terms - the certificate's reason strings never reach the agent. """ -import asyncio -from typing import Any - import numpy as np from gym.spaces import Box from predicators import utils -from predicators.agent_sdk.tools import ToolContext, create_mcp_tools +from predicators.agent_sdk.belief_probe import BeliefProbe +from predicators.agent_sdk.tools import ToolContext from predicators.structs import Action, GroundAtom, LowLevelTrajectory, \ Object, ParameterizedOption, Predicate, State, Task, TaskEvaluator, Type @@ -41,7 +38,7 @@ def _noop_policy(_s, _m, _o, _p): terminal=lambda _s, _m, _o, _p: False, ) -_SKETCH_TEXT = "Move(block0:block) -> {ReachedHi(block0:block)}" +_SKETCH_TEXT = "Move(block0:block)[] -> {ReachedHi(block0:block)}" class _Model: @@ -83,10 +80,12 @@ def _certify(self, states, step_options, sim_env=None): return False, "band: outside the certified interval" -def _run_refine(evaluator, attempts=3): +def _run_refine(evaluator): utils.reset_config({ - "agent_bilevel_refine_evaluator_attempts": attempts, - "agent_bilevel_max_samples_per_step": 50, + # 200 uniform draws in [0, 1] make "no goal-reaching draw at all" + # (p = 0.1 each) a 1e-9 event, so the certified/rejected paths + # below are exercised at every probe seed, not just lucky ones. + "agent_bilevel_max_samples_per_step": 200, "agent_bilevel_use_llm_initial_params": False, }) init = State({_block: np.array([0.0], dtype=np.float32)}) @@ -103,46 +102,35 @@ def _run_refine(evaluator, attempts=3): option_model=model, current_task=task, ) - tools = { - t.name: t.handler - for t in create_mcp_tools(ctx, tool_names=["refine_plan_sketch"]) - } - try: - loop = asyncio.get_event_loop() - except RuntimeError: - loop = asyncio.new_event_loop() - asyncio.set_event_loop(loop) - result: Any = loop.run_until_complete(tools["refine_plan_sketch"]({ - "plan": - _SKETCH_TEXT, - "timeout": - 10, - })) - return result["content"][0]["text"] + probe = BeliefProbe(ctx).reset(task_idx=0) + return probe.refine(_SKETCH_TEXT, + timeout=10, + require_goal=True, + require_solved=True) def test_certified_refinement_reports_success(): """A wide certification band accepts the first goal-reaching params.""" goal = {GroundAtom(_ReachedHi, [_block])} - text = _run_refine(_BandEvaluator(goal, 0.9, 1.0)) + res = _run_refine(_BandEvaluator(goal, 0.9, 1.0)) + text = str(res) + assert res.success assert "SUCCESS" in text - assert "Parameters found" in text - assert "reward=" in text - assert "solved=True" in text - assert "SCORED_NON_SOLVE" not in text + assert "evaluator-solved" in res.verdict + assert 0.9 <= float(res.plan_lines[0].split("[")[1].split("]")[0]) assert "legitimate" not in text def test_all_non_solve_attempts_demote_to_failure(): """An empty certification band makes every goal-reaching rollout a - non-solve: the report is demoted, params are withheld, and no reason + non-solve: the search fails, no params are certified, and no reason string leaks.""" goal = {GroundAtom(_ReachedHi, [_block])} - text = _run_refine(_BandEvaluator(goal, 2.0, 3.0)) - assert "FAILURE: SCORED_NON_SOLVE" in text - assert "scored every such rollout as a non-solve" in text - assert "change the sketch" in text - assert "Parameters found" not in text + res = _run_refine(_BandEvaluator(goal, 2.0, 3.0)) + text = str(res) + assert not res.success + assert "FAILURE" in text + assert not res.verdict assert "band: outside the certified interval" not in text assert "legitimate" not in text @@ -154,8 +142,8 @@ class _RejectFirstParamEvaluator(TaskEvaluator): Keyed on rollout content, not call count: ``reward()`` re-invokes ``_certify`` within one scoring pass, so a call counter would give inconsistent verdicts inside a single evaluation. Deterministically - exercises the resample path: attempt 1 reaches the goal atoms but - scores as a non-solve, attempt 2 (fresh rng, a different accepted + exercises the resample path: the first candidate reaches the goal + atoms but scores as a non-solve, a later draw (a different accepted parameter) is certified. """ @@ -173,13 +161,13 @@ def _certify(self, states, step_options, sim_env=None): def test_non_solve_attempt_recovered_by_resampling(): - """A discarded first attempt is resampled; the report notes the discard and - still withholds the reason string.""" + """A discarded first candidate is resampled past; the result is a certified + SUCCESS that still withholds the reason string.""" goal = {GroundAtom(_ReachedHi, [_block])} - text = _run_refine(_RejectFirstParamEvaluator(goal)) - assert "SUCCESS" in text - assert "Parameters found" in text - assert "1 earlier parameterization(s) reached the goal atoms" in text - assert "were discarded" in text + res = _run_refine(_RejectFirstParamEvaluator(goal)) + text = str(res) + assert res.success + assert "evaluator-solved" in res.verdict + assert res.total_samples >= 2 assert "stub: first parameterization rejected" not in text assert "legitimate" not in text diff --git a/tests/agent_sdk/test_sampler_synthesis_tools.py b/tests/agent_sdk/test_sampler_synthesis_tools.py index 1375faa12..9ab8f50de 100644 --- a/tests/agent_sdk/test_sampler_synthesis_tools.py +++ b/tests/agent_sdk/test_sampler_synthesis_tools.py @@ -1,4 +1,4 @@ -"""Tests for the ``evaluate_sampler`` sampler-synthesis MCP tool. +"""Tests for the ``sim.samplers()`` loader (make_sampler_loader). Drives the real tool handler against a stub approach: loading ``LEARNED_SAMPLERS`` from ``samplers.py``, installing the validated dict @@ -6,16 +6,16 @@ versioning, and the sanity check's empty-subgoal-set contract. """ -import asyncio from typing import Any, Dict, Set import numpy as np from gym.spaces import Box from predicators import utils +from predicators.agent_sdk.belief_probe import BeliefProbe from predicators.agent_sdk.proposal_exec import build_exec_context, \ load_ground_samplers -from predicators.agent_sdk.tools import create_sampler_synthesis_tools +from predicators.agent_sdk.tools import ToolContext, make_sampler_loader from predicators.structs import Action, GroundAtom, Object, \ ParameterizedOption, Predicate, State, Task, Type @@ -36,7 +36,7 @@ class _StubApproach: - """The minimal approach surface create_sampler_synthesis_tools uses.""" + """The minimal approach surface make_sampler_loader uses.""" def __init__(self): init = State({_block: np.array([0.0], dtype=np.float32)}) @@ -52,27 +52,23 @@ def _get_all_options(self) -> Set[ParameterizedOption]: return {_Move} -def _run_evaluate_sampler(tmp_path, code=None): +def _run_sampler_loader(tmp_path, code=None): utils.reset_config({"seed": 0}) samplers_file = str(tmp_path / "samplers.py") if code is not None: with open(samplers_file, "w", encoding="utf-8") as f: f.write(code) approach = _StubApproach() - tools = create_sampler_synthesis_tools( + loader = make_sampler_loader( samplers_file=samplers_file, samplers_versions_dir=str(tmp_path / "samplers_versions"), approach=approach, cycle_index_provider=lambda: 1, ) - handlers = {t.name: t.handler for t in tools} - try: - loop = asyncio.get_event_loop() - except RuntimeError: - loop = asyncio.new_event_loop() - asyncio.set_event_loop(loop) - result: Any = loop.run_until_complete(handlers["evaluate_sampler"]({})) - return result["content"][0]["text"], approach + # Through the probe, exactly as the agent reaches it. + ctx = ToolContext() + ctx.probe_artifact_loaders["samplers"] = loader + return BeliefProbe(ctx).samplers(), approach _GOOD = """\ @@ -87,7 +83,7 @@ def _move_sampler(state, subgoal_atoms, rng, objects): def test_evaluate_sampler_installs_valid_samplers(tmp_path): """A valid samplers.py is installed onto the approach and passes the sanity check.""" - text, approach = _run_evaluate_sampler(tmp_path, _GOOD) + text, approach = _run_sampler_loader(tmp_path, _GOOD) assert "1 per-skill sampler(s) installed" in text assert "Move: OK" in text assert "3/3 within the params box" in text @@ -98,7 +94,7 @@ def test_evaluate_sampler_installs_valid_samplers(tmp_path): def test_evaluate_sampler_warns_unknown_option(tmp_path): """An entry keyed by a non-option name is skipped with a warning.""" code = _GOOD + "\nLEARNED_SAMPLERS['Teleport'] = _move_sampler\n" - text, approach = _run_evaluate_sampler(tmp_path, code) + text, approach = _run_sampler_loader(tmp_path, code) assert "Skipped 'Teleport' (not a known option name" in text assert set(approach._synthesized_samplers) == {"Move"} # pylint: disable=protected-access @@ -106,15 +102,15 @@ def test_evaluate_sampler_warns_unknown_option(tmp_path): def test_evaluate_sampler_warns_non_callable(tmp_path): """A non-callable value is skipped with a warning.""" code = _GOOD + "\nLEARNED_SAMPLERS['Move'] = 3.0\n" - text, approach = _run_evaluate_sampler(tmp_path, code) + text, approach = _run_sampler_loader(tmp_path, code) assert "Skipped 'Move' (value is not callable" in text assert not approach._synthesized_samplers # pylint: disable=protected-access def test_evaluate_sampler_reports_exec_error(tmp_path): """A samplers.py that raises at import time reports the traceback.""" - text, approach = _run_evaluate_sampler(tmp_path, - "raise RuntimeError('boom')") + text, approach = _run_sampler_loader(tmp_path, + "raise RuntimeError('boom')") assert "Error executing" in text assert "boom" in text assert not approach._synthesized_samplers # pylint: disable=protected-access @@ -122,19 +118,19 @@ def test_evaluate_sampler_reports_exec_error(tmp_path): def test_evaluate_sampler_reports_missing_symbol(tmp_path): """A file without LEARNED_SAMPLERS names the missing symbol.""" - text, _ = _run_evaluate_sampler(tmp_path, "x = 1\n") + text, _ = _run_sampler_loader(tmp_path, "x = 1\n") assert "LEARNED_SAMPLERS" in text def test_evaluate_sampler_missing_file_hint(tmp_path): """A missing samplers.py returns the Write hint, not a crash.""" - text, _ = _run_evaluate_sampler(tmp_path, code=None) + text, _ = _run_sampler_loader(tmp_path, code=None) assert "Use Write to create it" in text def test_evaluate_sampler_empty_dict_message(tmp_path): """An empty LEARNED_SAMPLERS asks for entries instead of sanity lines.""" - text, _ = _run_evaluate_sampler(tmp_path, "LEARNED_SAMPLERS = {}\n") + text, _ = _run_sampler_loader(tmp_path, "LEARNED_SAMPLERS = {}\n") assert "LEARNED_SAMPLERS is empty" in text assert "Sanity check" not in text @@ -145,22 +141,15 @@ def test_evaluate_sampler_version_tag_bumps_on_edit(tmp_path): utils.reset_config({"seed": 0}) samplers_file = tmp_path / "samplers.py" samplers_file.write_text(_GOOD, encoding="utf-8") - tools = create_sampler_synthesis_tools( + loader = make_sampler_loader( samplers_file=str(samplers_file), samplers_versions_dir=str(tmp_path / "samplers_versions"), approach=_StubApproach(), cycle_index_provider=lambda: 1, ) - handler = {t.name: t.handler for t in tools}["evaluate_sampler"] - try: - loop = asyncio.get_event_loop() - except RuntimeError: - loop = asyncio.new_event_loop() - asyncio.set_event_loop(loop) def _tag(): - result: Any = loop.run_until_complete(handler({})) - return result["content"][0]["text"].split("]")[0].lstrip("[") + return loader().split("]")[0].lstrip("[") tag1 = _tag() tag2 = _tag() # unchanged file: same tag (snapshot deduped) @@ -184,7 +173,7 @@ def _needs_subgoal(state, subgoal_atoms, rng, objects): LEARNED_SAMPLERS = {"Move": _needs_subgoal} """ - text, _ = _run_evaluate_sampler(tmp_path, code) + text, _ = _run_sampler_loader(tmp_path, code) assert "ERROR" in text assert "subgoal_atoms=set()" in text assert "must not crash on an empty set" in text @@ -235,6 +224,18 @@ def _bad_shape(state, subgoal_atoms, rng, objects): LEARNED_SAMPLERS = {"Move": _bad_shape} """ - text, _ = _run_evaluate_sampler(tmp_path, code) + text, _ = _run_sampler_loader(tmp_path, code) assert "ERROR" in text assert "returned shape (2,), expected (1,)" in text + + +def test_probe_samplers_unavailable_without_a_loader(): + """Outside a sampler-synthesis session the probe has no samplers.py surface + and says so instead of silently doing nothing.""" + probe = BeliefProbe(ToolContext()) + try: + probe.samplers() + except RuntimeError as e: + assert "sim.samplers is unavailable" in str(e) + else: + raise AssertionError("sim.samplers() must raise without a loader") diff --git a/tests/agent_sdk/test_solve_prompt_strategy.py b/tests/agent_sdk/test_solve_prompt_strategy.py index 16d26d890..9e853303e 100644 --- a/tests/agent_sdk/test_solve_prompt_strategy.py +++ b/tests/agent_sdk/test_solve_prompt_strategy.py @@ -115,7 +115,7 @@ def test_explore_mode_experiment_delivery_contract() -> None: assert "This is an EXPLORE query" in prompt solve_prompt = _render(_make_task(None)) assert "This is an EXPLORE query" not in solve_prompt - assert "do NOT finish until evaluate_option_plan CONFIRMS" in solve_prompt + assert "do NOT finish until submit_plan CONFIRMS" in solve_prompt # The contract must survive param-free sketch mode too (the search # finds continuous params, but the delivery semantics are the same). sketch_prompt = _render_explore(_make_task(None), propose_params=False) @@ -151,8 +151,9 @@ def test_explore_mode_early_stop_note_credits_exploration_plans() -> None: assert ("The loop concludes early once the exploration plans solve " "training") in prompt assert "learned model solves training" not in prompt - assert "the plan still runs in full" in prompt - assert "falls back to the explicit parameters you proposed" in prompt + assert "runs in the real environment EXACTLY as written" in prompt + assert "nothing is searched or substituted" in prompt + assert "falls back to the explicit parameters" not in prompt assert "truncated just after that step" not in prompt diff --git a/tests/agent_sdk/test_solve_restart_journal.py b/tests/agent_sdk/test_solve_restart_journal.py index d8e214428..ba5820d92 100644 --- a/tests/agent_sdk/test_solve_restart_journal.py +++ b/tests/agent_sdk/test_solve_restart_journal.py @@ -1,13 +1,14 @@ """Tests for the solve journal and the wall-clock exploration budgets. -Covers the journal module (entry caps, prompt-injection trimming), the -``record_journal`` MCP tool, the cooperative probe deadline -(:class:`ProbeBudgetExceeded`), and ``explore_python``'s budget handling +Covers the journal module (entry caps, prompt-injection trimming, the +harness-owned attempt log), the cooperative probe deadline +(:class:`ProbeBudgetExceeded`), and ``run_python``'s budget handling (refusal after the attempt deadline, per-call timeout with partial output, ``[budget]`` footer). """ # pylint: disable=protected-access import asyncio +import os import time from typing import Any @@ -21,6 +22,8 @@ from predicators.structs import Action, GroundAtom, LowLevelTrajectory, \ Object, ParameterizedOption, Predicate, State, Task, Type +_A = journal_mod.ATTEMPTS_FILENAME # the harness writer's file + _block_type = Type("block", ["x"]) _block = Object("block0", _block_type) _ReachedHi = Predicate("ReachedHi", [_block_type], @@ -106,7 +109,8 @@ def test_journal_append_and_read(tmp_path): assert journal_mod.read_journal(sandbox) == "" assert journal_mod.append_entry(sandbox, "task 0 attempt 1 (auto)", "- outcome: no capture") is None - content = journal_mod.read_journal(sandbox) + content = journal_mod.read_journal(sandbox, + filename=journal_mod.ATTEMPTS_FILENAME) assert "### task 0 attempt 1 (auto)" in content assert "- outcome: no capture" in content @@ -116,7 +120,7 @@ def test_journal_entry_truncated_at_cap(tmp_path): sandbox = str(tmp_path) note = journal_mod.append_entry(sandbox, "big", "x" * 10000) assert note is not None and "truncated" in note - content = journal_mod.read_journal(sandbox, max_chars=10**6) + content = journal_mod.read_journal(sandbox, max_chars=10**6, filename=_A) assert "[entry truncated at the per-entry size cap]" in content assert len(content) < 5000 @@ -127,7 +131,7 @@ def test_journal_read_trims_head_at_entry_boundary(tmp_path): for i in range(20): journal_mod.append_entry(sandbox, f"entry {i}", f"body {i} " + "y" * 500) - content = journal_mod.read_journal(sandbox, max_chars=2000) + content = journal_mod.read_journal(sandbox, max_chars=2000, filename=_A) assert content.startswith("[journal truncated") assert "### entry 19" in content assert "### entry 0" not in content @@ -145,82 +149,54 @@ def test_journal_read_raw_and_restore(tmp_path): """read_raw snapshots faithfully and restore rolls entries back.""" sandbox = str(tmp_path) assert journal_mod.read_raw(None) is None - assert journal_mod.read_raw(sandbox) is None + assert journal_mod.read_raw(sandbox, filename=_A) is None journal_mod.append_entry(sandbox, "Agent notes (pre-test phase)", "- learning fact") - snapshot = journal_mod.read_raw(sandbox) + snapshot = journal_mod.read_raw(sandbox, filename=_A) assert snapshot is not None and "- learning fact" in snapshot journal_mod.append_entry(sandbox, "Agent notes (test task 0)", "- test-phase fact") - journal_mod.restore(sandbox, snapshot) - assert journal_mod.read_raw(sandbox) == snapshot + journal_mod.restore(sandbox, snapshot, filename=_A) + assert journal_mod.read_raw(sandbox, filename=_A) == snapshot # A None snapshot means no journal file existed: restore deletes. - journal_mod.restore(sandbox, None) - assert journal_mod.read_raw(sandbox) is None + journal_mod.restore(sandbox, None, filename=_A) + assert journal_mod.read_raw(sandbox, filename=_A) is None # Deleting an already-absent journal is a no-op, not an error. - journal_mod.restore(sandbox, None) + journal_mod.restore(sandbox, None, filename=_A) # --------------------------------------------------------------------------- -# record_journal tool +# attempt log (harness-owned file next to the agent's journal) # --------------------------------------------------------------------------- -def test_record_journal_tool_writes_stamped_entry(tmp_path): - """The tool appends an entry stamped with task and attempt.""" - utils.reset_config({"agent_solve_use_journal": True}) - ctx = _make_ctx(sandbox_dir=str(tmp_path)) - ctx.test_task_idx = 0 - ctx.attempt_index = 2 - text = _call(_get_tool(ctx, "record_journal"), - {"entry": "- tried yaw 0-15 deg, all stopped short"}) - assert "Recorded" in text - content = journal_mod.read_journal(str(tmp_path)) - assert "### Agent notes (test task 0, attempt 2)" in content - assert "tried yaw 0-15 deg" in content - - -def test_record_journal_tool_stamps_learning_cycle(tmp_path): - """During a synthesis session the header names the learning cycle.""" - utils.reset_config({"agent_solve_use_journal": True}) - ctx = _make_ctx(sandbox_dir=str(tmp_path)) - ctx.learn_cycle_index = 2 - # learn_cycle_index wins even if a stale test_task_idx is set. - ctx.test_task_idx = 0 - text = _call(_get_tool(ctx, "record_journal"), - {"entry": "- glue latch needs 3 in-zone steps"}) - assert "Recorded" in text - content = journal_mod.read_journal(str(tmp_path)) - assert "### Agent notes (learning cycle 2)" in content - - -def test_record_journal_tool_stamps_offline_learning(tmp_path): - """A negative cycle index (the offline pass) is labeled ``offline - learning``, not a numeric cycle.""" - utils.reset_config({"agent_solve_use_journal": True}) - ctx = _make_ctx(sandbox_dir=str(tmp_path)) - ctx.learn_cycle_index = -1 - text = _call(_get_tool(ctx, "record_journal"), - {"entry": "- demo shows the latch closing"}) - assert "Recorded" in text - content = journal_mod.read_journal(str(tmp_path)) - assert "### Agent notes (offline learning)" in content - - -def test_record_journal_tool_rejects_empty(tmp_path): - """An empty entry is an error, not a silent no-op.""" - utils.reset_config({"agent_solve_use_journal": True}) - ctx = _make_ctx(sandbox_dir=str(tmp_path)) - text = _call(_get_tool(ctx, "record_journal"), {"entry": " "}) - assert "required" in text - - -def test_record_journal_tool_absent_when_disabled(tmp_path): - """With the journal flag off, the tool is not built at all.""" - utils.reset_config({"agent_solve_use_journal": False}) - ctx = _make_ctx(sandbox_dir=str(tmp_path)) - tools = {t.name for t in create_mcp_tools(ctx, ["record_journal"])} - assert "record_journal" not in tools +def test_attempt_log_is_a_separate_file(tmp_path): + """Harness entries land in attempts.md; the agent's journal.md is a plain + file the harness never writes, and each is read, snapshotted and restored + on its own.""" + sandbox = str(tmp_path) + assert journal_mod.append_entry(sandbox, "task 0 attempt 1/1 (auto)", + "- outcome: no capture") is None + assert not os.path.isfile(journal_mod.journal_path(sandbox)) + assert os.path.isfile(journal_mod.attempts_path(sandbox)) + assert journal_mod.read_journal(sandbox) == "" + attempts = journal_mod.read_journal(sandbox, + filename=journal_mod.ATTEMPTS_FILENAME) + assert "### task 0 attempt 1/1 (auto)" in attempts + # The agent writes its journal with the file tools. + with open(journal_mod.journal_path(sandbox), "w", encoding="utf-8") as f: + f.write("### task 0 attempt 1\n- tried x=0.5: stopped 3 cm short\n") + assert "stopped 3 cm short" in journal_mod.read_journal(sandbox) + snapshot = journal_mod.read_raw(sandbox, + filename=journal_mod.ATTEMPTS_FILENAME) + journal_mod.append_entry(sandbox, "task 1 attempt 1/1 (auto)", + "- outcome: captured") + journal_mod.restore(sandbox, + snapshot, + filename=journal_mod.ATTEMPTS_FILENAME) + assert "task 1" not in journal_mod.read_journal( + sandbox, filename=journal_mod.ATTEMPTS_FILENAME) + assert "stopped 3 cm short" in journal_mod.read_journal(sandbox) # --------------------------------------------------------------------------- @@ -303,56 +279,54 @@ def test_probe_counts_rollouts(): # --------------------------------------------------------------------------- -# explore_python budgets +# run_python budgets # --------------------------------------------------------------------------- -def test_explore_python_refuses_after_attempt_deadline(tmp_path): +def test_run_python_refuses_after_attempt_deadline(tmp_path): """A call arriving past the attempt deadline is refused unrun.""" - utils.reset_config({"agent_planner_use_explore_python": True}) + utils.reset_config({}) ctx = _make_ctx(sandbox_dir=str(tmp_path)) ctx.attempt_start = time.monotonic() - 10.0 ctx.attempt_deadline = time.monotonic() - 1.0 - text = _call(_get_tool(ctx, "explore_python"), + text = _call(_get_tool(ctx, "run_python"), {"code": "print('should not run')"}) assert "wall-clock exploration budget" in text assert "should not run" not in text assert "[budget]" in text -def test_explore_python_call_timeout_returns_partial_output(tmp_path): +def test_python_call_timeout_returns_partial_output(tmp_path): """A per-call timeout stops the sweep and returns printed output.""" utils.reset_config({ - "agent_planner_use_explore_python": True, - "agent_sdk_explore_python_call_timeout": 1e-9, + "agent_sdk_python_call_timeout": 1e-9, }) ctx = _make_ctx(sandbox_dir=str(tmp_path)) ctx.attempt_start = time.monotonic() - text = _call(_get_tool(ctx, "explore_python"), + text = _call(_get_tool(ctx, "run_python"), {"code": "print('partial results'); sim.reset()"}) assert "partial results" in text assert "TIME BUDGET" in text assert "exceeded its" in text -def test_explore_python_budget_footer(tmp_path): +def test_run_python_budget_footer(tmp_path): """Results carry the [budget] footer with rollout deltas.""" utils.reset_config({ - "agent_planner_use_explore_python": True, - "agent_sdk_explore_python_call_timeout": 0, + "agent_sdk_python_call_timeout": 0, }) ctx = _make_ctx(sandbox_dir=str(tmp_path)) ctx.attempt_start = time.monotonic() ctx.attempt_deadline = ctx.attempt_start + 2700 code = "sim.reset(); print(sim.run('Move(block0:block)[0.95]', " \ "render=False).goal_reached)" - text = _call(_get_tool(ctx, "explore_python"), {"code": code}) + text = _call(_get_tool(ctx, "run_python"), {"code": code}) assert "[budget] attempt time" in text assert "/45 min" in text assert "sim rollouts this attempt: 1 (+1 this call)" in text -def test_explore_python_watchdog_stops_sim_free_code(tmp_path): +def test_run_python_watchdog_stops_sim_free_code(tmp_path): """Pure-Python code that never touches the probe is hard-stopped. exec() blocks the event loop, so cooperative checks and the sandbox @@ -360,8 +334,7 @@ def test_explore_python_watchdog_stops_sim_free_code(tmp_path): preemption that reaches a sim-free loop. """ utils.reset_config({ - "agent_planner_use_explore_python": True, - "agent_sdk_explore_python_call_timeout": 0.3, + "agent_sdk_python_call_timeout": 0.3, }) ctx = _make_ctx(sandbox_dir=str(tmp_path)) code = ("import time\n" @@ -371,19 +344,18 @@ def test_explore_python_watchdog_stops_sim_free_code(tmp_path): " pass\n" "print('done')\n") t0 = time.monotonic() - text = _call(_get_tool(ctx, "explore_python"), {"code": code}) + text = _call(_get_tool(ctx, "run_python"), {"code": code}) assert time.monotonic() - t0 < 5.0 assert "start" in text assert "TIME BUDGET" in text assert "done" not in text -def test_explore_python_call_timeout_exempts_synthesis_sessions(tmp_path): +def test_python_call_timeout_exempts_synthesis_sessions(tmp_path): """Synthesis probes (candidate simulator; slower rollouts, refits) are exempt from the per-call cap.""" utils.reset_config({ - "agent_planner_use_explore_python": True, - "agent_sdk_explore_python_call_timeout": 0.1, + "agent_sdk_python_call_timeout": 0.1, }) ctx = _make_ctx(sandbox_dir=str(tmp_path)) ctx.probe_option_model_provider = lambda: ctx.option_model @@ -392,19 +364,18 @@ def test_explore_python_call_timeout_exempts_synthesis_sessions(tmp_path): "while time.monotonic() - t0 < 0.3:\n" " pass\n" "print('done')\n") - text = _call(_get_tool(ctx, "explore_python"), {"code": code}) + text = _call(_get_tool(ctx, "run_python"), {"code": code}) assert "done" in text assert "TIME BUDGET" not in text -def test_explore_python_no_footer_outside_attempt(tmp_path): +def test_run_python_no_footer_outside_attempt(tmp_path): """No attempt in flight (e.g. exploration phase): no footer noise.""" utils.reset_config({ - "agent_planner_use_explore_python": True, - "agent_sdk_explore_python_call_timeout": 0, + "agent_sdk_python_call_timeout": 0, }) ctx = _make_ctx(sandbox_dir=str(tmp_path)) - text = _call(_get_tool(ctx, "explore_python"), {"code": "print('hi')"}) + text = _call(_get_tool(ctx, "run_python"), {"code": "print('hi')"}) assert "[budget]" not in text @@ -425,15 +396,21 @@ def test_solve_prompt_includes_journal_section(): prompt = build_solve_prompt(task, all_predicates={_ReachedHi}, all_options={_Move}, - journal=journal_text) - assert "## Solve Journal" in prompt + journal="### notes\n- tried x=0.5", + attempts=journal_text) + assert "## Attempt Log" in prompt assert "- outcome: no capture" in prompt + assert "## Solve Journal" in prompt + assert "- tried x=0.5" in prompt assert "treat any recorded conclusion skeptically" in prompt + assert "./journal.md" in prompt + assert "record_journal" not in prompt # Without journal content the section is absent entirely. prompt_no_journal = build_solve_prompt(task, all_predicates={_ReachedHi}, all_options={_Move}) assert "## Solve Journal" not in prompt_no_journal + assert "## Attempt Log" not in prompt_no_journal def test_read_strategy_absent_present_and_truncated(tmp_path): @@ -450,3 +427,44 @@ def test_read_strategy_absent_present_and_truncated(tmp_path): assert content.startswith("HEADLINE") assert "[strategy truncated at the prompt cap" in content assert len(content) < 4300 + + +# --------------------------------------------------------------------------- +# run_python path argument +# --------------------------------------------------------------------------- + + +def test_run_python_path_runs_a_sandbox_file_in_the_namespace(tmp_path): + """``path`` executes a sandbox .py file in the same persistent namespace as + inline ``code``, so helpers developed as files are reusable.""" + utils.reset_config({"agent_sdk_python_call_timeout": 0}) + (tmp_path / "helpers.py").write_text( + "def double(v):\n return 2 * v\n\nprint('loaded')\n", + encoding="utf-8") + ctx = _make_ctx(sandbox_dir=str(tmp_path)) + tool = _get_tool(ctx, "run_python") + text = _call(tool, {"path": "helpers.py"}) + assert "loaded" in text + text = _call(tool, {"code": "print(double(21))"}) + assert "42" in text + + +def test_run_python_path_stays_inside_the_sandbox(tmp_path): + """A ``path`` that resolves outside the sandbox is refused unrun, as is a + call that passes neither or both arguments.""" + utils.reset_config({"agent_sdk_python_call_timeout": 0}) + sandbox = tmp_path / "sandbox" + sandbox.mkdir() + (tmp_path / "outside.py").write_text("print('escaped')\n", + encoding="utf-8") + ctx = _make_ctx(sandbox_dir=str(sandbox)) + tool = _get_tool(ctx, "run_python") + text = _call(tool, {"path": "../outside.py"}) + assert "must stay inside the sandbox" in text + assert "escaped" not in text + text = _call(tool, {"path": "missing.py"}) + assert "not a file" in text + text = _call(tool, {}) + assert "exactly one of" in text + text = _call(tool, {"code": "print(1)", "path": "missing.py"}) + assert "exactly one of" in text diff --git a/tests/agent_sdk/test_evaluate_option_plan_capture.py b/tests/agent_sdk/test_submit_plan_capture.py similarity index 94% rename from tests/agent_sdk/test_evaluate_option_plan_capture.py rename to tests/agent_sdk/test_submit_plan_capture.py index 993a6681f..95fb6108a 100644 --- a/tests/agent_sdk/test_evaluate_option_plan_capture.py +++ b/tests/agent_sdk/test_submit_plan_capture.py @@ -1,4 +1,4 @@ -"""Capture-gating tests for the ``evaluate_option_plan`` tool. +"""Capture-gating tests for the ``submit_plan`` tool. Drives the real MCP tool handler with a fake option model (no PyBullet), covering the two gates in front of ``ctx.solved_plan``: @@ -123,7 +123,7 @@ def _call_tool(ctx, plan_text=_PLAN_TEXT, extra_args=None): """Invoke the real tool handler once against ``ctx``.""" tools = { t.name: t.handler - for t in create_mcp_tools(ctx, tool_names=["evaluate_option_plan"]) + for t in create_mcp_tools(ctx, tool_names=["submit_plan"]) } try: loop = asyncio.get_event_loop() @@ -133,8 +133,7 @@ def _call_tool(ctx, plan_text=_PLAN_TEXT, extra_args=None): call_args = {"plan": plan_text} if extra_args: call_args.update(extra_args) - result: Any = loop.run_until_complete( - tools["evaluate_option_plan"](call_args)) + result: Any = loop.run_until_complete(tools["submit_plan"](call_args)) return result["content"][0]["text"] @@ -768,7 +767,7 @@ def test_validation_rollouts_arg_cannot_lower_the_gate(): def test_flaky_report_names_seeds_and_reproduction_path(): """A FLAKY rejection names each rollout's planner seed and tells the agent - how to reproduce the failed rollout (``rollout_seed``).""" + how to reproduce the failed rollout (``sim.run(plan, seed=S)``).""" model = _Model(succeed_first_n=1) text, _ = _run_tool(model, rollouts=3) from predicators.settings import \ @@ -776,49 +775,7 @@ def test_flaky_report_names_seeds_and_reproduction_path(): base = CFG.seed assert f"rollout 1 (planner seed {base}): goal reached" in text assert f"(planner seed {base + 1}): FAILED" in text - assert "rollout_seed=" in text - - -def test_rollout_seed_with_trials_runs_consecutive_seeds(): - """``rollout_seed=S`` + ``validation_rollouts=N`` runs N diagnostic trials - at planner seeds S..S+N-1 (mirroring ``sim.run(plan, trials=N, seed=S)``), - reports each with its seed, and still never captures.""" - model = _SeedRecordingModel2() - text, ctx = _run_tool(model, - rollouts=5, - extra_args={ - "rollout_seed": 900, - "validation_rollouts": 3 - }) - # Exactly the requested 3 trials - the configured gate (5) does not - # inflate a diagnostic run. - assert model.seeds == [900, 901, 902] - assert "Diagnostic trials: 3/3 reached the goal" in text - assert "rollout 2 (planner seed 901): goal reached" in text - assert "rollout 3 (planner seed 902): goal reached" in text - assert "Captured as the current answer" not in text - assert ctx.solved_plan is None - - -def test_rollout_seed_is_diagnostic_only(): - """A seeded rollout runs at exactly the given planner seed. - - It reports fully and is never captured - agent-chosen seeds must - not pass the capture gate. - """ - model = _SeedRecordingModel2() - text, ctx = _run_tool(model, rollouts=3, extra_args={"rollout_seed": 4242}) - assert "DIAGNOSTIC rollout at planner seed 4242" in text - assert model.seeds == [4242] # no validation repeats either - assert "Goal achieved: True" in text - assert "Captured as the current answer" not in text - assert "Validated" not in text - assert ctx.solved_plan is None - from predicators.settings import \ - CFG # pylint: disable=import-outside-toplevel - - # The base seed is restored after the seeded rollout. - assert CFG.seed != 4242 + assert "sim.run(plan, seed=" in text # --------------------------------------------------------------------------- diff --git a/tests/agent_sdk/test_evaluate_policy_capture.py b/tests/agent_sdk/test_submit_policy_capture.py similarity index 90% rename from tests/agent_sdk/test_evaluate_policy_capture.py rename to tests/agent_sdk/test_submit_policy_capture.py index f154466d4..511e34683 100644 --- a/tests/agent_sdk/test_evaluate_policy_capture.py +++ b/tests/agent_sdk/test_submit_policy_capture.py @@ -1,6 +1,6 @@ -"""Capture-gating tests for the ``evaluate_policy`` tool (policy mode). +"""Capture-gating tests for the ``submit_policy`` tool (policy mode). -Mirrors test_evaluate_option_plan_capture.py's fixtures: drives the real +Mirrors test_submit_plan_capture.py's fixtures: drives the real MCP handler with a fake option model, covering the policy-mode gates in front of ``ctx.solved_policy_source`` - multi-rollout validation with fresh policy memory per rollout, source snapshotting, the @@ -100,15 +100,15 @@ def _write_policy(sandbox_dir, source): def _call_tool(ctx, extra_args=None): tools = { t.name: t.handler - for t in create_mcp_tools(ctx, tool_names=["evaluate_policy"]) + for t in create_mcp_tools(ctx, tool_names=["submit_policy"]) } try: loop = asyncio.get_event_loop() except RuntimeError: loop = asyncio.new_event_loop() asyncio.set_event_loop(loop) - result: Any = loop.run_until_complete(tools["evaluate_policy"](extra_args - or {})) + result: Any = loop.run_until_complete(tools["submit_policy"](extra_args + or {})) return result["content"][0]["text"] @@ -171,17 +171,6 @@ def test_source_snapshot_not_rereading_file(tmp_path): assert "Move(block0:block)" in ctx.solved_policy_source -def test_diagnostic_seed_never_captures(tmp_path): - """A seeded diagnostic rollout is reported but never captured.""" - model = _Model() - text, ctx, _ = _run_tool(model, - tmp_path, - rollouts=1, - extra_args={"rollout_seed": 999}) - assert "DIAGNOSTIC rollout at planner seed 999" in text - assert ctx.solved_policy_source is None - - def test_recovered_option_failure_still_captures(tmp_path): """Closed-loop: a surfaced-and-recovered failure does not disqualify.""" model = _Model() diff --git a/tests/agent_sdk/test_tool_registry.py b/tests/agent_sdk/test_tool_registry.py index b99b5e6be..cfd89d4bf 100644 --- a/tests/agent_sdk/test_tool_registry.py +++ b/tests/agent_sdk/test_tool_registry.py @@ -12,8 +12,7 @@ from typing import Any, Iterable, List, Optional, Set, cast from predicators.agent_sdk.tools import ALL_TOOL_NAMES, BUILTIN_TOOLS, \ - MCP_SERVER_NAME, PREDICATE_SYNTHESIS_TOOL_NAMES, SYNTHESIS_TOOL_NAMES, \ - ToolContext, create_mcp_tools, create_predicate_synthesis_tools, \ + MCP_SERVER_NAME, SYNTHESIS_TOOL_NAMES, ToolContext, create_mcp_tools, \ create_synthesis_tools, get_allowed_tool_list, list_session_tool_names from predicators.approaches.agent_session_mixin import AgentSessionMixin @@ -32,29 +31,15 @@ def _names(tools: Iterable[Any]) -> Set[str]: def test_create_mcp_tools_matches_all_tool_names() -> None: """``create_mcp_tools`` exposes exactly the ``ALL_TOOL_NAMES`` names. - That holds when the config opts into every conditionally-built tool; - without the opt-ins the surface must NOT carry them (baseline arms - would otherwise gain the code-execution tool silently). - Conditionally-built tools MUST still appear in ``ALL_TOOL_NAMES``: - the session-open sanity check classifies any declared name outside - it as a dynamic tool and asserts when no builder attached it - (run_20260718_124622 failed every solve query because - ``record_journal`` was missing from the roster). + The session-open sanity check classifies any declared name outside + ``ALL_TOOL_NAMES`` as a dynamic tool and asserts when no builder + attached it, so a static tool missing from the roster fails every + query (run_20260718_124622 lost the whole solve phase that way). """ from predicators import utils - utils.reset_config({ - "agent_planner_use_explore_python": True, - "agent_solve_use_journal": True, - }) + utils.reset_config({}) tools = create_mcp_tools(ToolContext()) assert _names(tools) == set(ALL_TOOL_NAMES) - utils.reset_config({ - "agent_planner_use_explore_python": False, - "agent_solve_use_journal": False, - }) - tools = create_mcp_tools(ToolContext()) - disabled = {"explore_python", "record_journal"} - assert _names(tools) == set(ALL_TOOL_NAMES) - disabled def test_create_synthesis_tools_matches_constant(tmp_path) -> None: @@ -122,19 +107,6 @@ def test_sysid_fit_gate_traj_idxs_vs_fixed(tmp_path) -> None: assert "traj_idxs is empty" in out -def test_create_predicate_synthesis_tools_matches_constant(tmp_path) -> None: - """Predicate-synthesis builder matches the predicate-synthesis name - tuple.""" - approach_stub = SimpleNamespace(_fitted_params={}) - tools = create_predicate_synthesis_tools( - predicates_file=str(tmp_path / "predicates.py"), - predicates_versions_dir=str(tmp_path / "predicates_versions"), - approach=approach_stub, - trajectories=[], - ) - assert _names(tools) == set(PREDICATE_SYNTHESIS_TOOL_NAMES) - - def test_list_session_tool_names_defaults() -> None: """Default ``list_session_tool_names`` returns all MCP + builtin tools.""" grouped = list_session_tool_names() @@ -147,12 +119,12 @@ def test_list_session_tool_names_filters_and_combines() -> None: """Filtered MCP names drop unknowns; ``extra_mcp_tools`` pass through.""" fake = SimpleNamespace(name="run_python") grouped = list_session_tool_names( - mcp_filter=["inspect_options", "not_a_tool", "inspect_trajectories"], + mcp_filter=["submit_plan", "not_a_tool", "run_python"], extra_mcp_tools=[fake], include_builtin=False, ) assert grouped == { - "mcp": ["inspect_options", "inspect_trajectories"], + "mcp": ["submit_plan", "run_python"], "extra": ["run_python"], } @@ -171,18 +143,14 @@ def test_solve_and_synthesis_tool_names_are_independent() -> None: class _Approach(AgentSessionMixin): def _get_solve_tool_names(self) -> Optional[List[str]]: - return ["inspect_options", "evaluate_option_plan"] + return ["run_python", "submit_plan"] def _get_synthesis_tool_names(self) -> Optional[List[str]]: - return ["inspect_trajectories", "run_python"] + return ["run_python"] obj = _Approach() - assert obj._get_solve_tool_names() == [ - "inspect_options", "evaluate_option_plan" - ] - assert obj._get_synthesis_tool_names() == [ - "inspect_trajectories", "run_python" - ] + assert obj._get_solve_tool_names() == ["run_python", "submit_plan"] + assert obj._get_synthesis_tool_names() == ["run_python"] def test_get_allowed_tool_list_passes_dynamic_names_through() -> None: @@ -190,15 +158,15 @@ def test_get_allowed_tool_list_passes_dynamic_names_through() -> None: list is the single source of truth, with no silent filtering against ``ALL_TOOL_NAMES``.""" allowed = get_allowed_tool_list([ - "inspect_options", # static + "submit_plan", # static "run_python", # dynamic synthesis tool - "evaluate_predicate_quality", # dynamic predicate-synthesis + "my_dynamic_tool", # a dynamic tool the roster never lists ]) prefix = f"mcp__{MCP_SERVER_NAME}__" assert allowed == [ - f"{prefix}inspect_options", + f"{prefix}submit_plan", f"{prefix}run_python", - f"{prefix}evaluate_predicate_quality", + f"{prefix}my_dynamic_tool", ] @@ -207,7 +175,7 @@ def test_coercing_tool_accepts_numeric_strings() -> None: Coercion happens before the handler runs - harness-side validation used to hard-reject '0' for an integer arg, costing agents whole - tools (inspect_trajectories went 0-for-6 in one audited run). + tools (a trajectory-inspection tool went 0-for-6 in one audited run). """ import asyncio @@ -343,47 +311,15 @@ def test_agent_render_resolution() -> None: assert CFG.pybullet_camera_width == 900 -def test_explore_python_replaces_refine() -> None: - """When explore_python is on, the tools it subsumes are dropped unless - agent_planner_explore_python_keep_replaced_tools asks for both.""" - from predicators import utils - from predicators.approaches.agent_model_based_approach import \ - AgentModelBasedApproach - base = { - "env": "cover", - "approach": "agent_bilevel", - "agent_planner_use_simulator": True, - } - obj = object.__new__(AgentModelBasedApproach) - - utils.reset_config(base) - names = _required_names(obj._get_solve_tool_names()) - assert "refine_plan_sketch" in names - assert "explore_python" not in names - - utils.reset_config({**base, "agent_planner_use_explore_python": True}) - names = _required_names(obj._get_solve_tool_names()) - assert "explore_python" in names - assert "refine_plan_sketch" not in names # subsumed: sim.refine - - utils.reset_config({ - **base, "agent_planner_use_explore_python": True, - "agent_planner_explore_python_keep_replaced_tools": True - }) - names = _required_names(obj._get_solve_tool_names()) - assert "explore_python" in names - assert "refine_plan_sketch" in names - - -def test_synthesis_tool_names_explore_python() -> None: - """Synthesis sessions never surface explore_python: the probe rides inside - run_python's namespace (one exec namespace per session) and the inspect - digests are prompt-injected. +def test_synthesis_tool_names_run_python() -> None: + """Every session offers one ``run_python``: the synthesis roster carries + its own instance (fit data + the candidate-simulator probe in one + namespace), the solve roster the probe over the deployed belief model. Fitting, residual reports, plan validation, and scene work are probe methods (``sim.fit`` / ``sim.residuals`` / ``sim.refine`` / ``sim.run`` / ``sim.reset`` + ``sim.render``), not tools, so the - roster carries only ``run_python`` (+ per-arm evaluators). + synthesis roster carries only ``run_python`` (+ per-arm evaluators). """ from predicators import utils from predicators.approaches.agent_sim_learning_approach import \ @@ -396,43 +332,47 @@ def test_synthesis_tool_names_explore_python() -> None: invention = object.__new__(AgentSimPredicateInventionApproach) invention._do_synthesize_samplers = False - for use_probe_tool in (False, True): - utils.reset_config( - {"agent_planner_use_explore_python": use_probe_tool}) - names = _required_names(sim_learn._get_synthesis_tool_names()) - assert "explore_python" not in names # probe rides inside run_python - # Digests replaced the inspect tools on every synthesis surface. - assert not any(n.startswith("inspect_") for n in names) - assert "run_python" in names - names = _required_names(invention._get_synthesis_tool_names()) - assert "explore_python" not in names - assert "evaluate_plan_refinement" not in names # sim.refine + sim.run - assert "evaluate_step_fit" not in names # sim.fit - assert "evaluate_predicate_quality" in names - - # On the solve side the probe subsumes the remaining inspect tools - # (trajectories in its namespace, sim.task for the task digest). + utils.reset_config({}) + names = _required_names(sim_learn._get_synthesis_tool_names()) + assert names.count("run_python") == 1 + names = _required_names(invention._get_synthesis_tool_names()) + assert names.count("run_python") == 1 + assert "evaluate_predicate_quality" not in names # sim.predicates() + + # On the solve side every arm with a simulator gets the same surface: + # the probe (trajectories in its namespace, sim.task for the task + # digest) plus the submission tool. utils.reset_config({ "env": "cover", "approach": "agent_sim_predicate_invention", "agent_planner_use_simulator": True, - "agent_planner_use_explore_python": True, }) names = _required_names(invention._get_solve_tool_names()) - assert "explore_python" in names - assert not any(n.startswith("inspect_") for n in names) + assert names.count("run_python") == 1 + assert "submit_plan" in names - # Without the probe, the solve roster keeps the trajectory/task - # inspect tools (there is no namespace to subsume them into) but - # never inspect_options/inspect_types (digests are in the prompt). + # Without a simulator there is nothing to probe or validate against. utils.reset_config({ "env": "cover", "approach": "agent_sim_predicate_invention", - "agent_planner_use_simulator": True, - "agent_planner_use_explore_python": False, + "agent_planner_use_simulator": False, }) names = _required_names(invention._get_solve_tool_names()) - assert "inspect_trajectories" in names - assert "inspect_train_tasks" in names - assert "inspect_options" not in names - assert "inspect_types" not in names + assert "run_python" not in names + assert "submit_plan" not in names + + +def test_attached_run_python_replaces_the_static_instance() -> None: + """A session that attaches its own ``run_python`` (synthesis) gets. + + exactly that instance from ``create_mcp_tools`` - the solve-phase + probe instance is neither built nor offered alongside it. + """ + fake = SimpleNamespace(name="run_python", handler=None) + ctx = ToolContext() + ctx.extra_mcp_tools = [fake] + tools = create_mcp_tools(ctx, tool_names=["run_python"]) + assert tools == [fake] + tools = create_mcp_tools(ctx) + assert [t for t in tools + if getattr(t, "name", "") == "run_python"] == [fake] diff --git a/tests/approaches/test_agent_model_based_approach.py b/tests/approaches/test_agent_model_based_approach.py index ce4d083c4..f8bcd360d 100644 --- a/tests/approaches/test_agent_model_based_approach.py +++ b/tests/approaches/test_agent_model_based_approach.py @@ -1536,7 +1536,7 @@ def test_nudge_returns_captured_best_effort_plan(self): def _fake_query(message, **kwargs): del message, kwargs # unused - # Simulate evaluate_option_plan's best-effort capture. + # Simulate submit_plan's best-effort capture. assert approach._tool_context.capture_best_effort_plan approach._tool_context.solved_plan = plan approach._tool_context.solved_sketch = sketch diff --git a/tests/approaches/test_agent_sim_learning_approach.py b/tests/approaches/test_agent_sim_learning_approach.py index 71dd8b1a7..5f7857043 100644 --- a/tests/approaches/test_agent_sim_learning_approach.py +++ b/tests/approaches/test_agent_sim_learning_approach.py @@ -833,17 +833,14 @@ def test_base_sim_reference_provisioning() -> None: assert obj._base_sim_reference_paths() == [] -def test_synthesis_tool_names_gate_record_journal(): - """The learn session offers record_journal iff the journal is enabled.""" +def test_synthesis_tool_names_are_run_python_only(): + """The learn session's only tool is run_python; the journal is a plain file + the agent edits, whatever the journal flag says.""" stub = SimpleNamespace(_do_synthesize_samplers=False) - utils.reset_config({"agent_solve_use_journal": True}) - names = AgentSimLearningApproach._get_synthesis_tool_names(stub) - assert "record_journal" in names - utils.reset_config({"agent_solve_use_journal": False}) - names = AgentSimLearningApproach._get_synthesis_tool_names(stub) - assert "record_journal" not in names - # run_python is always present regardless of the journal flag. - assert "run_python" in names + for use_journal in (True, False): + utils.reset_config({"agent_solve_use_journal": use_journal}) + names = AgentSimLearningApproach._get_synthesis_tool_names(stub) + assert names == ["run_python"] # --------------------------------------------------------------------------- diff --git a/tests/approaches/test_agent_solve_restart.py b/tests/approaches/test_agent_solve_restart.py index daae0f3d8..7d98fdd9a 100644 --- a/tests/approaches/test_agent_solve_restart.py +++ b/tests/approaches/test_agent_solve_restart.py @@ -185,7 +185,8 @@ def test_journal_auto_entries_record_each_attempt(tmp_path): ]) approach._solve_attempt = script approach._solve(task, timeout=10) - content = journal_mod.read_journal(str(tmp_path)) + content = journal_mod.read_journal(str(tmp_path), + filename=journal_mod.ATTEMPTS_FILENAME) assert "### task 0 attempt 1/2 (auto)" in content assert "- outcome: no capture" in content assert "### task 0 attempt 2/2 (auto)" in content @@ -346,15 +347,17 @@ def test_journal_task_context_written_once_even_on_resolve(tmp_path): approach._solve_attempt = script approach._solve(task, timeout=10) approach._solve(task, timeout=10) - content = journal_mod.read_journal(str(tmp_path)) + content = journal_mod.read_journal(str(tmp_path), + filename=journal_mod.ATTEMPTS_FILENAME) assert content.count("### task 0 goal + initial state (auto)") == 1 assert content.index("- goal:") < content.index("- outcome:") def test_test_phase_journal_archived_and_rolled_back(tmp_path): - """Learning entries persist across evaluations; each evaluation's own - entries are archived outside the sandbox, then rolled back so the next - evaluation starts from learning knowledge only (no test-task leaks).""" + """Learning content persists across evaluations; each evaluation's own + additions (harness attempt-log entries and agent journal notes) are + archived outside the sandbox, then rolled back so the next evaluation + starts from learning knowledge only (no test-task leaks).""" sandbox = tmp_path / "sandbox" log_dir = tmp_path / "run_logs" approach, task = _make_approach( @@ -365,48 +368,59 @@ def test_test_phase_journal_archived_and_rolled_back(tmp_path): }, sandbox_dir=str(sandbox)) ctx = approach._tool_context - # A learning-phase entry, recorded before any evaluation. - journal_mod.append_entry(str(sandbox), "Agent notes (pre-test phase)", - "- learning fact") - # First evaluation: one test-task solve writes auto entries. + attempts = journal_mod.ATTEMPTS_FILENAME + # A learning-phase note, written by the agent before any evaluation. + sandbox.mkdir(parents=True, exist_ok=True) + (sandbox / journal_mod.JOURNAL_FILENAME).write_text( + "### learn cycle notes\n- learning fact\n", encoding="utf-8") + # First evaluation: one test-task solve writes attempt-log entries + # and the agent adds a note. approach.begin_test_phase() ctx.test_task_idx = 0 script = _AttemptScript(approach, [("validated", 0.95), ("validated", 0.96)]) approach._solve_attempt = script approach._solve(task, timeout=10) - content = journal_mod.read_journal(str(sandbox)) - assert "- learning fact" in content # learning knowledge visible in eval + with open(sandbox / journal_mod.JOURNAL_FILENAME, "a", + encoding="utf-8") as f: + f.write("### task 0 attempt 1\n- eval-time note\n") + content = journal_mod.read_journal(str(sandbox), filename=attempts) assert "### task 0 goal + initial state (auto)" in content + assert "- learning fact" in journal_mod.read_journal(str(sandbox)) approach.end_test_phase() - # Rolled back: the learning entry survives, eval entries are gone. - content = journal_mod.read_journal(str(sandbox)) - assert "- learning fact" in content - assert "task 0" not in content - # The full eval journal was archived outside the sandbox first, one - # copy per evaluation phase. This first evaluation precedes any - # online learning, so it archives as the initial test. - archived = (log_dir / "journal_eval_initial.md").read_text() - assert "- learning fact" in archived + # Rolled back: learning content survives, eval additions are gone. + assert "task 0" not in journal_mod.read_journal(str(sandbox), + filename=attempts) + notes = journal_mod.read_journal(str(sandbox)) + assert "- learning fact" in notes + assert "eval-time note" not in notes + # Both files were archived outside the sandbox first, one copy per + # evaluation phase. This first evaluation precedes any online + # learning, so it archives as the initial test. + archived = (log_dir / "attempts_eval_initial.md").read_text() assert "### task 0 goal + initial state (auto)" in archived + archived_notes = (log_dir / "journal_eval_initial.md").read_text() + assert "- learning fact" in archived_notes + assert "eval-time note" in archived_notes # Second evaluation on the same task, after a learning phase advanced # the cycle: the context-entry dedup key was rolled back too, so the - # goal + init entry is re-written (else the journal's attempt records - # would be uninterpretable). + # goal + init entry is re-written (else the attempt records would be + # uninterpretable). approach._online_learning_cycle = 1 approach.begin_test_phase() ctx.test_task_idx = 0 approach._solve(task, timeout=10) - content = journal_mod.read_journal(str(sandbox)) + content = journal_mod.read_journal(str(sandbox), filename=attempts) assert content.count("### task 0 goal + initial state (auto)") == 1 approach.end_test_phase() # The second evaluation ran after cycle 0's learn advanced the # counter to 1, so it archives under the 0-based cycle it evaluates. - assert sorted(p.name for p in log_dir.glob("journal_eval*.md")) == [ - "journal_eval_cycle0.md", "journal_eval_initial.md" + assert sorted(p.name for p in log_dir.glob("attempts_eval*.md")) == [ + "attempts_eval_cycle0.md", "attempts_eval_initial.md" ] assert journal_mod.read_raw(str(sandbox)) is not None - assert "task 0" not in journal_mod.read_journal(str(sandbox)) + assert "task 0" not in journal_mod.read_journal(str(sandbox), + filename=attempts) def test_test_phase_journal_rollback_noop_without_journal(tmp_path): @@ -438,7 +452,8 @@ def _attempt(_task): approach._solve_attempt = _attempt with pytest.raises(ApproachFailure): approach._solve(task, timeout=10) - content = journal_mod.read_journal(str(tmp_path)) + content = journal_mod.read_journal(str(tmp_path), + filename=journal_mod.ATTEMPTS_FILENAME) assert ("- best refused submission (evaluator reward -0.05, " "not captured):") in content assert "Move(block0:block)[0.87]" in content diff --git a/tests/approaches/test_bridge_policy_approach.py b/tests/approaches/test_bridge_policy_approach.py index 9c1b63d73..500bdf97d 100644 --- a/tests/approaches/test_bridge_policy_approach.py +++ b/tests/approaches/test_bridge_policy_approach.py @@ -14,7 +14,7 @@ BridgePolicyApproach, RLBridgePolicyApproach from predicators.bridge_policies import BridgePolicyDone from predicators.cogman import CogMan -from predicators.envs import get_or_create_env +from predicators.envs import create_new_env from predicators.execution_monitoring import create_execution_monitor from predicators.ground_truth_models import get_gt_options from predicators.perception import create_perceiver @@ -42,7 +42,7 @@ def test_bridge_policy_approach(): "num_test_tasks": 1, } utils.reset_config(args) - env = get_or_create_env(CFG.env) + env = create_new_env(CFG.env) train_tasks = [t.task for t in env.get_train_tasks()] test_tasks = [t.task for t in env.get_test_tasks()] approach = BridgePolicyApproach(env.predicates, @@ -131,7 +131,7 @@ def second_policy(s): "num_test_tasks": 1, } utils.reset_config(args) - env = get_or_create_env(CFG.env) + env = create_new_env(CFG.env) train_tasks = [t.task for t in env.get_train_tasks()] test_tasks = [t.task for t in env.get_test_tasks()] approach = BridgePolicyApproach(env.predicates, @@ -163,7 +163,7 @@ def second_policy(s): "num_test_tasks": 1, } utils.reset_config(args) - env = get_or_create_env(CFG.env) + env = create_new_env(CFG.env) train_tasks = [t.task for t in env.get_train_tasks()] test_tasks = [t.task for t in env.get_test_tasks()] approach = BridgePolicyApproach(env.predicates, @@ -253,7 +253,7 @@ def test_rl_bridge_policy_approach(): "max_initial_demos": 0 } utils.reset_config(args) - env = get_or_create_env(CFG.env) + env = create_new_env(CFG.env) train_tasks = [t.task for t in env.get_train_tasks()] approach = RLBridgePolicyApproach(env.predicates, get_gt_options(env.get_name()), diff --git a/tests/approaches/test_published_fit_reuse.py b/tests/approaches/test_published_fit_reuse.py new file mode 100644 index 000000000..bc27d4a00 --- /dev/null +++ b/tests/approaches/test_published_fit_reuse.py @@ -0,0 +1,81 @@ +"""The deployed model is the agent's published ``sim.fit``. + +``_publish_probe_fit`` keeps the full FitResult; after the session the +approach reuses it when it fitted exactly the final simulator.py over +exactly the deployed parameter set, and falls back to a harness fit +otherwise. +""" +# pylint: disable=protected-access +from typing import Any + +import numpy as np + +from predicators.agent_sdk.tools import ToolContext +from predicators.approaches.agent_sim_learning_approach import \ + AgentSimLearningApproach +from predicators.code_sim_learning.fit_space import FitResult + + +def _fit(names: Any, values: Any) -> FitResult: + return FitResult(names=list(names), + samples=np.array([values], dtype=float), + log_probs=np.zeros(1), + jacobian=np.ones((3, len(names))), + noise_sigma=0.1, + prior_sigma=np.ones(len(names))) + + +def _approach() -> AgentSimLearningApproach: + approach = object.__new__(AgentSimLearningApproach) + approach._fitted_params = {} + approach._tool_context = ToolContext() + return approach + + +def test_published_fit_is_reused_for_the_fitted_file(tmp_path: Any) -> None: + """A canonical fit of the current file over the deployed parameter set is + returned with its SSE and version; a later edit or a changed parameter set + makes it stale.""" + sim_file = tmp_path / "simulator.py" + sim_file.write_text("RESIDUAL_RULES = []\n", encoding="utf-8") + approach = _approach() + assert approach._published_fit_for_file(str(sim_file), ["k"]) is None + + fit = _fit(["k", "gap"], [1.5, 0.02]) + approach._publish_probe_fit({ + "k": 1.5, + "gap": 0.02 + }, + "cycle_001_vers_003", + str(sim_file), + fit_result=fit, + sse=0.25) + assert approach._fitted_params == {"k": 1.5, "gap": 0.02} + assert approach._tool_context.probe_param_status == \ + "fitted (cycle_001_vers_003)" + published = approach._published_fit_for_file(str(sim_file), ["gap", "k"]) + assert published is not None + got, sse, version = published + assert got is fit + assert sse == 0.25 + assert version == "cycle_001_vers_003" + + # A different parameter set (spec added after the fit) is stale. + assert approach._published_fit_for_file(str(sim_file), + ["k", "gap", "mu"]) is None + # An UNFITTED edit of the file is stale. + sim_file.write_text("RESIDUAL_RULES = [] # edited\n", encoding="utf-8") + assert approach._published_fit_for_file(str(sim_file), ["k", "gap"]) \ + is None + + +def test_publish_without_a_fit_result_never_deploys(tmp_path: Any) -> None: + """Legacy publishes (values only) deploy to the probe but cannot stand in + for the cycle's fit.""" + sim_file = tmp_path / "simulator.py" + sim_file.write_text("RESIDUAL_RULES = []\n", encoding="utf-8") + approach = _approach() + approach._publish_probe_fit({"k": 2.0}, "cycle_001_vers_001", + str(sim_file)) + assert approach._fitted_params == {"k": 2.0} + assert approach._published_fit_for_file(str(sim_file), ["k"]) is None diff --git a/tests/explorers/test_agent_bilevel_explorer.py b/tests/explorers/test_agent_bilevel_explorer.py index d00e39427..60fe7f1ac 100644 --- a/tests/explorers/test_agent_bilevel_explorer.py +++ b/tests/explorers/test_agent_bilevel_explorer.py @@ -143,7 +143,6 @@ def _reset_config(**overrides): "num_test_tasks": 1, "seed": 42, "agent_bilevel_max_samples_per_step": 5, - "agent_bilevel_explorer_max_samples_per_step": 5, "agent_bilevel_check_subgoals": True, "agent_bilevel_log_state": False, "agent_explorer_fallback_to_random": True, @@ -213,93 +212,65 @@ def test_happy_path_returns_policy_and_stashes_subgoals(): assert query.await_count == 1 -def test_wait_memory_injection_on_refine(): - """Wait step with subgoal should have wait_target_atoms injected.""" +def test_wait_memory_injection_on_grounding(): + """A Wait step's annotated subgoal rides on the grounded option as + ``wait_target_atoms`` so WaitOption terminates on the intended atoms.""" _reset_config() - - captured: list = [] - - def side_effect(_state, option): - captured.append(option) - return (_make_state({_block0: [0.5, 0.6, 0.0]}), 3) - - option_model = MagicMock() - option_model.get_next_state_and_num_actions.side_effect = side_effect - - plan_text = ("Wait(robot0:robot) -> {On(block0:block, block1:block)}\n") - query = AsyncMock(return_value=_assistant_response(plan_text)) - explorer, _ = _make_explorer(option_model, query) - - explorer._get_exploration_strategy(0, timeout=5) - assert captured, "option_model was not invoked" - wait_opt = captured[0] - assert wait_opt.name == "Wait" - assert "wait_target_atoms" in wait_opt.memory - assert wait_opt.memory["wait_target_atoms"] == { + explorer, _ = _make_explorer(MagicMock(), None) + step = SketchStep(option=_Wait, + objects=[_robot], + subgoal_atoms={GroundAtom(_On, [_block0, _block1])}) + plan = explorer._ground_sketch_verbatim([step]) + assert len(plan) == 1 and plan[0].name == "Wait" + assert plan[0].memory["wait_target_atoms"] == { GroundAtom(_On, [_block0, _block1]) } -def test_plan_truncates_at_deepest_subgoal_failure_after_backtracking(): - """Regression: explorer returns the prefix up to (and including) the - deepest step whose subgoal backtracking couldn't satisfy. - - Reproduces the boil-task bug: the agent sketches ``Pick → Wait(Holding) - → Place`` and the mental model's Wait does NOT produce ``Holding``. - Backtracking runs normally — it retries Pick with different params - and re-runs Wait each time — but since the mental model simply can't - produce Holding under any params, Wait's subgoal keeps failing. - After exhaustion, the explorer returns ``[Pick, Wait]`` with the last - grounded attempts. Place is NEVER executed because refinement never - gets past Wait. - """ - _reset_config() - - # Mental model post-state: Holding(block0) NEVER holds (held=0). - no_holding_state = _make_state({_block0: [0.1, 0.2, 0.0]}) +def test_sketch_executes_verbatim_without_belief_refinement(): + """The agent's explicit parameters execute exactly as written: the belief + model is never rolled, the verdict is not-certified, and the cycle record + shows the executed values.""" + _reset_config(agent_bilevel_use_llm_initial_params=True) option_model = MagicMock() - option_model.get_next_state_and_num_actions.return_value = ( - no_holding_state, 3) - - plan_text = ("Pick(block0:block)\n" - "Wait(robot0:robot) -> {Holding(block0:block)}\n" - "Place(block0:block, block1:block) -> " - "{On(block0:block, block1:block)}\n") + plan_text = ("```\nPick(block0:block)[0.42] -> {Holding(block0:block)}\n" + "Place(block0:block, block1:block)[0.11, 0.22] -> " + "{On(block0:block, block1:block)}\n```") query = AsyncMock(return_value=_assistant_response(plan_text)) explorer, tool_context = _make_explorer(option_model, query) - - policy, _ = explorer._get_exploration_strategy(0, timeout=5) - assert callable(policy) - - # All three sketch steps recorded in metadata — the SKETCH is the full - # agent output; the TRUNCATION only applies to the refined plan. + policy, term_fn = explorer._get_exploration_strategy(0, timeout=5) + assert callable(policy) and term_fn(_make_state()) is False + assert not option_model.get_next_state_and_num_actions.called + assert tool_context.last_mental_model_solved is False + record = tool_context.cycle_scheduled_plans[-1] + assert "Pick(block0)[0.4200]" in record + assert "Place(block0, block1)[0.1100, 0.2200]" in record + assert "-> {On(block0:block, block1:block)}" in record + assert "without belief-model certification" in record assert tool_context.last_sketch_options == [ ("Pick", ["block0"]), - ("Wait", ["robot0"]), ("Place", ["block0", "block1"]), ] - executed_names = [ - call.args[1].name - for call in option_model.get_next_state_and_num_actions.call_args_list + +def test_missing_params_get_one_draw_from_the_box(): + """A step the agent left without parameters is grounded on one uniform. + + draw from the option's box - no search, and no crash. + """ + _reset_config(agent_bilevel_use_llm_initial_params=True) + explorer, _ = _make_explorer(MagicMock(), None) + steps = [ + SketchStep(option=_Pick, objects=[_block0], subgoal_atoms=None), + SketchStep(option=_Place, + objects=[_block0, _block1], + subgoal_atoms=None, + initial_params=np.array([0.5], dtype=np.float32)), ] - # Pick and Wait were each executed at least once (backtracking likely - # retried Pick multiple times). - assert "Pick" in executed_names - assert "Wait" in executed_names - # Place must NEVER be executed in the mental model: backtracking never - # got past the Wait subgoal failure, so Place never reached sample_fn. - assert "Place" not in executed_names, ( - "Place must not be executed in the mental model — refinement " - f"should have stalled at Wait's unsatisfiable subgoal, got " - f"{executed_names}") - # Pick has params (5 max_samples_per_step in test config), Wait has none. - # Each backtracking cycle runs Pick + Wait once, so we expect roughly - # 2 * max_samples_per_step mental-model calls — confirm backtracking - # actually exercised the upstream retries (at least 2 Picks). - assert executed_names.count("Pick") >= 2, ( - "Backtracking should have retried Pick at least twice before " - f"giving up, got {executed_names}") + plan = explorer._ground_sketch_verbatim(steps) + assert plan[0].params.shape == (1, ) and 0.0 <= plan[0].params[0] <= 1.0 + # Wrong arity counts as missing. + assert plan[1].params.shape == (2, ) def _make_captured(pick_params, place_params): @@ -319,20 +290,15 @@ def _make_captured(pick_params, place_params): def test_recovers_captured_plan_when_final_text_unparseable(): - """Agent validates a plan via evaluate_option_plan but ends in prose: + """Agent validates a plan via submit_plan but ends in prose: explorer recovers the captured plan instead of falling back to - random, and seeds the captured continuous params into refinement. + random and executes it at the captured continuous params. """ _reset_config() - - goal_state = _make_state({_block0: [0.5, 0.6, 0.0]}) option_model = MagicMock() - option_model.get_next_state_and_num_actions.return_value = (goal_state, 3) - pick_params, place_params = [0.42], [0.11, 0.22] grounded_plan, captured_sketch = _make_captured(pick_params, place_params) - explorer, tool_context = _make_explorer(option_model, None) async def query_impl(_msg, **_kw): @@ -344,9 +310,7 @@ async def query_impl(_msg, **_kw): return _assistant_response("Solved it. Plan: 1. pick 2. place. Done.") explorer._agent_session.query = query_impl - policy, term_fn = explorer._get_exploration_strategy(0, timeout=5) - # Recovered (not random fallback): subgoals/options come from the capture. assert callable(policy) assert term_fn(_make_state()) is False @@ -357,58 +321,11 @@ async def query_impl(_msg, **_kw): # The capture was consumed (cleared) so it can't leak into a later solve. assert tool_context.solved_plan is None assert tool_context.solved_sketch is None - # Captured params were seeded as initial_params: the option model is - # invoked with them (Pick tries them first; Place's are pooled). - called = [ - c.args[1] - for c in option_model.get_next_state_and_num_actions.call_args_list - ] - pick_calls = [o for o in called if o.name == "Pick"] - place_calls = [o for o in called if o.name == "Place"] - assert pick_calls and place_calls - np.testing.assert_allclose(pick_calls[0].params, pick_params) - assert any( - np.allclose(o.params, place_params) for o in place_calls), \ - "captured Place params were not seeded into refinement" - - -def test_captured_params_seed_info_gain_search(): - """With info-seeking ON, the recovered capture's continuous params are - seeded as candidates in the info-gain pool (not replayed verbatim).""" - _reset_config(agent_explorer_info_seeking=True, - agent_explorer_info_n_feasible_target=2, - agent_bilevel_explorer_max_samples_per_step=4) - - goal_state = _make_state({_block0: [0.5, 0.6, 0.0]}) - option_model = MagicMock() - option_model.get_next_state_and_num_actions.return_value = (goal_state, 3) - - place_params = [0.33, 0.44] - grounded_plan, captured_sketch = _make_captured([0.42], place_params) - - explorer, tool_context = _make_explorer(option_model, None) - # Wire a trivial ensemble scorer so info-seeking engages on annotated - # steps; constant score means the seeded candidate is chosen. - tool_context.atom_disagreement_fn = lambda _s, _atoms: 0.0 - - async def query_impl(_msg, **_kw): - tool_context.solved_plan = grounded_plan - tool_context.solved_sketch = captured_sketch - return _assistant_response("Done — summary only, no sketch block.") - - explorer._agent_session.query = query_impl - - policy, _ = explorer._get_exploration_strategy(0, timeout=5) - assert callable(policy) - # Place is the subgoal-annotated step that info-seeking pools; its captured - # params must appear among the candidates the pool evaluated. - place_calls = [ - c.args[1] - for c in option_model.get_next_state_and_num_actions.call_args_list - if c.args[1].name == "Place" - ] - assert any(np.allclose(o.params, place_params) for o in place_calls), \ - "captured Place params were not seeded into the info-gain pool" + # The captured params execute verbatim; the belief is not re-rolled. + assert not option_model.get_next_state_and_num_actions.called + record = tool_context.cycle_scheduled_plans[-1] + assert "Pick(block0)[0.4200]" in record + assert "Place(block0, block1)[0.1100, 0.2200]" in record def test_fallback_when_query_fails_and_flag_on(): @@ -492,9 +409,7 @@ def test_certified_capture_executes_verbatim_and_is_replayed(): True) is executed verbatim as a solve attempt with a True mental-model verdict; the cycle's next request on the task replays it with no new query.""" - _reset_config(agent_explorer_info_seeking=True, - agent_explorer_info_n_feasible_target=2, - agent_bilevel_explorer_max_samples_per_step=4) + _reset_config(agent_explorer_info_seeking=True) option_model = MagicMock() option_model.get_next_state_and_num_actions.return_value = (_make_state( {_block0: [0.5, 0.6, 0.0]}), 3) @@ -536,13 +451,12 @@ async def query_impl(msg, **_kw): assert tool_context.last_mental_model_solved is True -def test_uncertified_capture_still_seeds_the_search(): - """A capture whose gate verdict is not True (best-effort, flaky) keeps the - seed-then-search path and a non-True verdict.""" +def test_uncertified_capture_executes_its_plan_verbatim(): + """A capture whose gate verdict is not True (best-effort, flaky) is not + certified: it executes at its captured params as an experiment, with a + False mental-model verdict and no replay for the cycle.""" _reset_config() - goal_state = _make_state({_block0: [0.5, 0.6, 0.0]}) option_model = MagicMock() - option_model.get_next_state_and_num_actions.return_value = (goal_state, 3) grounded_plan, captured_sketch = _make_captured([0.42], [0.11, 0.22]) explorer, tool_context = _make_explorer(option_model, None) @@ -555,40 +469,6 @@ async def query_impl(_msg, **_kw): explorer._agent_session.query = query_impl policy, _ = explorer._get_exploration_strategy(0, timeout=5) assert callable(policy) - assert option_model.get_next_state_and_num_actions.called + assert not option_model.get_next_state_and_num_actions.called assert 0 not in tool_context.cycle_certified_plans - assert tool_context.last_mental_model_solved is not None - - -def test_pinned_params_run_verbatim_in_the_experiment_search(): - """With agent_explorer_pin_proposed_params the sketch's explicit params are - re-proposed on retry and never replaced, even with info-seeking on and an - ensemble that disagrees everywhere.""" - _reset_config(agent_explorer_info_seeking=True, - agent_explorer_info_n_feasible_target=3, - agent_bilevel_explorer_max_samples_per_step=6, - agent_explorer_pinned_step_retries=2, - agent_bilevel_use_llm_initial_params=True) - goal_state = _make_state({_block0: [0.5, 0.6, 0.0]}) - option_model = MagicMock() - option_model.get_next_state_and_num_actions.return_value = (goal_state, 3) - explorer, tool_context = _make_explorer(option_model, None) - tool_context.atom_disagreement_fn = lambda _s, _atoms: 1.0 - - async def query_impl(_msg, **_kw): - return _assistant_response( - "```\nPick(block0:block)[0.42] -> {Holding(block0:block)}\n" - "Place(block0:block, block1:block)[0.11, 0.22] -> " - "{On(block0:block, block1:block)}\n```") - - explorer._agent_session.query = query_impl - policy, _ = explorer._get_exploration_strategy(0, timeout=5) - assert callable(policy) - called = [ - c.args[1] - for c in option_model.get_next_state_and_num_actions.call_args_list - ] - assert called, "refinement never rolled the sketch out" - for opt in called: - expected = [0.42] if opt.name == "Pick" else [0.11, 0.22] - np.testing.assert_allclose(opt.params, expected) + assert tool_context.last_mental_model_solved is False diff --git a/tests/test_agent_sdk_tools.py b/tests/test_agent_sdk_tools.py index f5b07942c..1112815da 100644 --- a/tests/test_agent_sdk_tools.py +++ b/tests/test_agent_sdk_tools.py @@ -1,14 +1,12 @@ """Tests for agent SDK tool enhancements. Validates: -1. inspect_options with option_name saves source code to sandbox -2. evaluate_option_plan always saves scene images -3. evaluate_option_plan shows "Missing goal atoms" when goal not achieved -4. evaluate_option_plan shows object poses on failure -5. propose_options saves code to sandbox/proposed_code/ -6. format_object_poses helper -7. render_scene_image helper -8. _sync_tool_context sets ctx.env from option model +1. submit_plan always saves scene images +2. submit_plan shows "Missing goal atoms" when goal not achieved +3. submit_plan shows object poses on failure +4. format_object_poses helper +5. render_scene_image helper +6. _sync_tool_context sets ctx.env from option model Usage: python tests/test_agent_sdk_tools.py @@ -21,7 +19,6 @@ import tempfile from typing import Any -import numpy as np import pytest # Bootstrap circular imports @@ -43,8 +40,6 @@ "num_test_tasks": 1, "skill_phase_use_motion_planning": True, "pybullet_ik_validate": False, - "agent_sdk_propose_options": True, - "agent_planner_use_explore_python": True, # Match the experiment configs: without this, an option whose # first action no-ops against residual env state (wrist drift # from earlier tests) is killed as "stuck", making the @@ -86,10 +81,6 @@ def _setup(sandbox_dir: str | None = None) -> tuple[Any, Any]: if hasattr(option_model, '_simulator'): ctx.env = getattr(option_model._simulator, '__self__', None) - # Create sandbox subdirectories - if sandbox_dir: - os.makedirs(os.path.join(sandbox_dir, "proposed_code"), exist_ok=True) - return ctx, env @@ -134,111 +125,6 @@ def ctx() -> Any: # ===== Tests ===== -def test_inspect_options_list_all(ctx: Any) -> None: - """inspect_options with no args lists all options.""" - tools = _make_tools(ctx, ["inspect_options"]) - result = _run(tools["inspect_options"]({})) - text = result["content"][0]["text"] - assert "Current options:" in text - assert "Pick" in text or "Place" in text or "Push" in text - print(" PASS: inspect_options (list all)") - - -def test_inspect_options_detail(ctx: Any) -> None: - """inspect_options with option_name saves source to sandbox.""" - tools = _make_tools(ctx, ["inspect_options"]) - - # Pick an option that exists - opt_names = [o.name for o in ctx.options] - test_name = opt_names[0] - - result = _run(tools["inspect_options"]({"option_name": test_name})) - text = result["content"][0]["text"] - - # Should have the option header - assert f"## {test_name}" in text - # Should have params info - assert "params_dim" in text - - if ctx.sandbox_dir: - # Should point to the saved file - assert f"./proposed_code/{test_name}.py" in text - # File should exist in sandbox - saved_path = os.path.join(ctx.sandbox_dir, "proposed_code", - f"{test_name}.py") - assert os.path.exists(saved_path), \ - f"Expected file at {saved_path}" - # File should have content - with open(saved_path, encoding='utf-8') as f: - content = f.read() - assert len(content) > 0 - print(f" PASS: inspect_options (detail for '{test_name}', " - f"saved to sandbox)") - else: - # Fallback: should inline source code - assert "Source Code" in text - print(f" PASS: inspect_options (detail for '{test_name}', " - f"inlined — no sandbox)") - - -def test_inspect_options_unknown(ctx: Any) -> None: - """inspect_options with unknown option_name returns error.""" - tools = _make_tools(ctx, ["inspect_options"]) - result = _run(tools["inspect_options"]({ - "option_name": "NonExistentOption" - })) - assert result.get("is_error", False) - assert "Unknown option" in result["content"][0]["text"] - print(" PASS: inspect_options (unknown option)") - - -def test_inspect_options_proposed_code(ctx: Any) -> None: - """inspect_options returns path for option with code saved to sandbox.""" - from predicators.agent_sdk.tools.results import _save_option_to_sandbox - - # Save proposal code to sandbox - proposal_code = "# test proposal code\nx = 1" - _save_option_to_sandbox(ctx, "TestOpt", proposal_code) - - # Create a dummy option with that name - from gym.spaces import Box - - from predicators.structs import ParameterizedOption - dummy_opt = ParameterizedOption( - name="TestOpt", - types=[], - params_space=Box(low=np.array([]), high=np.array([])), - policy=lambda s, m, o, p: None, # type: ignore[arg-type, return-value] - initiable=lambda s, m, o, p: True, - terminal=lambda s, m, o, p: True, - ) - ctx.options = ctx.options | {dummy_opt} - - tools = _make_tools(ctx, ["inspect_options"]) - result = _run(tools["inspect_options"]({"option_name": "TestOpt"})) - text = result["content"][0]["text"] - - if ctx.sandbox_dir: - assert "./proposed_code/TestOpt.py" in text - # Verify file content - saved_path = os.path.join(ctx.sandbox_dir, "proposed_code", - "TestOpt.py") - with open(saved_path, encoding='utf-8') as f: - assert "# test proposal code" in f.read() - else: - # No sandbox — source inlined - assert "Source Code" in text - - # Clean up - ctx.options = {o for o in ctx.options if o.name != "TestOpt"} - if ctx.sandbox_dir: - saved_path = os.path.join(ctx.sandbox_dir, "proposed_code", - "TestOpt.py") - if os.path.exists(saved_path): - os.remove(saved_path) - print(" PASS: inspect_options (proposed code in sandbox)") - - def _get_valid_option_plan_step(ctx: Any) -> dict[str, Any] | None: """Find a valid single-step option plan for testing.""" # Find option with fewest type requirements @@ -278,8 +164,8 @@ def _get_valid_option_plan_step(ctx: Any) -> dict[str, Any] | None: def _plan_to_text(plan: Any, ctx: Any) -> str: - """Render structured option-plan steps as the text grammar that - evaluate_option_plan now expects (typed object refs + params in []).""" + """Render structured option-plan steps as the text grammar that submit_plan + now expects (typed object refs + params in []).""" type_of = {o.name: o.type.name for o in ctx.current_task.init} lines = [] for step in plan: @@ -291,19 +177,16 @@ def _plan_to_text(plan: Any, ctx: Any) -> str: def test_option_plan_missing_goal_atoms(ctx: Any) -> None: - """evaluate_option_plan reports missing goal atoms when goal not - achieved.""" - tools = _make_tools(ctx, ["evaluate_option_plan"]) + """submit_plan reports missing goal atoms when goal not achieved.""" + tools = _make_tools(ctx, ["submit_plan"]) step = _get_valid_option_plan_step(ctx) assert step is not None, "No valid option found for testing" plan = [step] - result = _run(tools["evaluate_option_plan"]({ - "plan": - _plan_to_text(plan, ctx), - "include_atoms": - True, + result = _run(tools["submit_plan"]({ + "plan": _plan_to_text(plan, ctx), + "include_atoms": True, })) text = result["content"][0]["text"] @@ -315,43 +198,32 @@ def test_option_plan_missing_goal_atoms(ctx: Any) -> None: # agents). assert ("Missing goal atoms:" in text or "Goal (natural language):" in text) - print(" PASS: evaluate_option_plan (failure diagnostic shown)") + print(" PASS: submit_plan (failure diagnostic shown)") elif "Goal achieved: True" in text: assert "Missing goal atoms:" not in text - print(" PASS: evaluate_option_plan (goal achieved, no missing atoms)") + print(" PASS: submit_plan (goal achieved, no missing atoms)") else: # Plan failed early (grounding error, NOT INITIABLE, etc.) assert ("NOT INITIABLE" in text or "FAILURE REASON:" in text or "EXECUTION ERROR" in text or "Failed to ground" in text) - print(" PASS: evaluate_option_plan (plan failed early, " + print(" PASS: submit_plan (plan failed early, " "goal check not reached)") def test_option_plan_description_submission_split(ctx: Any) -> None: - """With explore_python on, evaluate_option_plan's description routes - exploration to the probe and frames this tool as the submission path; with - it off, the description is unchanged.""" + """submit_plan's description routes exploration to the probe and frames + this tool as the submission path.""" from predicators.agent_sdk.tools import create_mcp_tools - prior = CFG.agent_planner_use_explore_python - try: - pred_utils.update_config({"agent_planner_use_explore_python": True}) - tool_obj = create_mcp_tools(ctx, - tool_names=["evaluate_option_plan"])[0] - desc = getattr(tool_obj, "description", "") - assert "explore_python" in desc and "SUBMIT" in desc - pred_utils.update_config({"agent_planner_use_explore_python": False}) - tool_obj = create_mcp_tools(ctx, - tool_names=["evaluate_option_plan"])[0] - assert "explore_python" not in getattr(tool_obj, "description", "") - finally: - pred_utils.update_config({"agent_planner_use_explore_python": prior}) - print(" PASS: evaluate_option_plan (submission-split description)") + tool_obj = create_mcp_tools(ctx, tool_names=["submit_plan"])[0] + desc = getattr(tool_obj, "description", "") + assert "run_python" in desc and "SUBMIT" in desc + print(" PASS: submit_plan (submission-split description)") -def test_explore_python_render_annotations(ctx: Any) -> None: +def test_run_python_render_annotations(ctx: Any) -> None: """sim.render(annotations=...) overlays temporary geometry for one render (bodies removed after) and surfaces bad annotations as loud errors.""" - tools = _make_tools(ctx, ["explore_python"]) + tools = _make_tools(ctx, ["run_python"]) with tempfile.TemporaryDirectory() as img_dir: prior_dir = ctx.image_save_dir ctx.image_save_dir = img_dir @@ -364,44 +236,44 @@ def test_explore_python_render_annotations(ctx: Any) -> None: ]) print("saved", path is not None) """ - result = _run(tools["explore_python"]({"code": code})) + result = _run(tools["run_python"]({"code": code})) text = result["content"][0]["text"] assert "saved True" in text assert len(os.listdir(img_dir)) == 1 # A malformed annotation errors loudly (after cleanup). bad = 'sim.render("bad", annotations=[{"type": "line"}])' - result = _run(tools["explore_python"]({"code": bad})) + result = _run(tools["run_python"]({"code": bad})) assert "Error" in result["content"][0]["text"] finally: ctx.image_save_dir = prior_dir - print(" PASS: explore_python (annotated render)") + print(" PASS: run_python (annotated render)") -def test_explore_python_unknown_mod_object(ctx: Any) -> None: +def test_run_python_unknown_mod_object(ctx: Any) -> None: """A reset() modification naming an unknown object is a loud error.""" - tools = _make_tools(ctx, ["explore_python"]) - result = _run(tools["explore_python"]({ + tools = _make_tools(ctx, ["run_python"]) + result = _run(tools["run_python"]({ "code": 'sim.reset(mods={"no_such_object": {"x": 0.5}})' })) assert "Unknown object 'no_such_object'" in result["content"][0]["text"] - print(" PASS: explore_python (unknown mod object error)") + print(" PASS: run_python (unknown mod object error)") -def test_explore_python_exec_and_persistence(ctx: Any) -> None: - """The solve-phase explore_python executes code and keeps its namespace.""" - tools = _make_tools(ctx, ["explore_python"]) - result = _run(tools["explore_python"]({"code": "x = 21\nprint(x * 2)"})) +def test_run_python_exec_and_persistence(ctx: Any) -> None: + """The solve-phase run_python executes code and keeps its namespace.""" + tools = _make_tools(ctx, ["run_python"]) + result = _run(tools["run_python"]({"code": "x = 21\nprint(x * 2)"})) assert result["content"][0]["text"].strip() == "42" - result = _run(tools["explore_python"]({"code": "print(x + 1)"})) + result = _run(tools["run_python"]({"code": "print(x + 1)"})) assert result["content"][0]["text"].strip() == "22" - print(" PASS: explore_python (exec + persistent namespace)") + print(" PASS: run_python (exec + persistent namespace)") -def test_explore_python_probe_sim(ctx: Any) -> None: +def test_run_python_probe_sim(ctx: Any) -> None: """BeliefProbe: reset with mods, full-precision state, run from the modified state, snapshot/restore - and nothing is ever captured.""" - tools = _make_tools(ctx, ["explore_python"]) + tools = _make_tools(ctx, ["run_python"]) domino = next(o for o in ctx.current_task.init if o.type.name == "domino") robot = next(o for o in ctx.current_task.init if o.type.name == "robot") ctx.capture_goal_reaching_plans = True @@ -423,7 +295,7 @@ def test_explore_python_probe_sim(ctx: Any) -> None: print("restx", sim.state("{domino.name}")["x"]) print("natoms", len(sim.atoms())) """ - result = _run(tools["explore_python"]({"code": code})) + result = _run(tools["run_python"]({"code": code})) saved = [f for f in os.listdir(tmpdir) if f.endswith(".png")] finally: ctx.capture_goal_reaching_plans = False @@ -433,7 +305,7 @@ def test_explore_python_probe_sim(ctx: Any) -> None: assert "steps 1" in text assert "restx 0.95" in text assert "natoms" in text - # sim.run saves the same per-step audit images evaluate_option_plan + # sim.run saves the same per-step audit images submit_plan # does, and reports their paths on each step; render=False (for # tight sweep loops) skips the render entirely. assert "quietimg None" in text @@ -445,14 +317,13 @@ def test_explore_python_probe_sim(ctx: Any) -> None: print(" NOTE: rendering not available, image save not checked") # The probe carries no scoring surface: nothing it ran was captured. assert ctx.solved_plan is None - print( - " PASS: explore_python (BeliefProbe reset/run/snapshot, no capture)") + print(" PASS: run_python (BeliefProbe reset/run/snapshot, no capture)") -def test_explore_python_probe_refine(ctx: Any) -> None: +def test_run_python_probe_refine(ctx: Any) -> None: """BeliefProbe.refine searches params from the current state, reports per- step samples and a refined plan line, and captures nothing.""" - tools = _make_tools(ctx, ["explore_python"]) + tools = _make_tools(ctx, ["run_python"]) domino = next(o for o in ctx.current_task.init if o.type.name == "domino") robot = next(o for o in ctx.current_task.init if o.type.name == "robot") ctx.capture_goal_reaching_plans = True @@ -467,7 +338,7 @@ def test_explore_python_probe_refine(ctx: Any) -> None: print("samples", res.total_samples, res.step_samples) print("line", res.plan_lines[0]) """ - result = _run(tools["explore_python"]({"code": code})) + result = _run(tools["run_python"]({"code": code})) finally: ctx.capture_goal_reaching_plans = False text = result["content"][0]["text"] @@ -475,13 +346,13 @@ def test_explore_python_probe_refine(ctx: Any) -> None: assert "samples" in text assert "line Pick(" in text and "Holding(" in text assert ctx.solved_plan is None - print(" PASS: explore_python (BeliefProbe.refine, no capture)") + print(" PASS: run_python (BeliefProbe.refine, no capture)") -def test_explore_python_probe_refine_verdict_line(ctx: Any) -> None: +def test_run_python_probe_refine_verdict_line(ctx: Any) -> None: """A refine SUCCESS carries a Verdict line saying exactly what it certifies (bare SUCCESS used to be read as goal-reached).""" - tools = _make_tools(ctx, ["explore_python"]) + tools = _make_tools(ctx, ["run_python"]) domino = next(o for o in ctx.current_task.init if o.type.name == "domino") robot = next(o for o in ctx.current_task.init if o.type.name == "robot") code = f""" @@ -492,18 +363,18 @@ def test_explore_python_probe_refine_verdict_line(ctx: Any) -> None: timeout=45) print(res) """ - result = _run(tools["explore_python"]({"code": code})) + result = _run(tools["run_python"]({"code": code})) text = result["content"][0]["text"] assert "Verdict: executed" in text assert "require_goal=True" in text - print(" PASS: explore_python (refine verdict line)") + print(" PASS: run_python (refine verdict line)") -def test_explore_python_probe_strlike_and_region_note(ctx: Any) -> None: +def test_run_python_probe_strlike_and_region_note(ctx: Any) -> None: """ProbeResult supports string slicing/containment, and a `~` region annotation is IGNORED with a NOTE when ground samplers are off (previously a hard error that cost turns of syntax guessing).""" - tools = _make_tools(ctx, ["explore_python"]) + tools = _make_tools(ctx, ["run_python"]) robot = next(o for o in ctx.current_task.init if o.type.name == "robot") assert not CFG.agent_bilevel_ground_samplers code = f""" @@ -513,19 +384,19 @@ def test_explore_python_probe_strlike_and_region_note(ctx: Any) -> None: print("slice_ok", len(res[-40:]) > 0) print(res) """ - result = _run(tools["explore_python"]({"code": code})) + result = _run(tools["run_python"]({"code": code})) text = result["content"][0]["text"] assert "contains True" in text assert "slice_ok True" in text assert "region annotation was IGNORED" in text assert "uniform" in text - print(" PASS: explore_python (str-like result + region note)") + print(" PASS: run_python (str-like result + region note)") -def test_explore_python_probe_trials(ctx: Any) -> None: +def test_run_python_probe_trials(ctx: Any) -> None: """sim.run(plan, trials=N) reports per-trial outcomes and a success count without advancing the current state.""" - tools = _make_tools(ctx, ["explore_python"]) + tools = _make_tools(ctx, ["run_python"]) domino = next(o for o in ctx.current_task.init if o.type.name == "domino") robot = next(o for o in ctx.current_task.init if o.type.name == "robot") code = f""" @@ -536,7 +407,7 @@ def test_explore_python_probe_trials(ctx: Any) -> None: print("n_trials", len(res.trials)) print("kept", abs(sim.state("{domino.name}")["x"] - before) < 1e-9) """ - result = _run(tools["explore_python"]({"code": code})) + result = _run(tools["run_python"]({"code": code})) text = result["content"][0]["text"] assert "Trials:" in text assert "n_trials 2" in text @@ -544,17 +415,17 @@ def test_explore_python_probe_trials(ctx: Any) -> None: # say the trials shared the session env (correlated). assert "shared session env" in text assert "kept True" in text - print(" PASS: explore_python (trials=N)") + print(" PASS: run_python (trials=N)") -def test_explore_python_refine_require_solved_guards(ctx: Any) -> None: +def test_run_python_refine_require_solved_guards(ctx: Any) -> None: """require_solved refuses to run from a modified start, and from a task. with no evaluator - both before any search. (The gate's accept/reject semantics are covered deterministically in test_bilevel_sketch_samplers.py with fake option models.) """ - tools = _make_tools(ctx, ["explore_python"]) + tools = _make_tools(ctx, ["run_python"]) domino = next(o for o in ctx.current_task.init if o.type.name == "domino") robot = next(o for o in ctx.current_task.init if o.type.name == "robot") plan_line = (f"Pick({robot.name}:robot, {domino.name}:domino)[0.06] " @@ -564,7 +435,7 @@ def test_explore_python_refine_require_solved_guards(ctx: Any) -> None: # Modified start: refused outright. code = (f'sim.reset(mods={{"{domino.name}": {{"x": 0.9}}}})\n' f'sim.refine("{plan_line}", require_solved=True)') - result = _run(tools["explore_python"]({"code": code})) + result = _run(tools["run_python"]({"code": code})) assert "unmodified initial state" in result["content"][0]["text"] # Pristine start but the task defines no evaluator: refused. @@ -573,32 +444,32 @@ def test_explore_python_refine_require_solved_guards(ctx: Any) -> None: try: ctx.current_task = dataclasses.replace(saved_task, evaluator=None) code = f'sim.reset()\nsim.refine("{plan_line}", require_solved=True)' - result = _run(tools["explore_python"]({"code": code})) + result = _run(tools["run_python"]({"code": code})) assert "defines no task evaluator" in result["content"][0]["text"] finally: ctx.current_task = saved_task - print(" PASS: explore_python (require_solved guards)") + print(" PASS: run_python (require_solved guards)") -def test_explore_python_run_solved_guards(ctx: Any) -> None: +def test_run_python_run_solved_guards(ctx: Any) -> None: """sim.run(solved=True) refuses single runs, modified starts, and tasks with no evaluator; contacts=True refuses trials mode.""" - tools = _make_tools(ctx, ["explore_python"]) + tools = _make_tools(ctx, ["run_python"]) domino = next(o for o in ctx.current_task.init if o.type.name == "domino") robot = next(o for o in ctx.current_task.init if o.type.name == "robot") wait_line = f"Wait({robot.name}:robot)[]" code = f'sim.reset()\nsim.run("{wait_line}", solved=True)' - result = _run(tools["explore_python"]({"code": code})) + result = _run(tools["run_python"]({"code": code})) assert "needs trials >= 2" in result["content"][0]["text"] code = f'sim.reset()\nsim.run("{wait_line}", trials=2, contacts=True)' - result = _run(tools["explore_python"]({"code": code})) + result = _run(tools["run_python"]({"code": code})) assert "single-run mode" in result["content"][0]["text"] code = (f'sim.reset(mods={{"{domino.name}": {{"x": 0.9}}}})\n' f'sim.run("{wait_line}", trials=2, solved=True)') - result = _run(tools["explore_python"]({"code": code})) + result = _run(tools["run_python"]({"code": code})) assert "unmodified initial state" in result["content"][0]["text"] import dataclasses @@ -606,17 +477,17 @@ def test_explore_python_run_solved_guards(ctx: Any) -> None: try: ctx.current_task = dataclasses.replace(saved_task, evaluator=None) code = f'sim.reset()\nsim.run("{wait_line}", trials=2, solved=True)' - result = _run(tools["explore_python"]({"code": code})) + result = _run(tools["run_python"]({"code": code})) assert "defines no task evaluator" in result["content"][0]["text"] finally: ctx.current_task = saved_task - print(" PASS: explore_python (run solved/contacts guards)") + print(" PASS: run_python (run solved/contacts guards)") -def test_explore_python_run_solved_trials(ctx: Any) -> None: +def test_run_python_run_solved_trials(ctx: Any) -> None: """sim.run(trials=N, solved=True) reports a per-trial task-evaluator verdict and a solved count in the headline.""" - tools = _make_tools(ctx, ["explore_python"]) + tools = _make_tools(ctx, ["run_python"]) robot = next(o for o in ctx.current_task.init if o.type.name == "robot") code = f""" sim.reset() @@ -624,19 +495,19 @@ def test_explore_python_run_solved_trials(ctx: Any) -> None: print(res) print("verdicts", [t["solved"] for t in res.trials]) """ - result = _run(tools["explore_python"]({"code": code})) + result = _run(tools["run_python"]({"code": code})) text = result["content"][0]["text"] assert "scored solved=True by the task evaluator" in text assert "evaluator: solved=" in text # A Wait-only plan cannot reach the goal, so no trial scores a solve. assert "verdicts [False, False]" in text - print(" PASS: explore_python (trials solved verdicts)") + print(" PASS: run_python (trials solved verdicts)") -def test_explore_python_run_contacts(ctx: Any) -> None: +def test_run_python_run_contacts(ctx: Any) -> None: """sim.run(contacts=True) reports per-step contact-pair spans; a Pick must show a robot-link contact with the grasped domino.""" - tools = _make_tools(ctx, ["explore_python"]) + tools = _make_tools(ctx, ["run_python"]) domino = next(o for o in ctx.current_task.init if o.type.name == "domino") robot = next(o for o in ctx.current_task.init if o.type.name == "robot") code = f""" @@ -645,19 +516,19 @@ def test_explore_python_run_contacts(ctx: Any) -> None: render=False, contacts=True) print(res) """ - result = _run(tools["explore_python"]({"code": code})) + result = _run(tools["run_python"]({"code": code})) text = result["content"][0]["text"] assert "contact recording unavailable" not in text assert "Contacts:" in text # The grasp squeeze puts a robot link in contact with the domino. assert "robot:" in text assert domino.name in text.split("Contacts:", 1)[1] - print(" PASS: explore_python (contact recording)") + print(" PASS: run_python (contact recording)") def test_option_plan_not_initiable_shows_poses(ctx: Any) -> None: - """evaluate_option_plan shows object poses when option is NOT INITIABLE.""" - tools = _make_tools(ctx, ["evaluate_option_plan"]) + """submit_plan shows object poses when option is NOT INITIABLE.""" + tools = _make_tools(ctx, ["submit_plan"]) # Find Place option and try it without Pick first place_opt = None @@ -667,7 +538,7 @@ def test_option_plan_not_initiable_shows_poses(ctx: Any) -> None: break if place_opt is None: - print(" SKIP: evaluate_option_plan (no Place option)") + print(" SKIP: submit_plan (no Place option)") return # Build object names from types @@ -689,37 +560,34 @@ def test_option_plan_not_initiable_shows_poses(ctx: Any) -> None: "params": params, }] - result = _run(tools["evaluate_option_plan"]({ - "plan": - _plan_to_text(plan, ctx), + result = _run(tools["submit_plan"]({ + "plan": _plan_to_text(plan, ctx), })) text = result["content"][0]["text"] if "NOT INITIABLE" in text: assert "Object poses at failure:" in text - print(" PASS: evaluate_option_plan (NOT INITIABLE shows poses)") + print(" PASS: submit_plan (NOT INITIABLE shows poses)") elif "Failed to ground" in text: - print(" SKIP: evaluate_option_plan (Place could not be grounded)") + print(" SKIP: submit_plan (Place could not be grounded)") else: - print(" SKIP: evaluate_option_plan (Place was initiable, " + print(" SKIP: submit_plan (Place was initiable, " "can't test NOT INITIABLE path)") def test_option_plan_saves_images(ctx: Any) -> None: - """evaluate_option_plan always saves scene images (never returns - inline).""" + """submit_plan always saves scene images (never returns inline).""" with tempfile.TemporaryDirectory() as tmpdir: ctx.image_save_dir = tmpdir - tools = _make_tools(ctx, ["evaluate_option_plan"]) + tools = _make_tools(ctx, ["submit_plan"]) step = _get_valid_option_plan_step(ctx) assert step is not None, "No valid option found for testing" plan = [step] - result = _run(tools["evaluate_option_plan"]({ - "plan": - _plan_to_text(plan, ctx), + result = _run(tools["submit_plan"]({ + "plan": _plan_to_text(plan, ctx), })) content = result["content"] @@ -730,25 +598,23 @@ def test_option_plan_saves_images(ctx: Any) -> None: # Check files were saved if env rendering works saved = [f for f in os.listdir(tmpdir) if f.endswith(".png")] if saved: - print(f" PASS: evaluate_option_plan ({len(saved)} images saved)") + print(f" PASS: submit_plan ({len(saved)} images saved)") else: - print(" SKIP: evaluate_option_plan (rendering not available)") + print(" SKIP: submit_plan (rendering not available)") ctx.image_save_dir = None def test_option_plan_failure_shows_poses(ctx: Any) -> None: - """evaluate_option_plan shows object poses when option returns 0 - actions.""" - tools = _make_tools(ctx, ["evaluate_option_plan"]) + """submit_plan shows object poses when option returns 0 actions.""" + tools = _make_tools(ctx, ["submit_plan"]) step = _get_valid_option_plan_step(ctx) assert step is not None, "No valid option found for testing" plan = [step] - result = _run(tools["evaluate_option_plan"]({ - "plan": - _plan_to_text(plan, ctx), + result = _run(tools["submit_plan"]({ + "plan": _plan_to_text(plan, ctx), })) text = result["content"][0]["text"] @@ -758,12 +624,12 @@ def test_option_plan_failure_shows_poses(ctx: Any) -> None: or "Testing option plan" in text) if "FAILURE REASON:" in text: assert "Object poses at failure:" in text - print(" PASS: evaluate_option_plan (failure shows poses)") + print(" PASS: submit_plan (failure shows poses)") elif "NOT INITIABLE" in text: assert "Object poses at failure:" in text - print(" PASS: evaluate_option_plan (NOT INITIABLE shows poses)") + print(" PASS: submit_plan (NOT INITIABLE shows poses)") else: - print(" PASS: evaluate_option_plan (no failures in output)") + print(" PASS: submit_plan (no failures in output)") def testformat_object_poses(ctx: Any) -> None: @@ -824,59 +690,6 @@ def test_render_scene_no_env(ctx: Any) -> None: print(" PASS: render_scene_image (no env → None)") -def test_propose_options_saves_to_sandbox(ctx: Any) -> None: - """propose_options saves proposal code to sandbox/proposed_code/.""" - tools = _make_tools(ctx, ["propose_options"]) - - # Get type names for a valid proposal - type_names = {t.name: t for t in ctx.types} - robot_type_name = "robot" if "robot" in type_names else list(type_names)[0] - - code = f"""\ -from gym.spaces import Box -import numpy as np - -proposed_options = [ - ParameterizedOption( - name="TestProposed", - types=[{robot_type_name}_type], - params_space=Box(low=np.array([0.0]), high=np.array([1.0])), - policy=lambda s, m, o, p: Action(np.zeros(s.get(o[0], "x").shape if hasattr(s.get(o[0], "x"), "shape") else (1,))), - initiable=lambda s, m, o, p: True, - terminal=lambda s, m, o, p: True, - ) -] -""" - - result = _run(tools["propose_options"]({ - "code": - code, - "description": - "Test option for unit test", - })) - text = result["content"][0]["text"] - - if "Successfully proposed" in text: - if ctx.sandbox_dir: - saved_path = os.path.join(ctx.sandbox_dir, "proposed_code", - "TestProposed.py") - assert os.path.exists(saved_path), \ - f"Expected file at {saved_path}" - with open(saved_path, encoding='utf-8') as f: - content = f.read() - assert "proposed_options" in content - print(" PASS: propose_options (code saved to sandbox)") - os.remove(saved_path) - else: - print(" PASS: propose_options (no sandbox, code not saved)") - - # Clean up - ctx.options = {o for o in ctx.options if o.name != "TestProposed"} - else: - # Code execution might fail due to env-specific types - print(f" SKIP: propose_options (code failed: {text[:100]})") - - def test_sync_tool_context_sets_env() -> None: """_sync_tool_context extracts env from option model.""" pred_utils.reset_config(_CFG_OVERRIDES) @@ -919,32 +732,21 @@ def main() -> None: print("=== Tool Enhancement Tests ===\n") - # inspect_options tests - print("1. inspect_options tests:") - test_inspect_options_list_all(ctx) - test_inspect_options_detail(ctx) - test_inspect_options_unknown(ctx) - test_inspect_options_proposed_code(ctx) - - # evaluate_option_plan tests - print("\n2. evaluate_option_plan tests:") + # submit_plan tests + print("1. submit_plan tests:") test_option_plan_missing_goal_atoms(ctx) test_option_plan_not_initiable_shows_poses(ctx) test_option_plan_saves_images(ctx) test_option_plan_failure_shows_poses(ctx) # Helper function tests - print("\n3. Helper function tests:") + print("\n2. Helper function tests:") testformat_object_poses(ctx) testrender_scene_image(ctx) test_render_scene_no_env(ctx) - # propose_options test - print("\n4. propose_options tests:") - test_propose_options_saves_to_sandbox(ctx) - # _sync_tool_context test (creates fresh env) - print("\n5. Context sync tests:") + print("\n3. Context sync tests:") test_sync_tool_context_sets_env() print("\n=== All tests passed! ===") diff --git a/tests/test_docker_option_plan.py b/tests/test_docker_option_plan.py index d7b5fa2b9..0f1f3d6e4 100644 --- a/tests/test_docker_option_plan.py +++ b/tests/test_docker_option_plan.py @@ -1,4 +1,4 @@ -"""Test that evaluate_option_plan produces correct results. +"""Test that submit_plan produces correct results. Validates that multi-step option plans (Pick→Place→Pick→Place→Push) produce non-zero actions at every step, both in-process and in a subprocess that @@ -119,7 +119,7 @@ def _run_option_plan(ctx: Any, plan = OPTION_PLAN task = ctx.current_task - all_options = ctx.options | ctx.iteration_proposals.proposed_options + all_options = ctx.options opt_map = {o.name: o for o in all_options} state = task.init diff --git a/tests/test_structs.py b/tests/test_structs.py index ac7492a0c..c1a213d5f 100644 --- a/tests/test_structs.py +++ b/tests/test_structs.py @@ -577,7 +577,7 @@ def test_option_ground_clamps_float_precision_boundary(): Regression: an agent-parsed yaw of pi stored as float32 exceeds the float64 pi upper bound (float32(pi) > pi) and crashed - refine_plan_sketch with a raw ValueError (run_20260707_112310). + sketch refinement with a raw ValueError (run_20260707_112310). """ params_space = Box(np.array([-np.pi]), np.array([np.pi]), dtype=np.float64)