Critic agent for trajectory evaluation

[20001LastOrder]

The multi-agent pub/sub design already anticipates critic agents subscribing to results via SharedMemory. Evaluates whole trajectories, not just final outputs. See how this can be used as a feedback to guide the agent execution.

• Critic agent subscribing to result events
• Trajectory-level rubric: action efficiency, loops, wasted iterations

[cybr-wisp]

Hey everyone,

I'm starting work on this and wanted to get feedback on how to pass trajectory data to the critic agent before writing code. Right now, subscribing to result events via SharedMemory only returns the final output string. Scoring action efficiency, loops, and wasted iterations requires the full sequence (action_start and action_finish) from the producing agent's belief.

Three options:
Option A (SharedMemory Log): Emit granular action events into SharedMemory for the critic to query.
Drawback: Floods SharedMemory and requires new sender-filtering methods.

Option B (Expand Result Payload): Pack internal_events into the standard result event content.
Drawback: Breaking change for subscribers expecting a plain string.

Option C (New TrajectoryCompleteEvent): Add a dedicated event carrying both the result and internal_events.
Drawback: None identified; producing agents emit both events for full backward compatibility.

I'm leaning toward Option C. Does this direction make sense, or is there an edge case where A or B works better?

[20001LastOrder]

I think C for the new event makes sense. Another thing I'm thinking is that rather than critic the entire trajectory, an additional useful thing is for the agent to get feedback on the completion of the task so far to better select subsequent actions, so maybe there are different events for either critic during the execution or after an entire execution is completed.

Whatever the direction to go, I want the implementation to be agnostic to the underlying agent to better support #515. There should be a mechanism for the agent (not only the Sherpa agent, but also Claude, Codex, etc) to access the trajectory data and decide to critique. Maybe it should be exposed as some kind of function