feat(claude-code): capture subagent text and thinking in transcripts - #137
Draft
Rodriguespn wants to merge 2 commits into
Draft
feat(claude-code): capture subagent text and thinking in transcripts#137Rodriguespn wants to merge 2 commits into
Rodriguespn wants to merge 2 commits into
Conversation
Bump the pinned Claude Code CLI to 2.1.220 and run with --forward-subagent-text and --thinking-display summarized, so subagent text/thinking blocks land in the stream-json transcript (tagged with parent_tool_use_id) and thinking carries summarized content instead of the empty blocks Opus 5-era models stream by default. The Claude Code parser tags forwarded lines with an agnostic subagent ref; the shared parser layer stays agent-agnostic. Subagent parts are captured in the adapted transcript for visibility but excluded from the main-thread surface (agentReport, steps, toolCalls) and from serializeTranscript output unless opted in, so judge inputs are unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
AI-979 Capture Claude Code subagent text and thinking in eval transcripts
Turn on |
Rodriguespn
added a commit
that referenced
this pull request
Jul 29, 2026
Swaps the two Opus 4.8 experiments to [Opus 5](https://www.anthropic.com/news/claude-opus-5) and updates the matching README reference, mirroring the earlier [Sonnet 4.6 → Sonnet 5 bump](#73). According to [Claude models overview](https://platform.claude.com/docs/en/about-claude/models/overview) this is a stable ID: > Every Claude model ID is a pinned snapshot. Models with a date in the ID (for example, 20250929) are fixed to that specific release. Starting with the Claude 4.6 generation, model IDs use a dateless format that is also a pinned snapshot, not an evergreen pointer Pricing is unchanged from Opus 4.8 ($5/$25 per MTok), so this stays within the same GitHub Actions budget. ## Claude Code compatibility Checked the [Claude Code changelog](https://github.com/anthropics/claude-code/blob/main/CHANGELOG.md) between our pinned CLI (`2.1.191`) and latest (`2.1.220`): no breaking changes to the `--output-format stream-json` transcript structure that `packages/core/src/agents/claude-code/parser.ts` depends on — only additive changes: - `mcp_server_errors` on the init event in 2.1.219 - opt-in `--forward-subagent-text` in 2.1.212. #137 adds support to capture subagents steps from the transcript Claude Code added Opus 5 as its default Opus alias in 2.1.219, but the pinned 2.1.191 passes the explicit `claude-opus-5` ID through to the API fine (same as Sonnet 5 did in #73, which predated CLI-side Sonnet 5 support), so no CLI bump is needed. Opus 5's only API breaking change vs 4.8 ([what's new](https://platform.claude.com/docs/en/about-claude/models/whats-new-opus-5): `thinking: disabled` with effort `xhigh`/`max` now 400s) doesn't affect these experiments — they run `reasoningEffort: 'high'` with default thinking. Note: `high` is also Claude Code's default effort for Opus 5 ([model config docs](https://code.claude.com/docs/en/model-config#adjust-effort-level). ## Testing Sanity-checked `claude-code-opus-5` against `resolve-dataapi-001-empty-results`: **7/7 checks passed** (1 attempt, 230s, 43 tool calls, `stoppedReason: stop`, transcript parsed cleanly with skill-load detection working). Closes AI-968 Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Captures delegated-subagent activity — text and thinking — in Claude Code eval transcripts, for full visibility into intermediate steps.
Bumps the pinned CLI
2.1.191→2.1.220and adds two flags to the runner (both visibility-only — they change what the transcript captures, not how the agent behaves):--forward-subagent-text(needs CLI ≥ 2.1.212): subagent text/thinking arrive as normal stream-json lines tagged withparent_tool_use_id,subagent_type, andtask_description.--thinking-display summarized: without it, Opus 5-era models stream thinking blocks with empty text; with it, main-thread and subagent thinking carry real summarized content.Closes AI-979