Skip to content

feat: bump Opus experiments to Claude Opus 5 - #136

Merged
Rodriguespn merged 1 commit into
mainfrom
pedrorodrigues/ai-968-replace-opus-48-with-opus-5-in-evals
Jul 29, 2026
Merged

feat: bump Opus experiments to Claude Opus 5#136
Rodriguespn merged 1 commit into
mainfrom
pedrorodrigues/ai-968-replace-opus-48-with-opus-5-in-evals

Conversation

@Rodriguespn

@Rodriguespn Rodriguespn commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Swaps the two Opus 4.8 experiments to Opus 5 and updates the matching README reference, mirroring the earlier Sonnet 4.6 → Sonnet 5 bump.

According to Claude models overview this is a stable ID:

Every Claude model ID is a pinned snapshot. Models with a date in the ID (for example, 20250929) are fixed to that specific release. Starting with the Claude 4.6 generation, model IDs use a dateless format that is also a pinned snapshot, not an evergreen pointer

Pricing is unchanged from Opus 4.8 ($5/$25 per MTok), so this stays within the same GitHub Actions budget.

Claude Code compatibility

Checked the Claude Code changelog between our pinned CLI (2.1.191) and latest (2.1.220): no breaking changes to the --output-format stream-json transcript structure that packages/core/src/agents/claude-code/parser.ts depends on — only additive changes:

Claude Code added Opus 5 as its default Opus alias in 2.1.219, but the pinned 2.1.191 passes the explicit claude-opus-5 ID through to the API fine (same as Sonnet 5 did in #73, which predated CLI-side Sonnet 5 support), so no CLI bump is needed.

Opus 5's only API breaking change vs 4.8 (what's new: thinking: disabled with effort xhigh/max now 400s) doesn't affect these experiments — they run reasoningEffort: 'high' with default thinking.

Note: high is also Claude Code's default effort for Opus 5 (model config docs.

Testing

Sanity-checked claude-code-opus-5 against resolve-dataapi-001-empty-results: 7/7 checks passed (1 attempt, 230s, 43 tool calls, stoppedReason: stop, transcript parsed cleanly with skill-load detection working).

Closes AI-968

Swaps the two Opus 4.8 experiments to Claude Opus 5 (claude-opus-5) and
updates the matching README reference. Pinned Claude Code CLI (2.1.191)
passes the explicit model ID through unchanged; sanity run passed 7/7.

Closes AI-968

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Rodriguespn
Rodriguespn requested a review from a team July 28, 2026 15:20
@linear-code

linear-code Bot commented Jul 28, 2026

Copy link
Copy Markdown
AI-968 Replace Opus 4.8 with Opus 5 in evals

Update the evals to use Opus 5 instead of Opus 4.8 ahead of launch. The thread discussed either replacing 4.8 or running both side by side, and the decision was to replace 4.8 with 5 because of the GitHub Actions jobs budget, similar to the earlier Sonnet 4.6 → 5 update.

Review in Linear

@vercel

vercel Bot commented Jul 28, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
evals Ignored Ignored Jul 28, 2026 3:20pm

Request Review

@mattrossman

Copy link
Copy Markdown
Collaborator

Tested the updated experiments locally and they seemed to work fine!

@Rodriguespn
Rodriguespn merged commit 879e0a1 into main Jul 29, 2026
6 checks passed
@raulb
raulb deleted the pedrorodrigues/ai-968-replace-opus-48-with-opus-5-in-evals branch July 29, 2026 11:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants