Skip to content

Honor requested shell timeouts and run evals in parallel - #498

Merged
TheGreatAxios merged 4 commits into
mainfrom
cl-6652-drop-the-10-minute-shell-timeout-ceiling-and-run-capability
Aug 22, 2026
Merged

Honor requested shell timeouts and run evals in parallel#498
TheGreatAxios merged 4 commits into
mainfrom
cl-6652-drop-the-10-minute-shell-timeout-ceiling-and-run-capability

Conversation

@TheGreatAxios

Copy link
Copy Markdown
Collaborator

Summary

  • A requested run_shell timeout is no longer clamped to 10 minutes. Omit still defaults to 15 seconds. Settings can still set shell.maxTimeoutMs.
  • Capability evals accept --concurrency <n> (and CORBITS_EVAL_CONCURRENCY) so independent case×variant×repeat cells run in parallel. Default remains 1.

Verification

  • bun test src/plugins/shell-guard-plugin.test.ts scripts/eval-capability.test.ts passes
  • bun run typecheck passes

Fixes CL-6652

@linear-code

linear-code Bot commented Aug 22, 2026

Copy link
Copy Markdown

CL-6652

@TheGreatAxios
TheGreatAxios force-pushed the cl-6652-drop-the-10-minute-shell-timeout-ceiling-and-run-capability branch from 9f516a8 to e40824c Compare August 22, 2026 17:49
@TheGreatAxios
TheGreatAxios merged commit 64542d7 into main Aug 22, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant