Skip to content

feat: add HooksManager lifecycle hook system to callModel#7

Open
mattapperson wants to merge 3 commits into
mainfrom
feat/hooks-manager
Open

feat: add HooksManager lifecycle hook system to callModel#7
mattapperson wants to merge 3 commits into
mainfrom
feat/hooks-manager

Conversation

@mattapperson

@mattapperson mattapperson commented Apr 3, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Adds an extensible hook system for callModel inspired by the Claude Agent SDK hooks pattern
  • Two usage modes: inline config (plain object for quick setup) and HooksManager class (custom hooks, dynamic registration, programmatic emit)
  • 8 built-in hooks: PreToolUse, PostToolUse, PostToolUseFailure, UserPromptSubmit, Stop, PermissionRequest, SessionStart, SessionEnd
  • Features: tool matchers (string/RegExp/function), filter predicates, mutation piping, short-circuit on block/reject, async fire-and-forget handlers, configurable error handling
  • Custom hook definitions via Zod schema pairs with full TypeScript autocomplete
  • Additive API — existing onTurnStart, onTurnEnd, requireApproval remain unchanged

Re-creation of OpenRouterTeam/openrouter-web#16735 on the standalone agent SDK repo.

Test plan

  • 35 new unit tests across 4 test files (matchers, emit engine, manager class, resolver)
  • All 186 unit tests pass
  • Build (pnpm build) passes cleanly
  • Lint (pnpm lint) passes cleanly
  • E2E testing with real API calls (PreToolUse block, mutation piping, Stop forceResume)

Open in Devin Review

@mattapperson mattapperson left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reviewed, no issues found

@robert-j-y robert-j-y left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1. SessionEnd fires on exactly one code path: a completed run that had tools AND made at least one tool call AND didn't throw.

packages/agent/src/lib/model-result.ts:1985-2184 — the execute IIFE has no try/catch/finally (grepped: only two try blocks in the file, at line 1587 and 2758, both unrelated). The only SessionEnd emit is at line 2178, at the very bottom, after four unconditional early returns:

  • line 1990: resuming from approval, still pending
  • line 2007: !this.options.tools?.length || !hasToolCallsa plain chat response with no tool calls never emits SessionEnd
  • line 2015: paused for approval
  • line 2021: no executable tool calls

Plus any throw in the loop (tool exec, hook emit, state save, followup request) rejects the promise and skips SessionEnd + hooksManager.drain(). SessionEndPayload.reason at hooks-types.ts:177 declares 'user' | 'error' | 'max_turns' | 'complete'; only 'complete' is ever emitted, and then only on the happy-tool path.

Fix: wrap the IIFE body in try { ... } finally { if (this.hooksManager) { await this.hooksManager.emit('SessionEnd', { sessionId, reason }); await this.hooksManager.drain(); } }, and pick reason based on exit path ('complete' on normal end, 'error' in the catch, 'max_turns' when the loop exits via shouldStopExecution without a Stop override).

2. Stop payload reason is hardcoded to 'end_turn' — verified as the only emit site.

model-result.ts:2043-2046:

const stopResult = await this.hooksManager.emit('Stop', {
  reason: 'end_turn' as const,
  sessionId: this.currentState?.id ?? '',
});

Grepped the whole file: one Stop emit site. StopPayload at hooks-types.ts:154 declares 'end_turn' | 'max_tokens' | 'stop_sequence' — the other two are unreachable. Also: shouldStopExecution() fires off arbitrary stopWhen conditions (default stepCountIs(DEFAULT_MAX_STEPS)), which semantically is closer to 'max_turns' than 'end_turn'. The three enum values are Anthropic-style stop reasons and don't cleanly map to this engine's stop causes.

Fix: either derive reason from the actual cause (stop-condition name vs. response finish_reason) and expand the schema to cover step-count/tool-absence cases, or drop the two unreachable variants until they're wired.

3. forceResumeCount is a lifetime counter; comment says "without new progress".

model-result.ts:2031 declares let forceResumeCount = 0; outside the while. Only mutation is forceResumeCount++ at line 2083. Never reset. So a Stop hook that force-resumes at turn 2, runs 50 clean turns, force-resumes at turn 53, runs 50 more, force-resumes at turn 104 — gets cut off on the third call despite 100 turns of real progress in between.

The console.warn at line 2079 asserts "forceResume honored 3 times without new progress", which the code does not enforce.

Fix: either reset forceResumeCount = 0 at the top of each while iteration that doesn't enter the shouldStopExecution() branch, or change the warning text and the comment at 2026-2030 to say "3 times total this session".

@mattapperson mattapperson left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reviewed, no issues found

@mattapperson mattapperson left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adversarial pass surfaced seven issues across the emit engine, matcher layer, and SessionStart/End pairing.

Comment thread packages/agent/src/lib/hooks-manager.ts Outdated
Comment thread packages/agent/src/lib/hooks-matchers.ts
Comment thread packages/agent/src/lib/hooks-types.ts
Comment thread packages/agent/src/lib/model-result.ts Outdated
Comment thread packages/agent/src/lib/hooks-emit.ts
Comment thread packages/agent/tests/unit/hooks-manager-adversarial.test.ts Outdated
Comment thread packages/agent/src/lib/hooks-resolve.ts

@mattapperson mattapperson left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the hooks engine and model-result integration; 5 concerns worth a second look on abort semantics, argument normalization, parse-error handling, clone depth, and timer lifetime.

Comment thread packages/agent/src/lib/hooks-emit.ts
Comment thread packages/agent/src/lib/model-result.ts Outdated
Comment thread packages/agent/src/lib/model-result.ts
Comment thread packages/agent/src/lib/hooks-emit.ts
Comment thread packages/agent/src/lib/hooks-emit.ts Outdated

@mattapperson mattapperson left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reviewed, no issues found

…to main)

Squashed rebase of feat/hooks-manager (fe319a2) onto origin/main (8edae63).
Conflicts resolved:
- async-params/call-model: keep both allowFinalResponse and hooks options
- model-result: runToolWithHooks now propagates executeTool's null (HITL
  pause); executeToolRound/processApprovalDecisions handle paused calls
  alongside parse_error/hook_blocked outcomes; broadcastToolResult calls
  updated for the new (id, source, result) signature; executeToolsIfNeeded
  merges the Stop-hook/forceResume/SessionEnd machinery with main's
  stoppedByStopWhen + allowFinalResponse + HITL-pause flow
The hooks integration made handleApprovalCheck pre-execute auto-approve
tools on every response and return false, after which the main loop ran
the same calls again via executeToolRound (and the round-0 + first-loop
double approval check made it 3x total) — a regression for all users,
hooks or not.

- handleApprovalCheck early-returns when nothing needs an approval gate
- PermissionRequest 'allow' defers execution to the normal tool round
- PermissionRequest 'deny' records the call id in hookDeniedCalls; the
  tool round synthesizes a rejected output instead of executing
- executeAutoApproveTools only runs when actually pausing for approval
  (results persisted as unsent for the resume path, as before)
- new loop-level regression tests pin exactly-once execution for: plain
  auto tools (no hooks), auto tools with hooks, hook-allowed gated
  tools, and hook-denied gated tools
@LukasParke LukasParke force-pushed the feat/hooks-manager branch from fe319a2 to 5373503 Compare July 14, 2026 21:25
devin-ai-integration[bot]

This comment was marked as resolved.

Engine correctness:
- executeHandlerChain: matcher/filter evaluation now runs under the same
  error policy as handlers -- a throwing user-supplied filter or function
  matcher is logged and the entry skipped in non-strict mode instead of
  rejecting the whole emit (addresses Devin review comment on
  hooks-emit.ts:119)
- HooksManager.emit: payload validation success now feeds parsed.data into
  the chain, so custom hooks with .transform()/.default()/.coerce see the
  schema OUTPUT type that zodInfer promises
- EmitResult gains a 'mutated' flag set only when a handler actually piped
  a mutation; model-result now uses it instead of reference comparisons
  (payload cloning by validation would false-positive those)
- isPlainMutableObject requires a true plain-object prototype, so Date/
  Map/class-instance custom payloads pass through un-mangled instead of
  being spread into {}

Contract fixes:
- asyncTimeout now aborts the emit's signal on expiry (cooperative
  cancellation) and warns; docs updated to state work cannot be forcibly
  cancelled -- handlers must observe context.signal
- the emit's AbortController stays registered until detached fire-and-
  forget work settles, so abortInflight() reaches async handlers that
  outlive the emit
- session teardown (finally block) is wrapped in try/catch so a throwing
  SessionEnd handler can never mask the original error (addresses Devin
  review comment on model-result.ts finally block); drain() now runs
  unconditionally so the approval-resume path (which skips SessionStart)
  still awaits fire-and-forget hook work
- SessionEnd is emitted (once, via emitSessionEndOnce) on the no-tools
  stream paths (getTextStream etc.), which previously fired SessionStart
  with no matching SessionEnd
- custom hooks with result: z.void() skip result validation by schema
  shape, matching built-in void hooks (was a hard-coded name list)
- PermissionRequest riskLevel accounts for call-level requireApproval
  functions; raw-string (JSON parse failure) arguments fail closed to
  ask_user before the hook fires
- matchesTool coerces function-matcher returns to boolean
- empty-string custom hook names are rejected at construction
- off() deletes empty entry lists from the map
- MUTATION_FIELD_MAP/BLOCK_HOOKS/BLOCK_FIELDS are frozen/readonly

API surface trim:
- index.ts no longer exports matchesTool, resolveHooks, BUILT_IN_HOOKS,
  BUILT_IN_HOOK_NAMES, or the raw Zod schema objects (mutable, semver
  surface); adds isAsyncOutput next to the AsyncOutput type

Tests:
- hooks-contract-fixes.test.ts: parsed.data piping, mutated flag,
  void-schema custom hooks, non-plain payloads, asyncTimeout/drain under
  fake timers, abortInflight reaching detached work
- hooks-session-lifecycle.test.ts: end-to-end SessionStart config,
  Start/End pairing on getText and getTextStream (no tools), SessionEnd
  reason max_turns, teardown error masking, inline-config e2e with
  matchers
- forceResume cap tests now drive the real executeToolsIfNeeded loop
  instead of reimplementing the cap locally
- adversarial tests updated from bug-documenting to regression-pinning
  (matcher coercion, filter error policy, empty hook name)
- afterEach(vi.restoreAllMocks) added to hook test files using inline
  console spies
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants