listSessions() spawns full CLI process — 900MB+ memory for metadata query

[tombelieber]

Summary

listSessions() from @anthropic-ai/claude-agent-sdk consumes ~900MB of memory per call because it spawns a full Claude Code CLI child process via stdio pipes to enumerate session files. This makes it unsuitable for periodic polling (e.g., a sidecar health check every 30 seconds).

Environment

• @anthropic-ai/claude-agent-sdk: latest (installed via npm)
• Node.js: v24.13.1
• macOS (Apple Silicon)
• ~50+ session JSONL files in ~/.claude/projects/

Reproduction

Minimal sidecar server that calls listSessions() on an HTTP endpoint:

import { listSessions } from '@anthropic-ai/claude-agent-sdk'

// On startup: RSS = 77 MB
// After 2 calls to listSessions(): RSS = 950 MB
const sessions = await listSessions()

Experiment: memory timeline

Time	Event	RSS
0s	Sidecar starts (Hono HTTP server, no SDK calls)	77 MB
3s	First GET /sessions → calls listSessions()	growing...
8s	Second GET /sessions → calls listSessions()	950 MB
5min	Idle, no further calls, 0 active sessions	1,340 MB

Memory breakdown (via Node.js inspector protocol)

RSS:          1,340 MB
├─ V8 Heap:     522 MB
├─ External:    341 MB
│  └─ ArrayBuffers: 337 MB  ← stdio pipe buffers
└─ Node overhead: ~77 MB

The 337 MB of ArrayBuffers strongly suggests stdio pipe buffering from the spawned CLI process. V8 does not return these pages to the OS after GC, so RSS ratchets up permanently with each call.

Control experiment

With listSessions() removed (only registry.list() for in-memory active sessions), the sidecar stays at 77 MB indefinitely.

Expected behavior

listSessions() should be a lightweight metadata query — reading file stats and first/last lines of JSONL files. It should not need to spawn a full Claude Code CLI process with its entire runtime just to enumerate sessions.

Ideally:

• < 10 MB memory overhead per call
• No child process spawn — direct filesystem scan
• Safe for polling every 10-30 seconds

Impact

Any application that periodically calls listSessions() (e.g., a session picker, dashboard, health monitor) will see memory grow to 1+ GB within seconds, even with zero active SDK sessions. The memory is never returned to the OS.

Workaround

We're replacing listSessions() with a direct filesystem scan of ~/.claude/projects/ to read session metadata, avoiding the SDK call entirely.

[tombelieber]

Update: Confirmed fix + before/after data

We replaced listSessions() with a direct in-memory registry lookup. Full before/after comparison after 30 polling calls to the endpoint:

Before (calling listSessions() on each request)

V8 Heap Used:   522 MB
External:       341 MB
ArrayBuffers:   337 MB
RSS:          1,340 MB

After (removed listSessions(), return registry data directly)

V8 Heap Used:    18 MB
External:         3 MB
ArrayBuffers:     0 MB
RSS:             81 MB

Timeline comparison

Scenario	Before	After
Baseline (startup)	77 MB	79 MB
After 10x polling	950 MB	80 MB
After 30x polling	1,340 MB	80 MB
Settled (idle)	1,340 MB (never drops)	80 MB (stable)

The 337 MB of ArrayBuffers (stdio pipe buffers from spawned CLI) drops to 0 when listSessions() is not called. The V8 heap drops from 522 MB to 18 MB.

Workaround

For listing sessions, we now scan ~/.claude/projects/ directly on the filesystem instead of calling listSessions(). This avoids spawning a CLI subprocess entirely.