Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions .pylon/features.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -226,3 +226,20 @@ decisions:
revisit_when:
- Prime upstream exposes an equivalent privacy-safe, capability-negotiated cleanup proof with durable descriptor-last retries.
- Comet and Pylon can remove the fork capability without weakening crash cleanup or stock-Prime fallback behavior.

bounded-public-daemon-ingress:
area: runtime-reliability
state: candidate
owner: shared
decision: hybridize
pylon_refs:
- https://github.com/pylon-code/prime-agent/issues/13
- https://github.com/pylon-code/pylon/issues/114
- https://github.com/rynfar/comet/issues/5
upstream_refs:
- https://github.com/PrimeIntellect-ai/prime-agent/pull/480
fork_change: bounded-daemon-ingress-v1
upstream_support: Prime through a903d4b6768f has an optional decoded-character JSONL line limit with prefix callbacks and discard/resume behavior, but its public DaemonClient does not bound raw inbound frames or terminally reject recovery state before decode and parse.
revisit_when:
- Prime upstream exposes an equivalent public-root SDK proof, finite raw-byte default and override, privacy-safe typed terminal error, and no-replay reconnect semantics.
- Comet and Pylon can remove the fork token without weakening stock-package fail-closed detection or host memory bounds.
11 changes: 11 additions & 0 deletions .pylon/upstream-review.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,3 +81,14 @@ This ledger records Prime upstream evidence and the decision taken for each over
- Fork boundaries remain intact: no inherited Linear gate or release workflow is restored; optional protocol behavior remains capability-negotiated; the `main` mirror and `pylon` product branch roles are unchanged.
- Validation: exact merge `npm run check`, installer render, browser smoke, root build, and `git diff --check` passed; the live generated model catalog was restored. A clean-env focused boundary set passed 269 executions (18 continuation, 12 correlated lifecycle, 18 lazy-subagent, 109 supervisor-monitor, and 112 recursion). Native-install full shard 1 passed 1,466 tests with 24 skipped; shard 3 passed 1,241 with 22 skipped. Shard 2 recorded 1,865 passes with 9 skips while three load-sensitive cases tripped wall-clock/process thresholds; all 28 affected cases passed immediately in isolation. The real-process supervisor suite passed 13 tests with 8 fixture-gated skips. Both stock/current `v0.8.1` adoption directions and the 36 MiB compiled transfer bound passed. Trusted exact-head CI remains mandatory before merge.
- Independent exact-commit security/regression review found no P0/P1, verified both-parent ancestry and all 1,245 tree entries, and approved the retained conflict semantics. Revisit each hybrid only when Prime supplies an explicitly negotiated equivalent without weakening Pylon privacy, lifecycle, recovery, cleanup, or stock-Prime fallback guarantees.

## 2026-08-30 — bounded public daemon client ingress candidate

- Pylon base: `pylon@e7871eb699d0f65047a21d179216ebfec7755d0c`, the reviewed merge of upstream gate PR #15. Upstream evidence remains fully audited through `PrimeIntellect-ai/prime-agent@a903d4b6768f484bd6d459b7b0aa7dee38e461e2`; this client-local candidate does not advance `reviewed_upstream_commit`.
- Overlap decision: **hybridize**. Prime upstream PR #480 improves reconnect/request replay and current upstream provides generic optional `maxLineLength`, but the public `DaemonClient` still reads unbounded frames. The generic option counts decoded JavaScript units, exposes a prefix callback, discards through LF, and resumes. It cannot prove or implement a raw-byte, privacy-safe, terminal public-client boundary.
- Public contract: the immutable root-exported `PRIME_AGENT_SDK_FEATURES` registry contains `bounded_daemon_ingress_v1`; `DaemonClientOptions.maxInboundFrameBytes` has a finite 128 MiB default; and `DaemonInboundFrameTooLargeError` exposes only code `daemon_inbound_frame_too_large` and the configured limit. This is local artifact metadata, not a daemon capability, hello offer, protocol/schema/version signal, constructor-arity probe, or hostile-code attestation.
- Framing and recovery: the public socket remains in byte mode and copies pending bytes into bounded 64 KiB owned pages. LF is excluded and CR before LF is counted then stripped. Overflow is terminal for the exact socket identity, releases reader state, rejects connect/hello/all requests with preservation disabled, clears stale hello and replay state, notifies close listeners once, suppresses the current automatic recovery loop, and permits only a later explicit empty-buffer reconnect. Existing decoded character-limit/discard-resume behavior remains unchanged for its current stderr and RPC callers.
- Consumer budget: Prime keeps a compatibility-first 128 MiB per-frame default. Comet #5 will require the exact registry token, explicitly select 64 MiB, run the isolated long-lived Node host with a 512 MiB heap, keep downstream control frames/queues separately small, and map the typed SDK error to a fixed privacy-safe host code. Stock 0.8.1 remains bootstrap-compatible but is rejected locally for session hosting before create/attach because it lacks the token.
- Design review: independent Prime transport and Comet consumer reviews agreed on the default, byte/CRLF semantics, fixed-page allocation, terminal no-replay state, explicit reconnect, error privacy, and stock migration. They identified raw decoding, request preservation, discard/resume, auto-reconnect loops, retained buffer backing stores, connecting-phase races, native-error projection, and token-without-semantics as P0 gates.
- Validation: the two focused files pass 53 tests, including a real Unix-socket immediate-on-accept overflow, exact byte/CRLF/multibyte boundaries, many tiny chunks, connect/hello/all-request cleanup, preserved-request rejection, stale-hello clearing, no replay, running-auto-loop fencing, survival of an immediate explicit reconnect and its later fresh auto policy, one privacy-safe close, and cumulative bounded snapshot chunks. The real public-client transport receipt reconstructed 100 MiB across 267 snapshot chunks and 269 total frames (maximum 393,433 raw bytes, SHA-256 `5b05b298e974f3b9e40f0a1a8188f50984a4f18fb329e050324296632d3d9dfc`) and 500 MiB across 1,334 snapshot chunks and 1,336 total frames (maximum 393,435 raw bytes, SHA-256 `07171a278ab228de25c3d7ece952e0562d9d42d16d91580a9db5b80023d2493b`) through an actual `DaemonClient` configured to 64 MiB. These are deterministic cumulative transport/reconstruction receipts, not claims that the legacy monolithic benchmark is bounded attach integration. Both stock/current 0.8.1 adoption directions and the compiled 36 MiB indivisible-message supervisor-to-public-client regression passed (3/3). The clean real-process supervisor suite passed 13 tests with 8 fixture-gated skips. A 16-file wider affected run recorded 640 passes before one update-restart subprocess inherited the host worker's `PRIME_AGENT_INTERNAL_*` state and timed out before hello; after scrubbing every internal prefix, that exact case passed alone in 4.4 seconds. Exact valid 64 MiB frames passed under a 512 MiB V8 heap with 458 MB peak RSS/252 MB peak heap for ASCII; the high-expansion invalid-UTF-8 replacement case also passed with 670 MB peak RSS/452 MB peak heap, confirming the need for small Comet queues and OS headroom beyond the V8 ceiling. An exact valid default 128 MiB frame passed under a 1 GiB heap with 730 MB peak RSS/455 MB peak heap. `npm run check`, root build, installer/browser checks, YAML parse, and `git diff --check` pass; live model generation was restored. The rebuilt public root exports the frozen registry, client, options d.ts, default, and error, and `npm pack --dry-run` contains their JS/d.ts plus both docs. Installed stock 0.8.1 has no registry and silently ignores the second constructor argument, so namespace token gating fails closed as designed. Fresh API/resource re-review found no other current P0/P1. Exact committed-head security renewal and trusted hosted CI remain mandatory before merge.
- Revisit when Prime upstream provides an equivalent public-root proof and raw-byte terminal client contract, and both Pylon and Comet can remove the fork token without weakening stock-package detection, privacy, recovery integrity, or host memory bounds.
1 change: 1 addition & 0 deletions packages/coding-agent/.changes/bounded-daemon-ingress.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
- Bounded public daemon client JSONL ingress by raw bytes and added explicit SDK safety metadata ([#13](https://github.com/pylon-code/prime-agent/issues/13)).
18 changes: 16 additions & 2 deletions packages/coding-agent/docs/daemon.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ Due ticks are claimed and advanced before prompt delivery. A crash therefore doe

Resident workers keep scheduling across supervisor replacement. Worker recovery marks uncertain claims interrupted, keeps the advanced schedule, and resumes future ticks only. The supervisor routes schedule commands and merges worker summaries for global listing.

## Public Daemon Protocol v4
## Public Daemon Protocol v7

The public local socket is JSONL-framed. The current protocol provides:

Expand All @@ -94,6 +94,18 @@ Protocol v1 is retained only for the one-release update handoff that prepares an

JSON and RPC client modes do not expose daemon greetings, envelopes, snapshot records, lifecycle events, or connection metadata.

### Public client ingress bound

The public SDK `DaemonClient` bounds each inbound JSONL frame before UTF-8 decoding or JSON parsing. The default is 128 MiB and embedders can set a lower finite limit with `maxInboundFrameBytes`. Limits are positive safe integers and stay fixed for the lifetime of the client, including reconnects.

The bound counts raw bytes before LF. LF is excluded. A CR immediately before LF counts toward the limit and is then removed. The limit applies to each frame, not to the cumulative snapshot: normal snapshots remain valid as a sequence of bounded chunks. A single indivisible message or legacy monolithic frame above the limit requires a deliberate higher finite override.

Overflow, including bytes without LF, raises `DaemonInboundFrameTooLargeError`. It is terminal for that socket epoch: the client does not truncate, discard-and-resume, process later bytes from that socket, preserve requests for replay, or automatically reconnect to the same peer. Every handshake and request waiter is rejected, close listeners are notified once, and an explicit later reconnect starts with an empty byte buffer and the same configured bound. The error exposes only its stable code and configured limit, never frame content.

`PRIME_AGENT_SDK_FEATURES` contains `bounded_daemon_ingress_v1` when this complete client-local contract is present. This token describes the installed SDK artifact. It is not a daemon capability, hello offer, protocol version, or schema revision.

The raw frame limit bounds pending reader-owned bytes. A valid near-limit frame also needs memory for decoded strings, parsed objects, and consumer state, so embedders must set their process heap and downstream queues separately.

## Reconnect, Replay, and Snapshots

Every sequenced event belongs to a worker generation. Clients retain the last `{ generation, sequence }` cursor and present it on attach. The server reports whether the requested interval is complete, partial, or unavailable.
Expand Down Expand Up @@ -160,7 +172,9 @@ npx tsx test/daemon-multiclient-bench.ts
npx tsx test/daemon-multiclient-bench.ts --generated-session-mib 100
npx tsx test/daemon-multiclient-bench.ts --generated-session-mib 500
npx tsx test/daemon-multiclient-bench.ts --session-file /path/to/session.jsonl
npx tsx test/daemon-bounded-ingress-bench.ts --generated-session-mib 100
npx tsx test/daemon-bounded-ingress-bench.ts --generated-session-mib 500
PRIME_AGENT_STRESS_WORKERS=50 npx tsx ../../node_modules/vitest/dist/cli.js --run test/daemon-supervisor-process.test.ts -t "hosts resident roots"
```

The benchmark compares fanout and attach paths, including serialization count, throughput, elapsed time, and sampled RSS. The stress case starts many resident roots and verifies that their schedules advance independently while sessions are busy.
The multi-client benchmark compares legacy and chunked fanout/attach paths, including serialization count, throughput, elapsed time, and sampled RSS. Its legacy monolithic path is not an ingress-safety receipt. The bounded-ingress benchmark sends deterministic snapshot-like chunks through a real public `DaemonClient`, asserts each raw frame stays within both its 512 KiB target and a 64 MiB client limit, then verifies the full 100/500 MiB cumulative byte count and SHA-256. The stress case starts many resident roots and verifies that their schedules advance independently while sessions are busy.
25 changes: 25 additions & 0 deletions packages/coding-agent/docs/sdk.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,31 @@ The SDK is included in the main package. No separate installation needed.

## Core Concepts

### SDK feature metadata and bounded daemon ingress

Long-lived daemon embedders should verify client-local transport features from the package's public root before constructing a client. Use a namespace import when the same code must also load an older package: a named import of a new export fails while linking old ESM builds.

```typescript
const sdk = await import("@earendil-works/pi-coding-agent");
const features: unknown = sdk.PRIME_AGENT_SDK_FEATURES;
if (!Array.isArray(features) || !features.includes("bounded_daemon_ingress_v1")) {
throw new Error("This SDK cannot safely host a long-lived daemon session");
}

const client = new sdk.DaemonClient(socketPath, {
maxInboundFrameBytes: 64 * 1024 * 1024,
});
await client.connect();
```

`PRIME_AGENT_SDK_FEATURES` is immutable metadata for behavior implemented by the local SDK artifact. Do not infer it from package versions, constructor arity, method presence, daemon hello capabilities, protocol versions, or schema revisions. Older JavaScript constructors can silently ignore an extra options argument.

`DaemonClientOptions.maxInboundFrameBytes` is the maximum raw bytes before LF in one inbound JSONL frame. It defaults to `DEFAULT_DAEMON_CLIENT_MAX_INBOUND_FRAME_BYTES` (128 MiB) and must be a positive safe integer. LF is excluded. A CR immediately before LF is counted and then stripped.

`DaemonInboundFrameTooLargeError` has code `daemon_inbound_frame_too_large` and exposes the configured limit. It never includes frame content. Overflow terminally closes that socket, rejects handshake and request waiters even when request recovery was enabled, suppresses automatic replay/reconnect, and discards the partial buffer. A later explicit reconnect uses a fresh reader with the same bound. Applications that surface errors across a trust boundary should map the class or code to their own fixed message rather than forwarding an SDK error, stack, socket path, or daemon log path.

The frame limit is not a total heap limit. Valid frames also allocate decoded strings, parsed values, and application state. Set a lower explicit frame limit only with enough heap for valid boundary frames, and bound all downstream queues independently.

### createAgentSession()

The main factory function for a single `AgentSession`.
Expand Down
4 changes: 4 additions & 0 deletions packages/coding-agent/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -317,13 +317,15 @@ export {
type DaemonClientCapability,
type DaemonClientId,
type DaemonClientMessageListener,
type DaemonClientOptions,
type DaemonCommand,
type DaemonCommandEnvelope,
type DaemonCommandId,
type DaemonEventEnvelope,
type DaemonEventId,
type DaemonEventMeta,
type DaemonEventSequence,
DaemonInboundFrameTooLargeError,
type DaemonModeOptions,
type DaemonOutbound,
type DaemonOwnedSessionCleanupResult,
Expand All @@ -336,6 +338,7 @@ export {
type DaemonResponse,
type DaemonResumeCursor,
type DaemonSessionSnapshot,
DEFAULT_DAEMON_CLIENT_MAX_INBOUND_FRAME_BYTES,
defaultDaemonSocketPath,
type InteractiveInitialPrompt,
InteractiveMode,
Expand Down Expand Up @@ -410,6 +413,7 @@ export {
Theme,
type ThemeColor,
} from "./modes/interactive/theme/theme.js";
export { PRIME_AGENT_SDK_FEATURES, type PrimeAgentSdkFeature } from "./sdk-features.js";
// Clipboard utilities
export { copyToClipboard } from "./utils/clipboard.js";
export { parseFrontmatter, stripFrontmatter } from "./utils/frontmatter.js";
Expand Down
Loading
Loading