Batch the telemetry transport - #423
Merged
TheGreatAxios merged 4 commits intoAug 9, 2026
Merged
Conversation
Two upcoming event catalogs turn telemetry volume from one event per turn into one per tool call, which the previous transport would have answered with an unbounded set of concurrent per-event POSTs. Queueing behind one request bounds both sockets and memory when the endpoint is unreachable.
Dropping the singleton on opt-out left the outgoing instance's batch timer armed, so events captured before the toggle would still reach the network afterwards. Opting out speaks to activity already generated, not only to activity still to come.
The only existing deadline test resolved its fetch immediately, so it would have kept passing even with the deadline race deleted. The new test gates the fetch open forever and asserts flush() still returns, proving the race is load-bearing. Also documents that a batch already on the wire when a user opts out cannot be recalled, matching what the code comment already says.
4 tasks
TheGreatAxios
merged commit Aug 9, 2026
100244b
into
cl-5727-posthog-identity-session-linking
3 checks passed
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces the per-event telemetry transport with a queued, batched one.
Why this has to land first
The transport today opens one
fetchPOST to/capture/per event, with noqueue, no cap, and no concurrency limit. That was tolerable while the event
catalog was O(turns). Two PRs about to land make it O(tool calls): one adds a
permission-prompt event per approval plus subagent start/end and compaction
events, the other adds a generation event per turn plus a span event per tool
call per turn.
A 100-turn session at five tool calls per turn is roughly 700 individual
HTTPS POSTs. A single 40-tool-call turn fires 41 concurrent fetches. Against a
hung endpoint - captive portal, corporate proxy, provider degradation - that
is hundreds of concurrent in-flight sockets and an unbounded set of pending
promises on a user's laptop, silently, for the length of the session. Either
of those PRs merged ahead of this one ships that behavior.
What changed
/batch/endpoint:{ api_key, batch: [{ event, timestamp, properties }] }, withdistinct_idinside each event'sproperties. Each event carries the timestamp of its capture, so batching
does not smear event times.
size, or when the batch interval elapses, whichever comes first.
the endpoint is unreachable the oldest queued events are also the least
worth reporting, and unbounded growth is not an acceptable alternative.
mid-flight are picked up by that loop's next iteration instead of opening a
second socket.
flush()keeps its existing 500ms deadline and drains through the samesingle-request path, so the fatal-crash handler still cannot be held up.
fetchFnseam is unchanged.One behavioral consequence worth naming: toggling telemetry off discards the
outgoing instance along with anything still queued on it. That is the right
direction for an opt-out.
Tests
New coverage for the size trigger, the time trigger, drop-oldest on overflow,
single-in-flight serialization,
flush()draining a partial queue inside itsdeadline, and the
/batch/endpoint itself.Separately, a test for the combined failure: the queue filling while a
request is hung. Drop-oldest and single-in-flight have to hold at the same
time - the queue caps and sheds oldest while one request is stuck and no
second request opens. That is the case that bites a user on a captive portal,
and a design can satisfy both rules independently and still fail it.