feat(eve): add braintrustEveProvider for eve's instrumentation bridge - #2363
Closed
Chad Hietala (chadhietala) wants to merge 1 commit into
Closed
Chad Hietala (chadhietala) wants to merge 1 commit into
Chad Hietala (chadhietala) wants to merge 1 commit into
Conversation
Chad Hietala (chadhietala)
force-pushed
the
feat/eve-instrumentation-provider
branch
5 times, most recently
from
August 13, 2026 18:59
fc82ee6 to
f40e377
Compare
Chad Hietala (chadhietala)
marked this pull request as ready for review
August 13, 2026 19:10
Chad Hietala (chadhietala)
force-pushed
the
feat/eve-instrumentation-provider
branch
from
August 13, 2026 19:12
f40e377 to
c9e5569
Compare
Chad Hietala (chadhietala)
marked this pull request as draft
August 13, 2026 19:15
Chad Hietala (chadhietala)
force-pushed
the
feat/eve-instrumentation-provider
branch
2 times, most recently
from
August 13, 2026 19:19
cec8024 to
329652f
Compare
Chad Hietala (chadhietala)
marked this pull request as ready for review
August 13, 2026 19:21
5 tasks
Abhijeet Prasad (AbhiPrasad)
requested review from
Abhijeet Prasad (AbhiPrasad) and
Luca Forstner (lforst)
August 13, 2026 19:46
Chad Hietala (chadhietala)
force-pushed
the
feat/eve-instrumentation-provider
branch
3 times, most recently
from
August 13, 2026 22:48
27e472c to
1c40da3
Compare
Add a new Braintrust instrumentation provider that consumes eve's lifecycle event bus (session/turn/step/model-call/action events) instead of parsing stream events manually. The provider declares capture: "content" or "metadata" and maps lifecycle events directly to Braintrust spans, using eve's replay-stable idempotency keys for deterministic span IDs and ctx.state for durable per-operation state. Content redaction follows eve's per-destination ContentOptions model: recordInputs and recordOutputs default to false, so the provider captures metadata only (structure, identity, usage, timing) unless the caller opts in. When both are false, the provider declares capture: "metadata" so eve never builds the content projection at all. When either is true, the provider declares capture: "content" and respects its own redaction policy in each handler. The version branching is internal to the SDK. braintrustEveInstrumentation detects eve >= 0.34.0 at module scope and delegates to the provider implementation; braintrustEveHook becomes a no-op on new eve since the provider handles everything. The user's code does not change — they keep importing braintrustEveInstrumentation and braintrustEveHook and passing defineState, and the SDK uses the right path automatically. New files: - js/src/vendor-sdk-types/eve-provider.ts: vendored lifecycle event types - js/src/instrumentation/plugins/eve-provider.ts: provider implementation Signed-off-by: Chad Hietala <chadhietala@gmail.com>
Chad Hietala (chadhietala)
force-pushed
the
feat/eve-instrumentation-provider
branch
from
August 14, 2026 14:03
1c40da3 to
9abc77f
Compare
Member
|
Thanks! I think I am gonna give it a shot myself. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add a new Braintrust instrumentation provider (
braintrustEveProvider) that consumes eve's lifecycle event bus — the instrumentation bridge proposed in vercel/eve#1799 and landed in eve ≥ 0.34.0.Instead of parsing stream events manually (the legacy
braintrustEveHook+braintrustEveInstrumentationpath), the provider receives structured lifecycle events directly from eve's bus:session.started/completed/failed→ session metadata + span cleanupturn.started/completed/failed→eve.turnTASK spans with subagent parent lineagestep.attempt.started/completed/failed→eve.stepLLM spansmodel.call.started/completed/failed→ input/output/usage logged to the step spanaction.started/completed/failed→ TOOL spans for tools, subagents, remote agentsThe provider uses eve's replay-stable idempotency keys for deterministic span IDs and
ctx.statefor durable per-operation state — replacing the olddefineState+ manual LLM-input capture machinery.Content redaction
recordInputsandrecordOutputsdefault tofalse(metadata only). When both arefalse, the provider declarescapture: "metadata"so eve never builds the content projection at all. When either istrue, the provider declarescapture: "content"and guards each field independently in its handlers. This mirrors eve's per-destinationContentOptionsmodel (agentRunsvslocalTraces).Usage
Enable the instrumentation providers layout in your agent config, then export the Braintrust provider from
agent/instrumentation/braintrust.ts:No
defineState, nosetup, noinitLogger, no hook file — the provider initializes the Braintrust logger fromBRAINTRUST_API_KEYby default. Passsetuponly to override:On eve ≥ 0.34.0 with
experimental.instrumentationProvidersenabled, the provider handles everything — the legacybraintrustEveHookis a no-op. On older eve (or without the flag), both the hook and instrumentation work as before. The version branching is internal — the user never sees it.Files
js/src/vendor-sdk-types/eve-provider.ts— vendored lifecycle event typesjs/src/instrumentation/plugins/eve-provider.ts— provider implementationjs/src/instrumentation/plugins/eve-plugin.ts— version detection +letbinding, defaultsetupthat callsinitLoggerfromBRAINTRUST_API_KEYjs/src/exports.ts/js/src/instrumentation/index.ts— export wiringe2e/scenarios/eve-instrumentation/agent/agent.ts— enablesexperimental.instrumentationProviderse2e/scenarios/eve-instrumentation/agent/instrumentation.ts— uses the new clean APITest plan
tsc --noEmitpasseseve-plugin.test.ts(23 tests) passes