Skip to content

feat(eve): add braintrustEveProvider for eve's instrumentation bridge - #2363

Closed
Chad Hietala (chadhietala) wants to merge 1 commit into
braintrustdata:mainfrom
chadhietala:feat/eve-instrumentation-provider
Closed

Chad Hietala (chadhietala) wants to merge 1 commit into
braintrustdata:mainfrom
chadhietala:feat/eve-instrumentation-provider

Conversation

@chadhietala

@chadhietala Chad Hietala (chadhietala) commented Aug 13, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Add a new Braintrust instrumentation provider (braintrustEveProvider) that consumes eve's lifecycle event bus — the instrumentation bridge proposed in vercel/eve#1799 and landed in eve ≥ 0.34.0.

Instead of parsing stream events manually (the legacy braintrustEveHook + braintrustEveInstrumentation path), the provider receives structured lifecycle events directly from eve's bus:

  • session.started/completed/failed → session metadata + span cleanup
  • turn.started/completed/failed → eve.turn TASK spans with subagent parent lineage
  • step.attempt.started/completed/failed → eve.step LLM spans
  • model.call.started/completed/failed → input/output/usage logged to the step span
  • action.started/completed/failed → TOOL spans for tools, subagents, remote agents

The provider uses eve's replay-stable idempotency keys for deterministic span IDs and ctx.state for durable per-operation state — replacing the old defineState + manual LLM-input capture machinery.

Content redaction

recordInputs and recordOutputs default to false (metadata only). When both are false, the provider declares capture: "metadata" so eve never builds the content projection at all. When either is true, the provider declares capture: "content" and guards each field independently in its handlers. This mirrors eve's per-destination ContentOptions model (agentRuns vs localTraces).

Usage

Enable the instrumentation providers layout in your agent config, then export the Braintrust provider from agent/instrumentation/braintrust.ts:

// agent/agent.ts
import { defineAgent } from "eve";

export default defineAgent({
  experimental: { instrumentationProviders: true },
  // ...model, tools, etc.
});
// agent/instrumentation/braintrust.ts
import { braintrustEveInstrumentation } from "braintrust";
import { defineInstrumentation } from "eve/instrumentation";

export default defineInstrumentation(
  braintrustEveInstrumentation({
    recordInputs: true,
    recordOutputs: true,
  }),
);

No defineState, no setup, no initLogger, no hook file — the provider initializes the Braintrust logger from BRAINTRUST_API_KEY by default. Pass setup only to override:

braintrustEveInstrumentation({
  recordInputs: true,
  recordOutputs: true,
  setup: ({ agentName }) => {
    initLogger({ projectName: agentName, apiKey: process.env.BRAINTRUST_API_KEY });
  },
})

On eve ≥ 0.34.0 with experimental.instrumentationProviders enabled, the provider handles everything — the legacy braintrustEveHook is a no-op. On older eve (or without the flag), both the hook and instrumentation work as before. The version branching is internal — the user never sees it.

Files

  • js/src/vendor-sdk-types/eve-provider.ts — vendored lifecycle event types
  • js/src/instrumentation/plugins/eve-provider.ts — provider implementation
  • js/src/instrumentation/plugins/eve-plugin.ts — version detection + let binding, default setup that calls initLogger from BRAINTRUST_API_KEY
  • js/src/exports.ts / js/src/instrumentation/index.ts — export wiring
  • e2e/scenarios/eve-instrumentation/agent/agent.ts — enables experimental.instrumentationProviders
  • e2e/scenarios/eve-instrumentation/agent/instrumentation.ts — uses the new clean API

Test plan

  • tsc --noEmit passes
  • Existing eve-plugin.test.ts (23 tests) passes
  • E2E scenario against eve ≥ 0.34.0 with provider path
  • E2E scenario against eve < 0.34.0 with legacy path (no regression)

@chadhietala
Chad Hietala (chadhietala) force-pushed the feat/eve-instrumentation-provider branch 5 times, most recently from fc82ee6 to f40e377 Compare August 13, 2026 18:59
@chadhietala
Chad Hietala (chadhietala) marked this pull request as ready for review August 13, 2026 19:10
@chadhietala
Chad Hietala (chadhietala) force-pushed the feat/eve-instrumentation-provider branch from f40e377 to c9e5569 Compare August 13, 2026 19:12
@chadhietala
Chad Hietala (chadhietala) marked this pull request as draft August 13, 2026 19:15
@chadhietala
Chad Hietala (chadhietala) force-pushed the feat/eve-instrumentation-provider branch 2 times, most recently from cec8024 to 329652f Compare August 13, 2026 19:19
@chadhietala
Chad Hietala (chadhietala) marked this pull request as ready for review August 13, 2026 19:21
@chadhietala
Chad Hietala (chadhietala) force-pushed the feat/eve-instrumentation-provider branch 3 times, most recently from 27e472c to 1c40da3 Compare August 13, 2026 22:48
Add a new Braintrust instrumentation provider that consumes eve's
lifecycle event bus (session/turn/step/model-call/action events)
instead of parsing stream events manually. The provider declares
capture: "content" or "metadata" and maps lifecycle events directly
to Braintrust spans, using eve's replay-stable idempotency keys for
deterministic span IDs and ctx.state for durable per-operation state.

Content redaction follows eve's per-destination ContentOptions model:
recordInputs and recordOutputs default to false, so the provider
captures metadata only (structure, identity, usage, timing) unless the
caller opts in. When both are false, the provider declares capture:
"metadata" so eve never builds the content projection at all. When
either is true, the provider declares capture: "content" and
respects its own redaction policy in each handler.

The version branching is internal to the SDK. braintrustEveInstrumentation
detects eve >= 0.34.0 at module scope and delegates to the provider
implementation; braintrustEveHook becomes a no-op on new eve since the
provider handles everything. The user's code does not change — they keep
importing braintrustEveInstrumentation and braintrustEveHook and passing
defineState, and the SDK uses the right path automatically.

New files:
- js/src/vendor-sdk-types/eve-provider.ts: vendored lifecycle event types
- js/src/instrumentation/plugins/eve-provider.ts: provider implementation

Signed-off-by: Chad Hietala <chadhietala@gmail.com>
@chadhietala
Chad Hietala (chadhietala) force-pushed the feat/eve-instrumentation-provider branch from 1c40da3 to 9abc77f Compare August 14, 2026 14:03
@lforst

Copy link
Copy Markdown
Member

Thanks! I think I am gonna give it a shot myself.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants