Preserve origin when mapping inline Playground rows - #213
Merged
David Elner (delner) merged 1 commit intoSep 16, 2026
Merged
Conversation
Luca Forstner (lforst)
approved these changes
Sep 16, 2026
Abhijeet Prasad (AbhiPrasad)
approved these changes
Sep 16, 2026
David Elner (delner)
deleted the
fix/preserve-origin-inline-playground-rows
branch
September 16, 2026 19:04
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What happened
Remote evals driven from a Playground never showed results. The dev server ran the eval to completion and streamed its output, but the Playground left every row in a loading state indefinitely.
Reproducing it depends on how the Playground sends data:
data.dataset_iddata.data(rows inline)To reproduce: start a dev server with an evaluator, point a Playground at it, link a dataset, select a subset of rows, and run. The server completes successfully while the UI keeps spinning.
Why it happened
When rows are sent inline they arrive detached from their dataset, so each one carries an
originpointer identifying the row it came from. The Playground matches streamed results back to its grid rows using that pointer.Eval#resolve_data_sourcerebuilt every inbound row from scratch and kept only two fields:origin,tagsandmetadatawere discarded. Everything downstream was already correct and waiting for them:Casedeclares anoriginmember,Runner#report_progressattaches origin to every progress event when present, and the SSE layer forwards it into the protocol. With origin dropped, progress events went out unidentified and the Playground had nothing to match them against.A second problem sat behind the first. Origin was represented as a JSON string on the dataset path but arrives from the wire as a hash. OpenTelemetry rejects hash attribute values without raising, so simply passing the hash through would have silently dropped
braintrust.originfrom eval spans and broken the dataset link on experiment rows.How this fixes it
The dev server stops rebuilding rows. It projects each inbound row onto the fields the SDK models, derived from the
Casecontract rather than restated alongside it:Deriving the list is what prevents a recurrence: adding a field to
Casenow carries it through the dev server automatically, instead of requiring a second edit that can be forgotten. Fields the SDK does not recognize are ignored rather than rejected, so a newer Playground cannot break an older SDK.Origin is also now a hash throughout the SDK, serialized once at the span boundary. That removes a serialize and re-parse round trip, and makes pointers received from the wire identical in shape to pointers built from the dataset API.