fix: honor dataset pipeline trace scope in the Python runner

[Abhijeet Prasad (AbhiPrasad)]

Problem

source.scope: "trace" was silently TypeScript-only.

The TS runner threads the source scope into transformArgsForCandidate, but the Python runner never read it — transform_args_for_candidate always took the span path. Trace-scoped discovery emits refs without a span row id (src/sync/discovery.rs:226), so a trace-scoped Python pipeline got input/output/expected/metadata all None, with no warning.

The Python SDK documents the opposite: DatasetPipelineTransformArgs marks those four as "for span-scoped transforms" and trace as "always available".

Fix

scripts/dataset-pipeline-runner.py resolves the merged source scope once per transform batch and passes only trace for trace scope, matching the TS runner. hydrate_discovery_refs now takes the already-merged source instead of re-merging it, and scripts/dataset-pipeline-runner.ts gets the same shape so the two runners stay diffable — that divergence is the mechanism that produced this bug.

An unknown scope value is left to the PipelineScope enum, which already rejects it while parsing the inspect response.

Verification

Against the real braintrust PyPI SDK (real DatasetPipeline + real call_user_fn dispatch; only login/LocalTrace stubbed):

trace-scope transform	before	after
transform(**kwargs)	['expected','input','metadata','output','trace'] (4 bogus Nones)	['trace']
transform(t)	t = None — the SDK's positional fallback handed it input	t = the trace

Both runners now emit byte-identical JSON for the same trace-scope pipeline and ref.

New smoke test python_runner_passes_scoped_transform_args covers both scopes against a stub braintrust package, reusing spawn/assert helpers with the existing TypeScript runner test. Against the pre-fix runner it fails with left: ["expected","input","metadata","output","trace"], right: ["trace"].

cargo test --bin bt: 854 passed, 0 failed. cargo fmt, tsc, oxlint, prettier clean.

Note

cargo clippy --all-features -- -D warnings fails on src/main.rs:193 (large_enum_variant for Trace(CLIArgs<bt_daemon::TraceArgs>)). Pre-existing, from the bt-daemon dependency, untouched here.

🤖 Generated with Claude Code

Co-authored by StarfolkAI (@starfolkai)[bot]

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-26T09:16:33.825240Z	32f9a66	New commits

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "Codex (@codex) review" or "Codex (@codex) security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[github-actions]

Latest downloadable build artifacts for this PR commit 32f9a669ff26:

• Workflow run: https://github.com/braintrustdata/bt/actions/runs/36232176367
• Download all artifacts (GitHub CLI): gh run download 36232176367 --repo braintrustdata/bt
• Installers are published from main automatically. To publish one for a PR branch, run release-canary manually via workflow_dispatch.

Available artifact names

• artifacts-build-local-x86_64-apple-darwin
• artifacts-build-global
• artifacts-build-local-x86_64-pc-windows-msvc
• artifacts-build-local-aarch64-unknown-linux-gnu
• artifacts-build-local-x86_64-unknown-linux-gnu
• artifacts-plan-dist-manifest
• cargo-dist-cache
• artifacts-build-local-x86_64-unknown-linux-musl
• artifacts-build-local-aarch64-apple-darwin
• artifacts-build-local-aarch64-pc-windows-msvc

[Stephen Belanger (Qard)]

Generally LGTM. Just one small, non-blocking nit.

[Stephen Belanger (Qard)]

There's a lot of Python fragments in here. Might be cleaner to just make a directory with all those files so they can be edited as plain files with proper syntax highlighting and not this awkward line formatting. Can just do include_str!("file.py") to load those in-place so you don't need to think about embedding the actual directory hierarchy into the Rust binary and can still just do the plain fs::write(...) calls here.

[Abhijeet Prasad (AbhiPrasad)]

Done in 06b69b2 — the stub braintrust package and the scope-probe pipeline are now plain files under src/datasets/pipeline-test-fixtures/:

src/datasets/pipeline-test-fixtures/
├── braintrust/{__init__,dataset_pipeline,framework,logger,trace}.py
└── scope_probe_pipeline.py

Loaded via include_str!, with the fs::write(...) calls kept — the five stub modules collapse into one (name, source) table, so write_fake_python_braintrust_package is now a 9-line loop. That drops 116 lines of embedded string from pipeline.rs.

The TypeScript test above it still has its fake braintrust package and pipeline inline. Same argument applies and it's mechanical — happy to move those too, just didn't want to grow this PR's diff into code it otherwise doesn't touch. Say the word.

[Abhijeet Prasad (AbhiPrasad)]

TypeScript fixtures extracted too, in 32f9a66. Fixtures are now grouped by runtime:

src/datasets/pipeline-test-fixtures/
├── node/
│   ├── braintrust/{package.json,index.cjs,index.mjs}
│   └── json_attachment_pipeline.ts
└── python/
    ├── braintrust/{__init__,dataset_pipeline,framework,logger,trace}.py
    └── scope_probe_pipeline.py

write_fake_node_braintrust_package mirrors the Python one — a (name, source) table of include_str!s and one fs::write loop. The setup in typescript_runner_defers_json_attachments_during_transform goes from ~120 lines of embedded string to four.

Across both commits pipeline.rs is down 249 lines of raw-string fixture. The fixture .ts/.cjs/.mjs files are picked up by prettier and oxlint (both pass); tsconfig.json only includes scripts/**, so the pipeline fixture isn't typechecked — it wasn't before either, since it was a string.

cargo test --bin bt: 854 passed, 0 failed.