Skip to content

fix: preserve reasoning content in streamed chat traces - #2522

Open
Mathew Stevens (mstevens843) wants to merge 1 commit into
braintrustdata:mainfrom
mstevens843:fix/2511-streamed-reasoning
Open

Mathew Stevens (mstevens843) wants to merge 1 commit into
braintrustdata:mainfrom
mstevens843:fix/2511-streamed-reasoning

Conversation

@mstevens843

Copy link
Copy Markdown

Related to #2511.

Preserve streamed delta.reasoning_content as message.reasoning_content, concatenating fragments independently per choice. Keep absent, empty and null-only values distinct without letting later nulls erase text. Ordinary content stays separate; reasoning and reasoning_details are unchanged.

Add constructed edge-case regressions and genuine DeepSeek recordings across six pinned/latest OpenAI SDK aliases through both wrapOpenAI and auto-hook instrumentation. Expectations come from ordered raw fragments, with paired snapshots and a patch changeset.

Support the scenario's dependency-bump workflow by forwarding DEEPSEEK_API_KEY to Docker by name and stripping it from dependency-install environments. Two dummy-only behavioral regressions exercise the actual command/environment paths and run in CI.

Fresh validation: 696 unique cases pass, including 87 focused, 589 compatibility, 15 helper/script, one genuine 12-path matrix and four changeset tests. The genuine matrix passes three consecutive keyless replays. Both credential defects were demonstrated RED before correction. Workspace/narrow typings, dependency builds, formatting, lint, script syntax and changeset checks pass.

Earlier identical genuine recordings demonstrated reasoning loss on untouched base after content passed. Optional whole-E2E type checking previously reported 323 identical base/patch diagnostics; it was not rerun in the maintenance pass. Actual Docker execution and other runtime/platform CI lanes remain unrun locally. No new provider requests were made for the maintenance correction.

Credit to the reporter and prior provider work linked in #2511. Prepared and independently reviewed with AI assistance; human review and any personal attestations remain with the submitter.

@mstevens843
Mathew Stevens (mstevens843) marked this pull request as ready for review September 25, 2026 07:10

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant