[bot] OpenAI Responses API instrumentation only captures response `id`, missing status/service_tier/model/etc.

[braintrust-bot]

Summary

The Responses API instrumentation in both the openai and ruby-openai integrations captures almost none of the response object's fields into span metadata after the call completes. finalize_metadata only ever sets metadata["id"]. This is materially less detail than the sibling Chat Completions instrumentation in this same SDK, which captures id, created, model, system_fingerprint, and service_tier from the response object after every call (including streaming).

This is a distinct gap from #145 (which covers wrong/missing request-side METADATA_FIELDS, e.g. max_tokens vs max_output_tokens). This issue is about response-side fields that are dropped after the call returns.

What is missing

The upstream Responses API response object (OpenAI::Models::Responses::Response) exposes at minimum:

• status — e.g. completed, incomplete, failed, in_progress — critical for knowing whether a generation actually finished successfully
• incomplete_details — explains why a response is incomplete (e.g. hit max_output_tokens) — directly useful for debugging truncated generations
• error — error details when the response object itself reports a failure
• model — the actual model that served the request (may differ from the requested model, e.g. due to aliasing)
• service_tier — the processing tier that actually served the request (may differ from the requested tier)
• created_at — response creation timestamp
• background — whether the response was run in background/async mode

None of these are captured. The current finalize_metadata implementations:

# lib/braintrust/contrib/openai/instrumentation/responses.rb:109-112
def finalize_metadata(span, metadata, response)
  metadata["id"] = response.id if response.respond_to?(:id) && response.id
  Support::OTel.set_json_attr(span, "braintrust.metadata", metadata)
end

# lib/braintrust/contrib/ruby_openai/instrumentation/responses.rb:137-140
def finalize_metadata(span, metadata, response_data)
  metadata["id"] = response_data["id"] if response_data["id"]
  Support::OTel.set_json_attr(span, "braintrust.metadata", metadata)
end

only ever set id. The same gap exists in the streaming path (ResponseStream#finalize_stream_span in lib/braintrust/contrib/openai/instrumentation/responses.rb:164-181, and Common.aggregate_responses_chunks/aggregate_responses_events in the respective common.rb files), which likewise only propagate id through to metadata.

For contrast, the parallel Chat Completions finalize_metadata (lib/braintrust/contrib/openai/instrumentation/chat.rb:129-136) captures id, created, model, system_fingerprint, and service_tier — showing this is an internal inconsistency between two instrumented surfaces for the same underlying gem, not a hard technical limitation.

Braintrust docs status

unclear — https://www.braintrust.dev/docs/guides/tracing states that Braintrust auto-instrumentation captures "Model parameters (model name, temperature, etc.)" generally, and the Braintrust docs' own Responses API example (openai.responses.create(...)) is presented as fully auto-traced with "inputs, outputs, latency, tokens, and cost" — but no page enumerates which specific response-object fields (status, incomplete_details, service_tier, etc.) should end up in span metadata for the Responses API.

Upstream sources

• Official OpenAI Ruby SDK: https://github.com/openai/openai-ruby — OpenAI::Models::Responses::Response (lib/openai/models/responses/response.rb) defines status, incomplete_details, error, model, service_tier, created_at, background
• OpenAI Responses API reference: https://platform.openai.com/docs/api-reference/responses/create — documents the response object schema, including status values (completed, failed, in_progress, incomplete) and incomplete_details.reason (e.g. max_output_tokens)
• OpenAI cookbook example response payload confirming these fields: https://cookbook.openai.com/examples/responses_api/reasoning_items (shows id, created_at, error, incomplete_details, model on a real response object)

Braintrust docs sources checked

• https://www.braintrust.dev/docs/guides/tracing
• https://www.braintrust.dev/docs

Local repo files inspected

• lib/braintrust/contrib/openai/instrumentation/responses.rb (lines 109-112, 164-181) — finalize_metadata and streaming finalize_stream_span only ever set metadata["id"]
• lib/braintrust/contrib/ruby_openai/instrumentation/responses.rb (lines 137-140) — identical finalize_metadata, same gap
• lib/braintrust/contrib/openai/instrumentation/chat.rb (lines 129-136) — sibling Chat Completions finalize_metadata captures id, created, model, system_fingerprint, service_tier, demonstrating the established pattern this SDK already uses elsewhere
• lib/braintrust/contrib/openai/instrumentation/common.rb (aggregate_responses_events, lines 112-129) — only returns id, output, usage from the completed event's response, dropping status/incomplete_details/model/service_tier/error
• lib/braintrust/contrib/ruby_openai/instrumentation/common.rb — analogous streaming aggregator for ruby-openai (not re-quoted here, same field-dropping pattern)