Skip to content

Raise on LiteLLM response.failed streams - #907

Open
ousamabenyounes wants to merge 1 commit into
usestrix:mainfrom
ousamabenyounes:fix/issue-719-litellm-response-failed
Open

Raise on LiteLLM response.failed streams#907
ousamabenyounes wants to merge 1 commit into
usestrix:mainfrom
ousamabenyounes:fix/issue-719-litellm-response-failed

Conversation

@ousamabenyounes

Copy link
Copy Markdown
Contributor

Fixes #719.

Summary

  • Adds a Strix LiteLLM compatibility shim that raises when a Responses API stream yields response.failed.
  • Preserves the provider failure message/code in the raised APIError, so SDK retry policy can treat the failed stream as an actual model failure instead of normal completion.
  • Covers the real LiteLLM BaseResponsesAPIStreamingIterator._process_chunk path with a regression test.

Validation

  • RED before fix: uv run pytest tests/test_litellm_response_failed.py -q -> Failed: DID NOT RAISE APIError
  • GREEN after fix: uv run pytest tests/test_litellm_response_failed.py -q -> 1 passed
  • Revert proof: temporarily reverted only strix/config/models.py patch -> Failed: DID NOT RAISE APIError
  • uv run pytest tests/test_litellm_response_failed.py tests/test_model_retry.py tests/test_cost_tracking.py -q -> 17 passed
  • uv run ruff check strix/config/models.py tests/test_litellm_response_failed.py -> pass
  • uv run ruff format --check strix/config/models.py tests/test_litellm_response_failed.py -> pass
  • uv run mypy strix/config/models.py tests/test_litellm_response_failed.py -> pass
  • uv run bandit -q -c pyproject.toml strix/config/models.py -> pass

@greptile-apps

greptile-apps Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Adds a LiteLLM compatibility shim that converts streamed response.failed events into APIError exceptions while preserving provider error details.

  • Hooks LiteLLM’s Responses API streaming iterator during SDK model configuration.
  • Handles dictionary and object-shaped response events and supplies fallback error text.
  • Adds a regression test against the real LiteLLM streaming iterator path.

Confidence Score: 5/5

The PR appears safe to merge, with the failed-stream conversion covered through LiteLLM’s actual chunk-processing path.

The compatibility patch is idempotent across repeated SDK configuration, tolerates supported event representations, and turns the previously silent failed event into the retry-classifiable exception intended by the change.

Important Files Changed

Filename Overview
strix/config/models.py Adds an idempotent LiteLLM streaming compatibility patch that raises an API error when a transformed stream event reports response.failed.
tests/test_litellm_response_failed.py Adds focused regression coverage proving failed response events preserve their provider message and code in the raised exception.

Reviews (1): Last reviewed commit: "fix: raise litellm failed response strea..." | Re-trigger Greptile

@ousamabenyounes
ousamabenyounes force-pushed the fix/issue-719-litellm-response-failed branch from d0d4495 to 0feea6b Compare July 27, 2026 20:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

LiteLLM Responses API bridge silently swallows response.failed events as empty successful completions

1 participant