feat(providers): ElevenLabs scribe_v2_realtime streaming transcription (RFC-0034 P2) - #184
Merged
Merged
Conversation
This was referenced Sep 13, 2026
eric8810
force-pushed
the
rfc-0034/elevenlabs-realtime
branch
from
September 15, 2026 04:02
9097575 to
9c2afbf
Compare
eric8810
marked this pull request as ready for review
September 15, 2026 04:23
…n (RFC-0034 P2)
ElevenLabsTranscriptionModel gains do_stream behind the existing
'realtime' feature, gated on scribe_v2_realtime* (inverse gating in
do_generate rejects realtime IDs with a clear message instead of
posting them to the batch endpoint).
Wire shape per the public API reference (2026-09), pinned
field-by-field by local mock-server tests:
- config travels on the URL query (model_id, audio_format
pcm_{rate}/ulaw_{rate}, commit_strategy=manual, optional
language_code/include_timestamps from providerOptions.elevenlabs);
xi-api-key rides the WS handshake headers
- audio: base64 in input_audio_chunk JSON with sample_rate; a
hold-one-chunk pipeline puts commit:true on the LAST real chunk (an
empty stream commits a single empty chunk); audio waits for
session_started
- partial_transcript -> TranscriptPartial (replace semantics),
committed_transcript[_with_timestamps] -> TranscriptFinal + Finish
(segments from words, spacing entries dropped), then client
close(1000)
- 14 documented error event names classified: rate_limited /
queue_overflow / resource_exhausted retry, the rest terminal
- the server never closes after commit, so a settle window (chunk_ms,
5s default) bounds the wait with an empty Finish — the stream always
terminates
- non-pcm/ulaw input formats fail fast without connecting
Live-API smoke pending (no key at implementation time) — same posture
as RFC-0028 D4; the do_stream doc comment says so.
Also: cartesia.rs gate comment now records the P3 deferral (RFC-0034
D6); RFC status/plan rows updated.
eric8810
force-pushed
the
rfc-0034/elevenlabs-realtime
branch
from
September 15, 2026 04:24
9c2afbf to
6eb5351
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements P2 of RFC-0034: ElevenLabs
scribe_v2_realtimestreaming transcription (do_streamonElevenLabsTranscriptionModel, behind the existingrealtimefeature).Stacked on #183 (P1, WS proxy) — until that merges, this diff includes P1's commits; it collapses to the P2 delta afterwards. P3 (Cartesia) is deferred with explicit restart triggers (RFC-0034 §4/D6); the gate comment in
cartesia.rsrecords it.What lands
scribe_v2_realtime*→do_stream; everything else →do_generate.do_generatenow rejects realtime IDs with a clear message instead of posting them to the batch REST endpoint.commit_strategy=manualfixed, D3;languageCode/includeTimestampsthe only provider options, D5),xi-api-keyhandshake header, base64input_audio_chunkframes withsample_rate,commit:trueriding the last real chunk (hold-one-chunk pipeline; empty stream commits an empty chunk),partial_transcript→TranscriptPartial(replace semantics),committed_transcript[_with_timestamps]→TranscriptFinal+Finish(word segments, spacing dropped), clientclose(1000)after the final.chunk_ms, 5 s default) bounds the wait with an emptyFinish— mirroring Align the request pipeline with the AI SDK #164's terminal-fuse principle.rate_limited/queue_overflow/resource_exhaustedretryable, the rest terminal; raw event preserved inresponse_body.Verification
aimux-providers/tests/elevenlabs_realtime_stream_test.rs(happy path with query/header/chunk-frame assertions, timestamps variant, empty-audio commit, 4-way error classification, silent-server settle, mid-session abort, symmetric gating, format rejection). Workspace clippy clean.do_streamdoc comment. Smoke through a proxy also closes P1's untested wss+rustls branch.Closes the P2 scope of #178.