streaming: one shared publisher for partial text, and two families that restated the transcript

[christopherthompson81]

Review copy in the fork — not for upstream yet. Fixes AudioCpp-Bindings#68.

0xShug0#68 turned out to be a family bug, and then a missing abstraction

The issue reported that transcript.text.delta carries a running total for some families and an increment for others, and proposed two server-side fixes: diff in the server, or add a "cumulative": true field so a client can branch.

Neither is needed. There is no ambiguity to resolve, because the engine already has one contract. app/cli/partial_render.h states it:

Partials are the text newly decoded since the last one, so they concatenate into the transcript.

The server forwards each partial as an OpenAI-shaped transcript.text.delta, which is incremental by specification, so the server was right all along. This PR does not touch it.

Why one family got it wrong: there was nothing to be right with

Every family that publishes partial text carried its own copy of the arithmetic, and the copies disagreed. Three shapes across seven families:

shape	families
common-prefix diff, duplicated verbatim (helper and caller)	higgs_audio_stt, vibevoice_asr
published-bytes offset, open-coded	voxtral_realtime, qwen3_asr, sense_asr
no diffing — publishes the whole running transcript	parakeet_tdt, kroko_asr

kroko_asr is a second instance of the reported bug, found by sweeping for the pattern rather than by tripping over it:

const auto decoded = combined_decoded();
const auto result = make_result(decoded, ...);
event.partial_text = result.text_output;   // everything decoded so far

Same corruption, same cause, and it would have survived a parakeet-only fix. That is what convinced me the scope had to be the abstraction rather than the symptom.

The change

engine::runtime::PartialTextPublisher is the single implementation; all seven families use it. It holds what has actually been published and returns the increment, with two properties none of the copies had:

• A delta never ends part way through a UTF-8 sequence. A tokenizer falls back to bytes for text its vocabulary does not cover, so a decode can stop mid-character. Publishing that puts half a code point on the wire, and it reaches the SSE JSON encoder as invalid UTF-8. The tail is held for the update that completes it. (Every prior implementation, including my own first version in this PR, got this wrong.)
• A delta never starts inside one either, when a decode revises published text rather than only extending it.

To be precise about that second one, since it is easy to over-read: on a genuine revision nothing can retract a delta already sent, so the consumer's transcript is wrong either way. The boundary handling stops it also being spliced into the middle of a character. It does not make revisions safe.

word_timestamps deliberately stays cumulative alongside the now-incremental partial_text. It is not a delta field — it is the finalized set so far, which is why the provisional last word is dropped. Each field matches its own contract rather than matching each other; commented in place, because the text field changing shape makes that newly confusing.

Verification

Every migrated family was installed and driven through the CLI on
assets/resources/sample_16k.wav, not just compiled:

family	package	partials	partials concatenate to text_output
parakeet_tdt	0.6B-v3 q8_0	7	yes
kroko_asr	community q8_0	11	yes, after a fix — see below
voxtral_realtime	Mini-4B q4_k	33	yes
higgs_audio_stt	v3-STT q8_0	4	yes
sense_asr	SenseVoice-Small q8	1	yes
qwen3_asr	0.6B q8_0	1	yes
vibevoice_asr_streaming	7B q4_k	5	unchanged from baseline — see below

voxtral_realtime is the strongest of these: 33 token-level partials, so the
publisher runs once per token rather than once per window. sense_asr and
qwen3_asr are the weakest — the clip fits one window, so one partial goes out
and the diff never runs twice.

Plus tests/unittests/test_partial_text.cpp (growth, 2/3/4-byte characters
arriving one byte at a time, a revision, a shrinking decode, reset(), empty
edges) and the AudioCpp-Bindings suite green against this build across
kokoro_tts, citrinet_asr, parakeet_tdt, sortformer_diar and
bs_roformer.

kroko_asr needed a fourth commit, and only running it found it

Its finalize() was:

auto event = process_streaming_audio(true);
result.text_output = event.partial_text;

It read the whole transcript out of the partial — which worked only because
the partial restated the whole transcript every time. Making partials
incremental therefore broke the final result: a run that correctly emitted
eleven deltas ended with text_output = " times longer than you".

Both sides are std::optional<Transcript>, so this compiles equally well before
and after and no test in the tree covered it. I confirmed it was mine rather
than pre-existing by rebuilding origin/main's copy of the two kroko files: the
baseline emits eleven copies of a growing transcript and a correct
text_output. finalize() now builds its result through make_result(), the
same path the offline route uses.

Swept for the shape elsewhere; the only other hit is
src/capi/audiocpp.cpp:226, which is correct and documented — it presents a
stream event's partial through that event's text accessor, which is what a
delta is.

A correction: the vibevoice_asr_streaming "bugs" were mine, not the code's

An earlier revision of this description claimed two pre-existing defects in
vibevoice_asr_streaming — that the first window's text never reached a partial,
and that text_output came back as a lone space. Both were wrong, and the
error was in how I read the output rather than in the engine.

That family's transcript legitimately begins "\n Speaker 0:". The CLI's
non-interactive format writes partial_text=<text>\n, so a transcript
containing a newline spills onto the next line, and the grep '^partial_text='
I was extracting with stopped at it. Everything after the newline was invisible
to my check, which is why the field looked empty and why the concatenation did
not match.

Instrumenting the session settled it — finalize() was returning all 174 bytes
the whole time — and parsing the output on the partial_text= markers instead of
per line gives:

partials: 5
    ' \n Speaker 0:Some call me nature, '
    'others call me mother nature. '
    "I've been here for over four point five "
    'billion years. Twenty two thousand '
    'five hundred times longer than you.'
concat == text_output: True

So this family is correct, and correct both before and after this PR. Nothing to
fix and nothing to file. Flagging it here because the claim was in this
description and someone may have read it.

The one thing the episode does surface is a real, minor diagnostic wart: the
CLI's partial_text=/text_output= lines are not parseable when a transcript
contains a newline, since nothing delimits or escapes it. That is a debug output
format, not an API, so I have not touched it.

What is still not covered

• The byte-fallback path is covered by unit test, not by a real multilingual
  decode. I have no sample that drives a tokenizer into fallback, so the
  held-back-character path has not run against a live model.
• nemotron_asr and the two vibevoice_asr chunk-append sites are deliberately
  untouched: they receive deltas produced upstream rather than diffing a running
  transcript, so the publisher does not apply. Whether those deltas can split a
  character is a question for someone who knows the decoders.

[christopherthompson81]

Reviewed and sent upstream as 0xShug0#552. Closing — this copy has done its job.