Place cuts with Silero rather than frame energy

[bojieli]

Answering "why did you not ship this?" — the blocker was real but solvable, and this solves it.

Why energy boundaries were worth replacing

Silero has been loaded and run on every recording since it replaced the noise-floor heuristic, but only ever as a yes/no gate. Where to cut was still frame energy against a 2nd-percentile floor, which fragments a long pause the moment a breath or keyboard tap crosses it:

pause ≥	energy	Silero
1.00 s	686	847
2.00 s	226	451

Measured on the 60 retained recordings past the splitting threshold:

	energy	Silero
median pause a cut lands in	0.76 s	2.14 s
cuts landing in a pause ≥ 1 s	40%	77%
final chunk, p50	17.9 s	27.7 s

The Swift implementation reproduces the offline prediction exactly — median 2.14 s across 54 cuts on the real corpus.

The longer final chunk is near-free on the offline path: request latency barely tracks chunk length (60 s → 2.43 s, 120 s → 2.98 s).

Why it wasn't done before

The live segmenter asks for a boundary every 200 ms once a minute is pending. Re-running the model over the pending buffer costs ~0.19 s of CPU per call at that size — roughly a whole core, sustained, for as long as no qualifying pause appears.

Silero is recurrent and built to stream, so the state and 64-sample context are now carried across calls and only new samples are analysed:

streaming: 300 s of audio in 1.62 s (185x real time)

fed in the 85 ms pieces the recorder actually delivers. A test holds that bound, to catch a regression back to re-scanning.

Two edges that are load-bearing

A still-open speech run counts when finding gaps. During capture the newest gap — the one worth cutting at — is always followed by a run that has not ended. Only its start is used, and Silero does not report a start until 250 ms of speech confirms it, which is exactly the evidence that the pause is over. I had this wrong first: excluding the open run made the newest pause invisible and the segmenter never cut. The live-session test caught it.

Fewer than two finalised runs hands the decision back to the energy finder, which needs no model and cannot fail. Audio Silero cannot parse still gets split rather than growing without bound — which matters more now, since an unbounded chunk is exactly the long-request case that truncates (#10).

Tests

SpeechStreamTests covers the risky part, the recurrent carry, on real speech:

• identical segments at 85 ms, 1 s and 4 s arrivals — arrival size cannot change the answer
• deliberately unaligned sizes (999/1237 bytes vs the 1024-byte window), so a dropped partial window is caught
• an open run is not finalised
• pause midpoints never land inside a speech run
• the throughput bound above

530 Swift tests, zero failures.

Scope

Swift only, so macOS and iOS. Android keeps energy boundaries — unchanged behaviour, no regression — and porting it needs the same streaming treatment. Windows has no live session. Worth a follow-up rather than bundling here.

Supersedes the "measured, not yet shipped" note in #12; that line needs updating once both land.

🤖 Generated with Claude Code

https://claude.ai/code/session_015XKX6zEAGiZFE5wAxJgYbV