webui: make the committed bundle reproducible - #4
Closed
christopherthompson81 wants to merge 2 commits into
Closed
christopherthompson81 wants to merge 2 commits into
christopherthompson81 wants to merge 2 commits into
Conversation
* webui: split TTS text on sentences in Latin-script text
splitTtsChunks treats 。！？!?；;… as sentence terminators. ASCII '.' is not
among them, so a paragraph of English prose is one unsplittable sentence
and falls through to the fixed-width cut, which lands mid-word:
splitTtsChunks("The first sentence is here. The second follows it
closely. A third arrives now. ...", 60)
[60] The first sentence is here. The second follows it closely. A
[60] third arrives now. And a fourth, rather longer than the oth
[60] ers, continues past the point where a small budget would hav
[26] e to cut. Finally a fifth.
Each chunk is a separate synthesis request, so a word split across two of
them is pronounced as two fragments.
'.' now terminates a sentence, with the guards that make it ambiguous in
the first place: not between digits, not after an abbreviation or a
single-letter initial, and only before whitespace. So 3.14159, Dr. Smith,
J. R. R. Tolkien, file.txt and example.com stay whole.
Where no sentence boundary fits the budget, the fallback breaks on words
rather than characters, so only a token longer than the entire budget is
cut mid-word. The same text now gives:
[58] The first sentence is here. The second follows it closely.
[20] A third arrives now.
[59] And a fourth, rather longer than the others, continues past
[49] the point where a small budget would have to cut.
[16] Finally a fifth.
Two smaller fixes alongside: a "Speaker 1:" prefix is counted against the
budget, since it is repeated onto every chunk a line produces and those
chunks otherwise exceed the caller's limit; and chunks are trimmed, so a
prefixed chunk no longer carries a double space.
CJK behaviour is unchanged -- the existing terminators still apply, and
'.' is additive.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EkxqpYvUbjCpRDnFiNiVfx
* webui: keep the spacing that separated the sentences
Packing threw away the whitespace splitSentences had carefully kept and put a
single space back in its place. Three coupled lines assumed that separator was
always one space: trimEnd() dropped it, the budget check added 1 for it, and
the join wrote it.
Sentences in Chinese, Japanese and Korean are adjacent -- a full-width
terminator and nothing else -- so this invented a space that was never in the
text. That is not only an extra request: it changes what the model is asked to
speak, and it spends a character of the budget, so three 20-character
sentences stopped fitting in two 40-character chunks.
before: [20] ...吧。 [20] ...吧。 [20] ...吧。
after: [40] ...吧。...吧。 [20] ...吧。
The separator that actually followed each sentence is carried instead. One
other case changes with it, deliberately: "First one. Second one." keeps
its four spaces rather than being silently collapsed to one. Collapsing was an
edit to the user's text that nobody asked the chunker to make.
Checked against a 17-case corpus covering abbreviations, initials, speaker
prefixes, over-long tokens, multiple spacing, newlines, CJK and mixed scripts;
those two cases are the only ones whose output moves.
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
christopherthompson81
force-pushed
the
webui/deterministic-bundle
branch
from
September 14, 2026 02:05
90a1c87 to
aee558f
Compare
dist/index.html is committed and embedded into the server binary at configure
time (CMakeLists.txt:2226), which is what lets the project build a working
server without a JavaScript toolchain. But the bundle is not reproducible:
SvelteKit defaults kit.version.name to Date.now() and derives the
__sveltekit_<id> global it embeds from it, so two builds of identical source
differ.
before: build 1 __sveltekit_1ia7gsf sha256 bb3f9c3c...
build 2 __sveltekit_1xqgjp8 sha256 6cd91a94...
after: build 1 __sveltekit_1vzi1g sha256 65bed68b...
build 2 __sveltekit_1vzi1g sha256 65bed68b...
Any two branches that rebuild the web UI therefore conflict in that file
whether or not their source changes overlap -- which is what happened to 0xShug0#539,
where dist/index.html was the only conflict while src/lib/text.ts merged
cleanly and no upstream commit had touched either file.
That the id is a pure function of this setting was confirmed by building with
the timestamp the committed bundle carries: the derived id came back as
__sveltekit_ega6lw, and the result was byte-identical to the committed file.
The package version is used rather than a constant. It is stable for a given
source tree, so the bundle reproduces, and it changes when the version is
bumped, so SvelteKit's client-side "app has been updated" check keeps working
across releases. A hard-coded string would have disabled that silently.
dist/index.html is rebuilt here so the tree is consistent: after this, running
npm run build leaves git clean instead of producing a diff every time.
Normalising the build id, the version constant and the Vite content hashes --
all three derived from this one setting -- leaves zero differing lines against
the committed bundle, so nothing else in the UI changes.
Reported as 0xShug0#545.
christopherthompson81
force-pushed
the
webui/deterministic-bundle
branch
from
September 14, 2026 02:08
aee558f to
0e6ebe5
Compare
Owner
Author
|
Reviewed and posted upstream as 0xShug0#546. Closing this staging PR. Two things changed during review here:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fork-side draft for review before it goes upstream. Addresses 0xShug0#545.
dist/index.htmlis committed and embedded into the server binary at configure time (CMakeLists.txt:2226) — that is what lets the project build a working server without a JavaScript toolchain. But it is not reproducible: SvelteKit defaultskit.version.nametoDate.now()and derives the embedded__sveltekit_<id>from it.__sveltekit_1ia7gsf/bb3f9c3c…__sveltekit_1xqgjp8/6cd91a94…__sveltekit_1vzi1g/65bed68b…__sveltekit_1vzi1g/65bed68b…So any two branches that rebuild the web UI conflict in that file regardless of whether their source overlaps — which is exactly what happened to 0xShug0#539.
Confirmed the id is a pure function of this setting by building with the exact timestamp the committed bundle carries (
1789348261391): the derived id comes back as__sveltekit_191iq4z, matching the committed one.Uses the package version rather than a constant: stable for a given source tree, so the bundle reproduces, but it changes when the version is bumped, so SvelteKit's "app has been updated" check keeps working across releases. A hard-coded string would have disabled that silently.
Why
dist/index.htmlis not rebuilt hereReview found that rebuilding it would smuggle in an unrelated change. With identical version strings our build still differs from the committed bundle — different Vite content hashes and ~80 bytes of chunk content — while our installed dependencies match
package-lock.jsonexactly:So the committed bundle is stale with respect to the lockfile, and replacing it is a separate decision from this one. Left alone; the first rebuild after this lands makes
distreproducible from then on.🤖 Generated with Claude Code
https://claude.ai/code/session_01EkxqpYvUbjCpRDnFiNiVfx