fix: clamp speaker mix weights to [0,1] to keep the frame mix convex - #6
Merged
LiuYunPlayer merged 1 commit intoJul 26, 2026
Conversation
DiffSingerSpeakerMix.Create 对 mix:<suffix> 采样值只判 NaN、不钳量程。
轨本身声明为 Continuous(color, 0, 0, 1)、宿主 UI 落笔时也钳制,但数据层无硬契约
(锚点+默认值合成、跨引擎同 key 复用、手改工程皆可越界)——与 CombineVariance
早已记录的兜底理由相同。
负权重危害最大:它把 Σ 拉回 ≤1,令下方「Σ>1 才归一」的判据失效,于是既不归一、
默认 suffix 又补上 1-Σ>1,混出凸包外的 embedding(外插而非插值,落到训练分布之外):
default=Alice, Alice=[1,0] Bob=[0,1]
Bob=+0.5 -> [0.5, 0.5] 正常插值
Bob=-0.5 -> [1.5, -0.5] 修复前:越界
Bob=-2.0 -> [3.0, -2.0] 修复前:越界
正负混合 Bob=2 / Carol=-1(Σ 恰为 1、不触发归一)-> [-0.5, 1.5] 修复前:越界
同形态的 phoneme_mix 轨在 DiffSingerSynthesisSession 采样处本就 clamp 到 [0,1],
此处对齐,消除两条同类轨的行为不一致。
测试:
- 把零依赖的 DiffSingerSpeakerMix.cs 加入测试项目(无 SDK 亦可跑)。
- 新增 DiffSingerSpeakerMixTests:默认补权、超额归一、NaN 视作未编辑等既有行为,
加 6 个越界回归(负权重、超 1、正负混合、逐帧钳制、凸包内断言)。
已确认这 6 个在修复前失败、修复后通过;其余 38 个基线测试前后皆通过。
44 tests pass;插件/测试/冒烟工具 Release 均 0 warning 0 error。
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
DiffSingerSpeakerMix.Createaccumulates eachmix:<suffix>sample with only a NaN check — the value is never clamped to the track's declared range:The track is declared
Continuous(color, 0, 0, 1)and the host UI clamps on draw, but the data layer has no hard contract — the codebase already says exactly this inCombineVariance:Negative values are the damaging case. They pull Σ back down to ≤ 1, which defeats the
sum > 1guard below: the weights are not normalized, and the default suffix is topped up by1 - Σ > 1. The result leaves the convex hull — extrapolation instead of interpolation, i.e. an embedding outside the training distribution.Reproduced against this repo's
master(bcd2396), default speaker Alice,Alice=[1,0],Bob=[0,1],Carol=[.5,.5]:Bob=+0.5[0.500, 0.500]Bob=-0.5[1.500, -0.500]Bob=-2.0[3.000, -2.000]Bob=2, Carol=-1[-0.500, 1.500]The last row is the subtle one: Σ is exactly 1, so the normalization branch never fires.
Fix
Clamp the sampled value to
[0,1]at accumulation. The siblingphoneme_mixtrack — declared with the identicalContinuous(color, 0, 0, 1)shape — is already clamped at its sampling site inDiffSingerSynthesisSession:so this only brings the two same-shaped tracks into line.
In-range behaviour is bit-for-bit unchanged: values within
[0,1]are unaffected by the clamp, and the normalization / default top-up logic is untouched.Tests
DiffSingerSpeakerMix.csto the test project'sCompileitems (zero external dependencies, same treatment asDiffSingerFrames), so it still runs without the TuneLab SDK checked out.DiffSingerSpeakerMixTestscovering existing behaviour (default top-up, over-unity normalization, NaN-as-unedited, short track) plus 6 out-of-range regressions (negative ×3, above-one, mixed-sign, per-frame clamping). The invariants asserted are the ones that actually matter: per-frame weights all in[0,1]and summing to 1, and the resulting embedding staying within the per-dimension convex hull of the speaker embeddings.I stashed the fix and ran the suite first to confirm those 6 genuinely fail without it (the other 38 pass either way), then restored it for 44/44 — otherwise "weights sum to 1" is the kind of assertion that can silently hold no matter what.
Local run of this repo's CI sequence (plugin / tests / smoke tool, Release): 0 warnings, 0 errors.
Found while reviewing an unrelated feature branch in my fork; confirmed independent of it (
git diff master...<that branch> -- DiffSingerSpeakerMix.csis empty), so it's sent on its own.