feat(heads): add multitask loss and sampling primitives#82
Merged
Conversation
Combines per-sample task losses using only the targets each sample marks available, weighting them by learned log variances. The primary emotion task keeps a weight floor so auxiliary objectives cannot suppress it. Groundwork only: nothing imports it yet outside its unit tests.
Gives each corpus mass proportional to the square root of its utterance count and each class inverse-square-root mass within its corpus, so large corpora and frequent classes stop dominating a batch. Long utterances draw a bounded, seeded window subset per epoch, keeping selection deterministic and replayable. Groundwork only: nothing imports it yet outside its unit tests.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Groundwork for masked multitask training and balanced batch construction. Both
modules are deliberately unwired — nothing in product code imports them yet,
so this carries no behavioural risk and can be reviewed purely on its own terms.
MaskedUncertaintyWeightedLoss(_internal/heads/multitask_loss.py)Combines per-sample task losses using only the targets a sample actually marks
available, weighted by learned log variances. Tasks with no available targets in
a batch are skipped rather than contributing zero. The primary emotion task gets
a weight floor (default
0.25) so auxiliary objectives cannot suppress it, andsupplying no available targets at all raises rather than silently returning zero.
Utterance sampling (
_internal/models/utterance_sampling.py)utterance_sampling_distribution— square-root corpus mass andinverse-square-root class mass within corpus, so large corpora and frequent
classes stop dominating. Probabilities are asserted to sum to one.
select_training_windows— bounded seeded window subset per epoch, keyed byseed:epoch:sample_id, so selection is deterministic and replayable.sampling_contributions— reports expected sample and duration contributionper corpus and class, for auditing a recipe before training on it.
Why land it unwired
These are reusable research primitives that the Torch trainer will need. Landing
them now keeps them reviewed, typed, and tested instead of drifting in a working
tree. They are covered by unit tests and can be deleted cheaply if the design
changes.
Verification
make lint,make type, andmake testpass. Pre-push gate passed.