Skip to content

feat(heads): add multitask loss and sampling primitives#82

Merged
jsugg merged 2 commits into
mainfrom
feat/multitask-research-primitives
Jul 15, 2026
Merged

feat(heads): add multitask loss and sampling primitives#82
jsugg merged 2 commits into
mainfrom
feat/multitask-research-primitives

Conversation

@jsugg

@jsugg jsugg commented Jul 15, 2026

Copy link
Copy Markdown
Owner

Groundwork for masked multitask training and balanced batch construction. Both
modules are deliberately unwired — nothing in product code imports them yet,
so this carries no behavioural risk and can be reviewed purely on its own terms.

MaskedUncertaintyWeightedLoss (_internal/heads/multitask_loss.py)

Combines per-sample task losses using only the targets a sample actually marks
available, weighted by learned log variances. Tasks with no available targets in
a batch are skipped rather than contributing zero. The primary emotion task gets
a weight floor (default 0.25) so auxiliary objectives cannot suppress it, and
supplying no available targets at all raises rather than silently returning zero.

Utterance sampling (_internal/models/utterance_sampling.py)

  • utterance_sampling_distribution — square-root corpus mass and
    inverse-square-root class mass within corpus, so large corpora and frequent
    classes stop dominating. Probabilities are asserted to sum to one.
  • select_training_windows — bounded seeded window subset per epoch, keyed by
    seed:epoch:sample_id, so selection is deterministic and replayable.
  • sampling_contributions — reports expected sample and duration contribution
    per corpus and class, for auditing a recipe before training on it.

Why land it unwired

These are reusable research primitives that the Torch trainer will need. Landing
them now keeps them reviewed, typed, and tested instead of drifting in a working
tree. They are covered by unit tests and can be deleted cheaply if the design
changes.

Verification

make lint, make type, and make test pass. Pre-push gate passed.

jsugg added 2 commits July 15, 2026 18:02
Combines per-sample task losses using only the targets each sample marks
available, weighting them by learned log variances. The primary emotion task
keeps a weight floor so auxiliary objectives cannot suppress it.

Groundwork only: nothing imports it yet outside its unit tests.
Gives each corpus mass proportional to the square root of its utterance count
and each class inverse-square-root mass within its corpus, so large corpora and
frequent classes stop dominating a batch. Long utterances draw a bounded,
seeded window subset per epoch, keeping selection deterministic and replayable.

Groundwork only: nothing imports it yet outside its unit tests.
@jsugg
jsugg merged commit 254e070 into main Jul 15, 2026
12 checks passed
@jsugg
jsugg deleted the feat/multitask-research-primitives branch July 15, 2026 21:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant