Tag audio sources with a kind to drive Opus encoder settings

[kixelated]

Summary

• Add Audio.Kind = 'voice' | 'music' | 'auto' and reshape Audio.Source to { track, kind } so each source self-describes its content type.
• Map kind to Opus application + signal when configuring the AudioEncoder (voice → voip/voice, music → audio/music, auto → encoder defaults). Built-in sources tag themselves: Microphone = voice, Screen = music, File = auto.
• Opus-only knobs are kept out of the catalog and composed onto a separate AudioEncoderConfig in toEncoderConfig().
• Backwards compatible: Source = StreamTrack | { track, kind }. Bare-track callers normalize to kind: 'auto'.

Why

Microphone and screen capture have very different signal characteristics. Opus has dedicated tuning for each via the application mode (VoIP pre-emphasis + SILK bias vs. fidelity-focused), and we were ignoring it. Putting the hint on the source means the orchestrator doesn't have to remember to re-set it on every source switch.

What's not in here

• DTX is intentionally off. WebRTC also ships it off-by-default; enabling it interacts with the watch-side jitter buffer (silence gaps, comfort-noise repetition) and needs validation. Tracked in Investigate enabling Opus DTX for voice sources #1445.
• FEC / useinbandfec also left off pending a future realtime flag.

Test plan

• just check passes (verified locally)
• Manual: publish from microphone, confirm encoder is configured with application: voip (devtools console: "encoding audio …" debug log)
• Manual: publish from screen capture with audio, confirm application: audio
• Manual: file source publishes without throwing
• No catalog change in the wire format (opus knobs stay in encoder-only config)

🤖 Generated with Claude Code

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: ac7f88fa-d0e5-4836-9190-a1aee87b9e32

📥 Commits

Reviewing files that changed from the base of the PR and between 26c1fbc and 511f624.

📒 Files selected for processing (6)

• js/publish/src/audio/encoder.ts
• js/publish/src/audio/types.ts
• js/publish/src/source/file.ts
• js/publish/src/source/microphone.ts
• js/publish/src/source/screen.ts
• rs/moq-mux/src/export/mkv.rs

✅ Files skipped from review due to trivial changes (1)

• rs/moq-mux/src/export/mkv.rs

---

Walkthrough

This PR adds audio source kind configuration to the publish pipeline. A new Kind type—distinguishing "voice" (microphone), "music" (screen share), and "auto" (file)—is introduced alongside a SourceConfig container that pairs a MediaStreamTrack with its kind. The Source type widens to accept either a raw track or a SourceConfig, with a normalizeSource function ensuring consistent handling. The encoder uses this kind hint to conditionally configure codec-specific Opus settings. File, Microphone, and Screen sources wrap their audio tracks with the appropriate kind before emission.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title directly and clearly summarizes the main change: adding kind tags to audio sources to configure Opus encoder settings.
Description check	✅ Passed	The description provides comprehensive context on the changeset, explaining the API changes, encoder mapping logic, and built-in source tagging.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch claude/sleepy-kepler-28fe08

✨ Simplify code

• Create PR with simplified code
• Commit simplified code in branch claude/sleepy-kepler-28fe08

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ESLint

If the error stems from missing dependencies, add them to the package.json file. For unrecoverable errors (e.g., due to private dependencies), disable the tool in the CodeRabbit configuration.

ESLint skipped: no ESLint configuration detected in root package.json. To enable, add eslint to devDependencies.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@js/publish/src/audio/types.ts`:
- Around line 16-17: The normalizeSource function uses instanceof
MediaStreamTrack which can fail across realms; replace that check with a
structural type guard that detects a MediaStreamTrack by shape (e.g., ensure
source is an object and has MediaStreamTrack-like properties such as a string
"kind" and a "stop" function). In normalizeSource, test something like typeof
source === "object" && source !== null && typeof (source as any).kind ===
"string" && typeof (source as any).stop === "function", and when true return {
track: source as StreamTrack, kind: "auto" } otherwise return source as before;
update the guard around the symbols normalizeSource, MediaStreamTrack,
StreamTrack, Source, and SourceConfig.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 0c0038e0-752f-4536-b5f1-32334b7443d0

📥 Commits

Reviewing files that changed from the base of the PR and between 1d4678d and 26c1fbc.

📒 Files selected for processing (5)

• js/publish/src/audio/encoder.ts
• js/publish/src/audio/types.ts
• js/publish/src/source/file.ts
• js/publish/src/source/microphone.ts
• js/publish/src/source/screen.ts