Skip to content

Audio support + DVR muxing rewrite - #139

Open
henkwiedig wants to merge 12 commits into
OpenIPC:masterfrom
henkwiedig:audio
Open

Audio support + DVR muxing rewrite#139
henkwiedig wants to merge 12 commits into
OpenIPC:masterfrom
henkwiedig:audio

Conversation

@henkwiedig

Copy link
Copy Markdown
Collaborator

Audio support + DVR muxing rewrite

image

Summary

Adds Opus audio to the ground station (live playback + recording) and, to make
recorded A/V stay in sync, replaces the minimp4 DVR muxer with in-pipeline
GStreamer splitmuxsink recorders
. Net −3,000 LOC (the vendored minimp4.h
and the old Dvr class are gone).

The air unit already muxes Opus into the same RTP flow as the video (split by
payload type); this branch consumes it.

Features

Live audio playback (opt-in)

  • --audio enables an Opus branch teed off the RTP flow:
    rtpopusdepay → opusdec → audioconvert → audioresample → volume → alsasink.
  • Output device selection--audio-device <card> or live from the OSD
    (System → Receiver → Output); accepts a /proc/asound/cards id or a full ALSA
    device string.
  • Volume — software volume element, 0–100%, live via OSD slider (works even
    on cards with no hardware mixer, e.g. HDMI).
  • Graceful fallback — an audio_stack_available() pre-flight checks the
    Opus/ALSA plugins and that the selected sink actually opens; if not, it silently
    drops to video-only so --audio can never take the video feed down. The OSD
    switch reports the effective state.

Air-side (VTX) audio menu

  • Drone → Camera → Audio (Enabled / Volume / Sample Rate), driving majestic.yaml
    .audio.* over SSH.

DVR rewrite (the big change)

The old raw DVR re-stamped video to a fixed framerate while audio kept the air
timestamps → A/V drift. Both DVR paths now record in-pipeline via
splitmuxsink + mp4mux, so video and Opus share one timebase:

  • Raw recorder — native video (no re-encode) + Opus, teed off the RTP flow.
    Perfectly synced.
  • Re-encode recorder — the MPP-re-encoded video is pushed through an appsrc
    and muxed with the same Opus into a second mp4. (Live mux, since the re-encoded
    video is on a different clock — small constant offset, no drift.)
  • Both honour --dvr-mode raw|reencode|both, size-splitting, and
    --dvr-sequenced-files naming (_raw/_reenc suffixes in BOTH mode).

Notable fix: dynamically adding two splitmuxsinks to a live pipeline
deadlocked both (async state change back-pressured the pipeline → 0-byte/missing
files). Fixed with async-handling=TRUE on each record bin + sink.

Cleanup

  • Deleted src/minimp4.h (~3,500 lines) and the entire Dvr class
    (dvr.cpp 559 → 60 lines) — dead once both recorders moved to splitmuxsink.
  • Removed the obsolete "Raw FPS" DVR menu option (raw path uses native
    timestamps).
  • --dvr-framerate is now deprecated — still accepted (won't break existing
    launch scripts) but logs a warning and is ignored.

Robustness

  • Disappearing sink handling — a USB headset unplugged mid-flight used to spin
    alsasink at 100% CPU and grow memory unbounded (error spam accumulating on an
    unwatched bus). The pull thread now drains the bus and, on an audio_sink error,
    rebuilds video-only — audio drops cleanly instead of taking video down with it.

Breaking changes / migration

  • DVR files are now standard mp4 (not fragmented). --dvr-framerate /
    PIXELPILOT_DVR_FRAMERATE are no longer needed (removed from the env file; the
    flag is kept as a deprecated no-op).
  • Reencode/BOTH mode requires the buildroot isomp4 (mp4mux) + multifile
    (splitmuxsink) good-plugins in addition to the audio-playback plugins.

Testing

  • Host-validated: splitmuxsink A/V muxing + size-splitting, the appsrc reencode
    path (h264 4.97s / opus 5.01s, in sync), BOTH-mode simultaneous start (5/5
    reliable after the async-handling fix), the sink-disconnect bus detection, and
    the filename/sequence logic.
  • Confirmed on-device: raw, reencode, and BOTH all record with synced audio; live
    audio via USB headset.

Fixes:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant