pjmedia: make VoiceProcessingIO other-audio ducking configurable on Apple platforms - #5178
Open
laconicman wants to merge 1 commit into
Open
pjmedia: make VoiceProcessingIO other-audio ducking configurable on Apple platforms#5178laconicman wants to merge 1 commit into
laconicman wants to merge 1 commit into
Conversation
…pple platforms The coreaudio backend opens the VoiceProcessingIO audio unit whenever echo cancellation is enabled. With the system defaults the OS applies a fixed duck to every other audio source for the whole duration of the call, so a user listening to music who takes a call has that music attenuated until hangup. macOS 14 / iOS 17 added kAUVoiceIOProperty_OtherAudioDuckingConfiguration, which exposes two independent axes: advanced ducking, which follows the voice activity of the chat participants instead of holding a constant duck, and the ducking level. Set the property after the audio unit is instantiated, only when EC is enabled, guarded by both an SDK check and a runtime @available check. The property is queried with AudioUnitGetPropertyInfo first, and any failure only logs a warning so audio is never broken by it. Two config.h macros control it: - PJMEDIA_AUDIO_DEV_COREAUDIO_ADVANCED_DUCKING (default 1) enables advanced ducking. Set it to 0 for the previous behaviour. - PJMEDIA_AUDIO_DEV_COREAUDIO_DUCKING_LEVEL (default kAUVoiceIOOtherAudioDuckingLevelDefault) sets the ducking depth, which is left at Apple's tuned value for a typical voice chat.
Member
Yes, I would prefer to leave the existing behaviour unchanged by setting the default as 0.
This also sounds good. Yes, please add it. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #5177
Motivation
The coreaudio backend opens the VoiceProcessingIO (VPIO) audio unit whenever echo cancellation
is enabled. VPIO ducks "other audio" — every audio stream on the system that is not the voice
chat — to improve the intelligibility of the call.
With the system defaults that duck is fixed and held for the entire call. A user listening
to music who accepts a call has that music attenuated from answer to hangup, even during the
long stretches when nobody is talking.
macOS 14 / iOS 17 added
kAUVoiceIOProperty_OtherAudioDuckingConfiguration, which exposes twoindependent controls over that behaviour:
mEnableAdvancedDucking— the style of ducking. When enabled, the duck follows the voiceactivity of the local and remote chat participants: more ducking while someone is talking,
less when nobody is. Apple compares it to FaceTime SharePlay, where media volume comes back
up between utterances.
mDuckingLevel— the amount of ducking:Default,Min,Mid,Max.pjproject currently sets neither, so it gets Apple's implicit default: advanced ducking off,
level
Default.Change
create_audio_unit()sets the property right after the audio unit is instantiated, only whenEC is enabled (i.e. only when the unit actually is VPIO). Gating follows the existing in-tree
style for macOS 14 / iOS 17 API in
pjmedia/src/pjmedia-videodev/darwin_dev.m: an SDK check(
__IPHONE_17_0/__MAC_14_0) plus a runtimeif (@available(macOS 14.0, iOS 17.0, *)).The property is queried with
AudioUnitGetPropertyInfofirst and skipped unless it is presentand writable. Every failure path logs a
PJ_LOG(4, …)warning and returns — configuring theduck must never be able to break audio.
Two
config.hmacros,#ifndef-guarded so they can be overridden fromconfig_site.h:PJMEDIA_AUDIO_DEV_COREAUDIO_ADVANCED_DUCKING1PJMEDIA_AUDIO_DEV_COREAUDIO_DUCKING_LEVEL0(kAUVoiceIOOtherAudioDuckingLevelDefault)No new
pjmedia_aud_dev_cap: this has no independent runtime axis — it is a refinement ofexisting EC/VPIO behaviour, not something an application would toggle per call through
pjmedia_aud_stream_set_cap()— and adding one would pushPJMEDIA_AUD_DEV_CAP_MAXpast itscurrent
16384.Why these defaults
Advanced ducking on. The complaint this addresses is precisely "other audio is attenuated
for the whole call". Following voice activity fixes exactly that, and it is the part of the
change that is unambiguously better for any VoIP application.
Depth left at Apple's default. Ducking depth is a much more opinionated axis, and real
deployments differ: a dispatch or contact-centre application may want a deep duck for
intelligibility, a consumer softphone a light one. Apple describes
Defaultas the tuned valuefor a typical voice chat, and it is the level pjproject already gets today. A library should
not pick a side there, so it is exposed as its own macro and left where it is. Keeping the
depth unchanged also holds the behavioural delta to one clearly-motivated axis.
Three things worth flagging
1. This changes existing behaviour. Anyone building against a macOS 14+ / iOS 17+ SDK with
EC enabled gets voice-activity-driven ducking instead of a constant duck. Setting
PJMEDIA_AUDIO_DEV_COREAUDIO_ADVANCED_DUCKINGto0restores the previous behaviour exactly:with both macros at
0the property is set to{advanced off, level Default}, whichAudioUnitProperties.hdocuments as identical to never setting the property at all — "If notset, the default ducking configuration is to disable advanced ducking, with a ducking level set
to
kAUVoiceIOOtherAudioDuckingLevelDefault." Happy to flip the default to opt-in (0) if youwould rather ship this as strictly additive.
2.
mDuckingLevelis meaningful whether or not advanced ducking is on. I checked ratherthan assumed. WWDC23 session 10235 states the struct "provides controls of two independent
aspects of ducking — the style of ducking, that is
mEnableAdvancedDucking, and the amount ofducking, that is
mDuckingLevel", and that "the two controls can be used independently".That is why they are two macros and not one enum.
3. Platform scope.
kAUVoiceIOProperty_OtherAudioDuckingConfigurationis declaredAPI_AVAILABLE(ios(17.0), macos(14.0)) API_UNAVAILABLE(watchos, tvos). That covers everyplatform this backend targets — macOS, iOS, and Mac Catalyst, which inherits the iOS
availability — and I verified Catalyst compiles. It does not cover tvOS or watchOS, where
the symbol is declared but unavailable and would fail to compile. The guard I used matches the
file's convention that
TARGET_OS_IPHONEmeans iOS, which is true for pjproject today (thereis no tvOS or watchOS build support anywhere in the tree). If you would rather harden it
against a future port,
&& !TARGET_OS_TV && !TARGET_OS_WATCHon theTARGET_OS_IPHONEarm isthe one-line change — say the word and I will add it.
Validation
makeinpjmedia/buildonaarch64-apple-darwin— clean, zero warnings, and the new codeis confirmed present in the built
coreaudio_dev.o.coreaudio_dev.mcompiles clean (-Wall -Wextra, no output) for: macOS at the hostdeployment target and at
-mmacosx-version-min=10.15; iOSarm64at deployment targets 15.0and 12.0 — i.e. both the compiled-in and the runtime-
@available-false paths; Mac Catalystarm64; withPJMEDIA_AUDIO_DEV_COREAUDIO_ADVANCED_DUCKING=0; withPJMEDIA_AUDIO_DEV_COREAUDIO_DUCKING_LEVEL=kAUVoiceIOOtherAudioDuckingLevelMin(confirmingthe macro accepts the Apple constants by name from
config_site.h); and withPJMEDIA_AUDIO_DEV_HAS_COREAUDIO=0.coreaudio_dev.m, andconfig.hgains only integer literals, so it still compiles with noApple headers in scope (verified by compiling
audiodev.cagainst it).and confirmed by ear that the music is no longer flattened for the call's duration and that
echo cancellation still works. That check is human-only and has not been done.