Skip to content

Make the CMake build path work on macOS (follow-up to #78) - #80

Open
jlivingston-Cipher wants to merge 2 commits into
AOMediaCodec:mainfrom
jlivingston-Cipher:macos-cmake-path
Open

Make the CMake build path work on macOS (follow-up to #78)#80
jlivingston-Cipher wants to merge 2 commits into
AOMediaCodec:mainfrom
jlivingston-Cipher:macos-cmake-path

Conversation

@jlivingston-Cipher

Copy link
Copy Markdown
Contributor

Follow-up to #78, which was merged this morning — thanks for taking those changes, and for the run_logged / cmake_install cleanups. They're an improvement on what was sent, and the clone() rewrite is measurably faster: dependency cloning went from ~4.5 min to 11 s on my machine. You wrote "if they don't work for your environment feel free to discuss or file another PR", so here is the environment half.

Scope: the optional CMake path only (build_cmake.sh + CMakeLists.txt, both new in #78). This is not a re-opening of #7/#8 — the Bazel build on macOS was fixed in 2024 and has had a macos-latest leg ever since. That leg doesn't cover the CMake path, which is currently linux-amd64-cmake only.

build_cmake.sh at 81517e1 does not build on macOS. Four causes, and three fail silently — which is the part I think is worth your time.

The one that matters: S2 cannot fail

ln -sf to a nonexistent target succeeds and returns 0. macOS has no /usr/include at all, so on a Mac the shim stage creates five dangling symlinks and reports success:

# S2 stage, verbatim from build_cmake.sh @ 81517e1, on macOS:
>>> exit code: 0
  DANGLING flac/include/FLAC                -> /usr/include/FLAC
  DANGLING expat/expat/lib/expat.h          -> /usr/include/expat.h
  DANGLING expat/expat/lib/expat_external.h -> /usr/include/expat_external.h
  DANGLING expat/expat/lib/expat_config.h   -> /usr/include/expat_config.h
  DANGLING opus/include/*.h                 -> /usr/include/opus/*.h
  symlinks created: 5 ; DANGLING: 5

Note the last one: for h in /usr/include/opus/*.h doesn't match, and with nullglob off the literal pattern passes through — so the script creates a symlink named *.h.

The failure then surfaces a few hundred lines later as a missing-header compile error, pointing at the wrong thing. The fix resolves each header before linking and re-checks the links after, so the stage fails where the problem actually is.

The other three

Cause Effect
/usr/include/eigen3 hardcoded in CMakeLists.txt (:74 obr_lib, :110 iamf_core) no such path on macOS; eigen is at $(brew --prefix)/include/eigen3
/usr/include hardcoded 5× in the S2 stage as above; expat lives in the Xcode CLT SDK
JOBS="${JOBS:-$(nproc 2>/dev/null || echo 2)}" no nproc on macOS ⇒ silent fallback to 2 jobs on an 8-core machine

The change

+77 / −8 across three files. It doesn't touch run_logged, cmake_install, or clone — your structure is preserved throughout.

  • find_path(EIGEN_INC NAMES Eigen/Core PATH_SUFFIXES eigen3 REQUIRED) and two call sites. I used find_path rather than find_package(Eigen3) because eigen here is header-only and consumed by include path, and find_package needs a config package that libeigen3-dev doesn't install on every Debian version — happy to switch if you'd rather have find_package.
  • A find_hdr() helper searching /usr/include, $(brew --prefix)/include, and $(xcrun --show-sdk-path)/usr/include, with an explicit S2 FAIL: + exit 1 when a header isn't found, plus a post-link dangling check.
  • JOBS falls back through nprocsysctl -n hw.ncpugetconf _NPROCESSORS_ONLN → 2.
  • A macOS line in the requirements comment block.

One deliberate Linux difference, and it's the only one

I ran the current and patched S2 stages side by side on Debian with libflac-dev libopus-dev libexpat1-dev installed and diffed the resulting shim trees. They are identical except for one link:

- shims/expat/expat/lib/expat_config.h -> /usr/include/expat_config.h

The patched version doesn't create it, because Debian ships that header at the multiarch path /usr/include/x86_64-linux-gnu/expat_config.h — so the current script's link is dangling on Debian too, which is what the 2>/dev/null || true is absorbing. linux-amd64-cmake is green on 81517e1 with it dangling, so nothing consumes it. I've left it out rather than teach find_hdr about multiarch, since that would be a larger change to Linux behaviour in service of a header nothing includes. Say the word if you'd rather it resolve properly.

Everything else on Linux resolves byte-for-byte as it does today.

Verification

Built from a clean checkout of 81517e1 + this patch on macOS / Apple silicon / Homebrew, into a fresh build root: 3 min 49 s, producing working arm64 encoder_main (10 M), decoder_main (6.9 M), probe_main (2.0 M).

The shim stage, three ways:

exit links dangling
81517e1 as merged 0 5 5
with this patch 0 9 0
with this patch, headers hidden 1S2 FAIL: FLAC headers not found in: …

The third row is the point: the stage now fails closed, where before it could not fail at all.

I also ran a two-point known-answer check on the resulting binaries, using ADM BW64 fixtures with and without a Dolby metadata chunk, at both profile versions — all four reproduced their expected verdicts (rc=0 + 1 .iamf / rc=5 + 0 .iamf). So the binaries work, not merely compile.

The CI leg

#8 was closed with "Let's add a macos-latest target to the GitHub Actions CI before closing this bug", and in #7 it was noted that a Mac test platform was unavailable at the time. Both still apply here, so I've added a macos-arm64-cmake job to cmake.yml mirroring the Linux one — Homebrew deps instead of apt, same smoke test. Without it this patch is just my word, and the path will drift again the next time someone touches it.

I added it as a second job rather than converting the existing one to a matrix, deliberately: a matrix would rename linux-amd64-cmake to something like cmake (ubuntu-latest) and silently invalidate any branch-protection rule referencing the old name. That costs some duplication in the smoke-test block — happy to switch to a matrix if you'd prefer it and the rename is safe on your side.

If you'd rather keep cmake.yml single-platform, say so and I'll drop that file — the portability fixes should stand on their own.

build_cmake.sh at 81517e1 does not build on macOS. Four causes, three of
which fail silently:

  * ln -sf to a nonexistent target succeeds, so the S2 shim stage creates
    dangling symlinks and reports success; the error surfaces hundreds of
    lines later as a missing-header compile failure. macOS has no
    /usr/include at all, so all five links dangle.
  * /usr/include/eigen3 is hardcoded at two sites in CMakeLists.txt.
  * /usr/include is hardcoded five times in the S2 stage; expat ships in
    the Xcode Command Line Tools SDK.
  * No nproc on macOS, so JOBS silently falls back to 2.

Resolve each system header before linking, fail closed when one is
missing, discover eigen with find_path, and fall back through
sysctl/getconf for the job count. Adds a macos-arm64-cmake CI job so the
path stays honest.

Linux behaviour is unchanged except that expat/lib/expat_config.h is no
longer created: Debian ships that header at a multiarch path, so the
existing link dangles there too and nothing consumes it.

Verified by a clean build on macOS arm64 producing working
encoder_main / decoder_main / probe_main.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@jwcullen

jwcullen commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

I added it as a second job rather than converting the existing one to a matrix, deliberately: a matrix would rename linux-amd64-cmake to something like cmake (ubuntu-latest) and silently invalidate any branch-protection rule referencing the old name. That costs some duplication in the smoke-test block — happy to switch to a matrix if you'd prefer it and the rename is safe on your side.

Yes there is a lot of duplication in the building and smoke-test block. It would be better to use a patttern that combines them. The other GitHub action configurations are split between a workflows and actions directory, which allows us to add other OS flavors over time.

@jlivingston-Cipher

Copy link
Copy Markdown
Contributor Author

Thanks — there was some room for interpretation regarding your preference for preserving the current pattern or going with something which would be a change to the current workflows.

Claude responded to my guidance wrt the need for clarification thusly: the workflows/actions split is a better fit than the matrix I offered, and it keeps the job names stable, so the branch-protection concern in my note above goes away rather than needing a decision.

I've prepared it: a composite .github/actions/iamf-tools-cmake-builder with a platform input, and cmake.yml reduced to two job blocks that consume it. The install step is the only part that branches; the build and smoke-test steps are byte-identical to what is in the PR now. It follows iamf-tools-builder, so adding another OS is a six-line job block.

One scope question before I push it: the shared action necessarily covers linux-amd64-cmake as well, which is already on main. Happy to fold it into this PR, or to keep this PR to the macOS fix and send the refactor separately — whichever is easier to review. I'll hold until you say.

Let me know what you prefer and I will order the follow on actions for this PR accordingly.

@jlivingston-Cipher

Copy link
Copy Markdown
Contributor Author

Looking at the build failure now.

@jwcullen

jwcullen commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

One scope question before I push it: the shared action necessarily covers linux-amd64-cmake as well, which is already on main. Happy to fold it into this PR, or to keep this PR to the macOS fix and send the refactor separately — whichever is easier to review. I'll hold until you say.

I think it's OK to combine them.

You can ignore the macos-arm64 build failure, because it's unrelated to your PRs.

The linux and macOS jobs in cmake.yml carried the same build and smoke-test steps verbatim; only the package install differs between them. Move the shared steps into .github/actions/iamf-tools-cmake-builder with a platform input, following the existing iamf-tools-builder action, so cmake.yml is two job blocks and adding another OS is one more.

Job names and runs-on are unchanged, and the build and smoke-test steps are byte-identical to the ones they replace.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@jlivingston-Cipher

Copy link
Copy Markdown
Contributor Author

Pushed as 4962bb6. The shared steps now live in .github/actions/iamf-tools-cmake-builder with a platform input, and cmake.yml is two job blocks that consume it, following iamf-tools-builder. Only the package install branches — the build and smoke-test steps are byte-identical to the ones they replace, and linux-amd64-cmake / macos-arm64-cmake keep their names.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants