Make the CMake build path work on macOS (follow-up to #78) - #80
Make the CMake build path work on macOS (follow-up to #78)#80jlivingston-Cipher wants to merge 2 commits into
Conversation
build_cmake.sh at 81517e1 does not build on macOS. Four causes, three of which fail silently: * ln -sf to a nonexistent target succeeds, so the S2 shim stage creates dangling symlinks and reports success; the error surfaces hundreds of lines later as a missing-header compile failure. macOS has no /usr/include at all, so all five links dangle. * /usr/include/eigen3 is hardcoded at two sites in CMakeLists.txt. * /usr/include is hardcoded five times in the S2 stage; expat ships in the Xcode Command Line Tools SDK. * No nproc on macOS, so JOBS silently falls back to 2. Resolve each system header before linking, fail closed when one is missing, discover eigen with find_path, and fall back through sysctl/getconf for the job count. Adds a macos-arm64-cmake CI job so the path stays honest. Linux behaviour is unchanged except that expat/lib/expat_config.h is no longer created: Debian ships that header at a multiarch path, so the existing link dangles there too and nothing consumes it. Verified by a clean build on macOS arm64 producing working encoder_main / decoder_main / probe_main. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Yes there is a lot of duplication in the building and smoke-test block. It would be better to use a patttern that combines them. The other GitHub action configurations are split between a workflows and actions directory, which allows us to add other OS flavors over time. |
|
Thanks — there was some room for interpretation regarding your preference for preserving the current pattern or going with something which would be a change to the current workflows. Claude responded to my guidance wrt the need for clarification thusly: the workflows/actions split is a better fit than the matrix I offered, and it keeps the job names stable, so the branch-protection concern in my note above goes away rather than needing a decision. I've prepared it: a composite .github/actions/iamf-tools-cmake-builder with a platform input, and cmake.yml reduced to two job blocks that consume it. The install step is the only part that branches; the build and smoke-test steps are byte-identical to what is in the PR now. It follows iamf-tools-builder, so adding another OS is a six-line job block. One scope question before I push it: the shared action necessarily covers linux-amd64-cmake as well, which is already on main. Happy to fold it into this PR, or to keep this PR to the macOS fix and send the refactor separately — whichever is easier to review. I'll hold until you say. Let me know what you prefer and I will order the follow on actions for this PR accordingly. |
|
Looking at the build failure now. |
I think it's OK to combine them. You can ignore the macos-arm64 build failure, because it's unrelated to your PRs. |
The linux and macOS jobs in cmake.yml carried the same build and smoke-test steps verbatim; only the package install differs between them. Move the shared steps into .github/actions/iamf-tools-cmake-builder with a platform input, following the existing iamf-tools-builder action, so cmake.yml is two job blocks and adding another OS is one more. Job names and runs-on are unchanged, and the build and smoke-test steps are byte-identical to the ones they replace. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Pushed as |
Follow-up to #78, which was merged this morning — thanks for taking those changes, and for the
run_logged/cmake_installcleanups. They're an improvement on what was sent, and theclone()rewrite is measurably faster: dependency cloning went from ~4.5 min to 11 s on my machine. You wrote "if they don't work for your environment feel free to discuss or file another PR", so here is the environment half.Scope: the optional CMake path only (
build_cmake.sh+CMakeLists.txt, both new in #78). This is not a re-opening of #7/#8 — the Bazel build on macOS was fixed in 2024 and has had amacos-latestleg ever since. That leg doesn't cover the CMake path, which is currentlylinux-amd64-cmakeonly.build_cmake.shat81517e1does not build on macOS. Four causes, and three fail silently — which is the part I think is worth your time.The one that matters: S2 cannot fail
ln -sfto a nonexistent target succeeds and returns 0. macOS has no/usr/includeat all, so on a Mac the shim stage creates five dangling symlinks and reports success:Note the last one:
for h in /usr/include/opus/*.hdoesn't match, and withnullgloboff the literal pattern passes through — so the script creates a symlink named*.h.The failure then surfaces a few hundred lines later as a missing-header compile error, pointing at the wrong thing. The fix resolves each header before linking and re-checks the links after, so the stage fails where the problem actually is.
The other three
/usr/include/eigen3hardcoded inCMakeLists.txt(:74obr_lib,:110iamf_core)$(brew --prefix)/include/eigen3/usr/includehardcoded 5× in the S2 stageJOBS="${JOBS:-$(nproc 2>/dev/null || echo 2)}"nprocon macOS ⇒ silent fallback to 2 jobs on an 8-core machineThe change
+77 / −8across three files. It doesn't touchrun_logged,cmake_install, orclone— your structure is preserved throughout.find_path(EIGEN_INC NAMES Eigen/Core PATH_SUFFIXES eigen3 REQUIRED)and two call sites. I usedfind_pathrather thanfind_package(Eigen3)because eigen here is header-only and consumed by include path, andfind_packageneeds a config package thatlibeigen3-devdoesn't install on every Debian version — happy to switch if you'd rather havefind_package.find_hdr()helper searching/usr/include,$(brew --prefix)/include, and$(xcrun --show-sdk-path)/usr/include, with an explicitS2 FAIL:+exit 1when a header isn't found, plus a post-link dangling check.JOBSfalls back throughnproc→sysctl -n hw.ncpu→getconf _NPROCESSORS_ONLN→ 2.One deliberate Linux difference, and it's the only one
I ran the current and patched S2 stages side by side on Debian with
libflac-dev libopus-dev libexpat1-devinstalled and diffed the resulting shim trees. They are identical except for one link:The patched version doesn't create it, because Debian ships that header at the multiarch path
/usr/include/x86_64-linux-gnu/expat_config.h— so the current script's link is dangling on Debian too, which is what the2>/dev/null || trueis absorbing.linux-amd64-cmakeis green on81517e1with it dangling, so nothing consumes it. I've left it out rather than teachfind_hdrabout multiarch, since that would be a larger change to Linux behaviour in service of a header nothing includes. Say the word if you'd rather it resolve properly.Everything else on Linux resolves byte-for-byte as it does today.
Verification
Built from a clean checkout of
81517e1+ this patch on macOS / Apple silicon / Homebrew, into a fresh build root: 3 min 49 s, producing working arm64encoder_main(10 M),decoder_main(6.9 M),probe_main(2.0 M).The shim stage, three ways:
81517e1as mergedS2 FAIL: FLAC headers not found in: …The third row is the point: the stage now fails closed, where before it could not fail at all.
I also ran a two-point known-answer check on the resulting binaries, using ADM BW64 fixtures with and without a Dolby metadata chunk, at both profile versions — all four reproduced their expected verdicts (
rc=0+ 1.iamf/rc=5+ 0.iamf). So the binaries work, not merely compile.The CI leg
#8 was closed with "Let's add a
macos-latesttarget to the GitHub Actions CI before closing this bug", and in #7 it was noted that a Mac test platform was unavailable at the time. Both still apply here, so I've added amacos-arm64-cmakejob tocmake.ymlmirroring the Linux one — Homebrew deps instead of apt, same smoke test. Without it this patch is just my word, and the path will drift again the next time someone touches it.I added it as a second job rather than converting the existing one to a matrix, deliberately: a matrix would rename
linux-amd64-cmaketo something likecmake (ubuntu-latest)and silently invalidate any branch-protection rule referencing the old name. That costs some duplication in the smoke-test block — happy to switch to a matrix if you'd prefer it and the rename is safe on your side.If you'd rather keep
cmake.ymlsingle-platform, say so and I'll drop that file — the portability fixes should stand on their own.