Skip to content

ENH: Build, test, and wrap external modules against an installed ITK - #6891

Open
hjmjohnson wants to merge 5 commits into
InsightSoftwareConsortium:mainfrom
hjmjohnson:wrapping-install-optin
Open

hjmjohnson wants to merge 5 commits into
InsightSoftwareConsortium:mainfrom
hjmjohnson:wrapping-install-optin

Conversation

@hjmjohnson

@hjmjohnson hjmjohnson commented Sep 22, 2026 •

Copy link
Copy Markdown
Member

Let an external (remote) module be configured, built, and Python-wrapped against an installed ITK, not only a build tree. Opt-in via ITK_INSTALL_WRAPPING_DEVELOPMENT_FILES (default OFF, tens of MB). Verified by building and wrapping 16 remote modules against an installed ITK 6 tree, and by configuring, building, and running module test suites (BUILD_TESTING=ON) against it; every fix the modules themselves needed is already in review upstream.

Why this matters. Today a remote module can only be wrapped from an ITK build tree. That is the reason the ITK Python wheel cache (ITKPythonBuilds) ships whole build trees, and it is why nobody can wrap a module against the ITK a distribution or a cmake --install provides. With this PR an installed prefix is a complete wrapping SDK, which is the precondition for a smaller wheel cache and for remote-module wheels built from an installed ITK.

Remote-module PRs needed for the verification below (GitHub shows each one's merge state):

None of them depends on this PR, and this PR does not depend on them: they are ITK 6 API updates and wrap files that never worked, needed to build the modules at all.

What was broken (five commits)
  1. The module build files are not installed. An external module starts with include(ITKModuleExternal); that file and its transitive includes were absent from the install tree, so configuring a module against an installed ITK failed at the first line. Installs the closure of that entry point, plus BuildHeaderTest.py, which ITKModuleHeaderTest.cmake runs by a path that only resolves in a source tree (about 115 KB).
  2. The wrapping infrastructure and SWIG type indices exist only in the build tree. ITK_INSTALL_WRAPPING_DEVELOPMENT_FILES=ON installs both under the package directory, where ITK_CMAKE_DIR and ITK_DIR already point for an installed consumer; the generated indices carry no absolute paths and relocate as-is. Also repairs ITKModuleExternal.cmake's install-tree branch, which read EXISTS"${ITK_CMAKE_DIR}/..." with no space and therefore never evaluated true.
  3. castxml could not find ITK's headers. From a build tree the dependency targets supply ITK's include directories; from an install tree they supply nothing, so castxml failed with 'itkCommand.h' file not found on an installed header. Adds ITK_INCLUDE_DIRS, which ITKConfig.cmake sets in both trees. It is not empty in a build tree, where it accumulates the include directories of every module find_package requested, so the castxml response file widens there too.
  4. The wrapping library inherits UseITK's IO factory register manager. UseITK adds ITK_<FACTORY>_FACTORY_REGISTER_MANAGER to every target in a consumer tree. A wrapped class that includes itkImageFileReader.h then references the register function of every IO factory the installed ITK enables, and the wrapping library, which links only its module's dependencies, fails to load (25 unresolved symbols in LesionSizingToolkit). Python registers factories at import (itk/support/base.py), so ITKModuleExternal.cmake sets an ITK-private ITK_WRAPPING_NO_FACTORY_REGISTER_MANAGER before adding the wrapping directory, and itk_wrap_module removes the definitions there with remove_definitions. Only the wrapping directory is affected: a module's C++ library, tests, and examples keep automatic factory registration (verified by symbol inspection of a module example executable, which still carries all 25 register references and passes its IO tests).
  5. Module tests cannot be configured or built from an install tree. With BUILD_TESTING=ON, ITK_USE_KWSTYLE defaults on and, with no KWStyle found, the module includes Utilities/KWStyle/BuildKWStyle.cmake to build one, which exists only in a source tree, so configure fails. The <Module>InDoxygenGroup test ran Utilities/Doxygen/mcdoc.py by a source-relative path. The header test and the test driver both pick up the register manager from item 4 and linked too few IO libraries (the test driver's ITKTestKernel covers the default IO modules, not the ones a maximal install enables). KWStyle is built only when the source tree provides it, mcdoc.py is installed and resolved like BuildHeaderTest.py, and both test executables link ITK_LIBRARIES for external modules.
Verification: 16 remote modules against an installed ITK 6

ITK at upstream main plus this branch (re-verified 2026-09-23 at the current head), configured with ITK_WRAP_PYTHON=ON, ITK_INSTALL_WRAPPING_DEVELOPMENT_FILES=ON, and every EXCLUDE_FROM_DEFAULT module enabled that builds (57 modules ON, 0 failed targets), then ninja install. Each remote module was configured with ITK_DIR=<prefix>/lib/cmake/ITK-6.0, ITK_WRAP_PYTHON=ON, built, and its Python module built; LesionSizingToolkit was additionally installed and exercised from Python.

Result Modules
Pass unchanged VkFFTBackend, Cleaver, BioCell, SkullStrip, SimpleITKFilters, HASI
Pass at current upstream Shape (SlicerSALT main), TractographyTRX (tee-ar-ex/ITKTractographyTRX#30)
Pass with ITK 6 fixes now in review the eight modules listed below
Out of scope Impact (needs LibTorch)

With BUILD_TESTING=ON against the same prefix, four modules configure, build their header test and test driver, and run: Dissolve 2/2, LesionSizingToolkit 42/42, RingArtifact 1/2, Ultrasound 37/48. All four resolve the header-test script to <prefix>/lib/cmake/ITK-6.0/BuildHeaderTest.py, the install-tree branch. The remaining failures are module-side and tracked on the module PRs: RingArtifact's baseline is FFT-backend dependent, Ultrasound has one truncated data object and ten Python tests blocked by the PYTHONPATH limitation below. Running any of them needs #6894 or DYLD_LIBRARY_PATH.

Companion ITK PRs:

None of the module-side fixes is specific to an installed ITK: they are ITK 6 API changes (ITK_DISALLOW_COPY_AND_MOVE, override, the VNL FFT removal), wrap files that never worked, and examples that only compiled inside an ITK source tree. ITK's own build never includes UseITK.cmake, so in-tree wrapping is unchanged. Two things do change for an external module built against a build tree: its castxml response file gains ITK_INCLUDE_DIRS (item 3), and its wrapping directory no longer carries the IO factory register managers (item 4).

Known limitations, deliberately left for follow-up
  • Generated wrapping artifacts and an external module's executables are written into the prefix (<prefix>/lib/cmake/ITK-6.0/Wrapping/...) because the output directories come from the installed wrapping CMake. Modules therefore must be built serially against one prefix, and a read-only prefix will not work yet. Fixing this means routing the output directories through the consumer's binary dir; it is independent of the five changes here.
  • The installed itkTestDriver carries no install RPATH (ITK sets none), so on macOS it cannot load @rpath/libITKTestKernel from <prefix>/lib without DYLD_LIBRARY_PATH; every module test that runs through ITK_TEST_DRIVER aborts until it is set. Pre-existing and independent of this PR; fixed by BUG: Give the installed itkTestDriver an RPATH to ITK's libraries #6894.
  • Python tests of an external module (itk_python_add_test) do not run from an install tree: ITKModuleTest.cmake puts ${ITK_DIR}/Wrapping/Generators/Python first on PYTHONPATH, which in an install tree is the installed wrapping infrastructure (a partial itk package without ITK's own module configurations), not ITK's Python package. Same root cause as the output-directory limitation above, and the same follow-up.

@github-actions github-actions Bot added type:Infrastructure Infrastructure/ecosystem related changes, such as CMake or buildbots type:Enhancement Improvement of existing methods or implementation labels Sep 22, 2026
@hjmjohnson
hjmjohnson force-pushed the wrapping-install-optin branch from 38a9615 to 6cb9163 Compare September 22, 2026 17:24
@github-actions github-actions Bot added the area:Documentation Issues affecting the Documentation module label Sep 22, 2026
@hjmjohnson
hjmjohnson force-pushed the wrapping-install-optin branch from 6cb9163 to bcf8398 Compare September 22, 2026 17:33
@dzenanz

dzenanz commented Sep 22, 2026

Copy link
Copy Markdown
Member

same ITK_WRAP_* type options

Doesn't this get taken by the remote module from the ITK_DIR it is pointed at?

@dzenanz dzenanz left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for pushing this Hans.

Comment thread Documentation/docs/contributing/module_workflows.md Outdated
Comment thread CMake/ITKModuleExternal.cmake Outdated
Comment thread CMake/ITKModuleKWStyleTest.cmake Outdated
@hjmjohnson

Copy link
Copy Markdown
Member Author

Two force-pushes just now, on purpose: 72b3385da07..0e529fa590b is review fixes only (same base, so the compare link shows only the patch changes), and 0e529fa590b..5346910ab27 is a plain rebase onto main with no content change.

@blowekamp

Copy link
Copy Markdown
Member

TANGENTAL COMMENT: Keeping ITK wrapping part of the ITK build process is useful for development and maintanance. However, some packaging seniors being able to build ITK Wrapping against an installed ITK is useful. For example in the condo-forge infrastructire. IMHO it would be a good feature to add to the wrapping directory.

@hjmjohnson
hjmjohnson marked this pull request as ready for review September 23, 2026 15:55
@hjmjohnson

Copy link
Copy Markdown
Member Author

TANGENTAL COMMENT: Keeping ITK wrapping part of the ITK build process is useful for development and maintanance. However, some packaging seniors being able to build ITK Wrapping against an installed ITK is useful. For example in the condo-forge infrastructire. IMHO it would be a good feature to add to the wrapping directory.

The conda-forge infrastructure for separate packaging was the motivation for this as a first step. There is followup work that would be needed for completing this work, and hopefully we will find funding to support that larger effort :).

@greptile-apps

greptile-apps Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

Safe to merge; no blocking issues remain.

Summary

This draft makes installed ITK packages usable by external modules for configuration, testing, and optional Python wrapping development. It installs the required CMake and wrapping support files, resolves build-tree versus install-tree paths, and documents installed-package consumption.

Reviews (2) · Last reviewed commit: "COMP: Build external module tests agains..."

Comment thread Wrapping/CMakeLists.txt Outdated
@greptile-apps

This comment has been minimized.

An external module starts with include(ITKModuleExternal), but that file
and the ones it reaches are not installed, so configuring a module against
an installed ITK fails immediately.

Adds the transitive closure of that entry point over ITK-owned CMake files,
the templates they configure, and BuildHeaderTest.py, which the header test
runs by a path that only resolves in a source tree. About 115 KB.
The wrapping infrastructure and the generated SWIG type indices exist only
in a build tree, so an installed ITK cannot wrap an external module.

ITK_INSTALL_WRAPPING_DEVELOPMENT_FILES=ON installs both under the package
directory, where ITK_CMAKE_DIR and ITK_DIR already point for an installed
consumer. It defaults to OFF because the indices are tens of megabytes.

Also repairs the install-tree branch in ITKModuleExternal.cmake: it read
EXISTS"${ITK_CMAKE_DIR}/..." with no space, which CMake evaluates as false
even when the file is present, so the branch was unreachable.
Wrapping an external module against an installed ITK failed in castxml with
"'itkCommand.h' file not found" although the header was installed: the
generated response file carried only the module's own include directories.
In a build tree the dependency targets supply ITK's; from an install tree
they contribute nothing.

ITKConfig.cmake sets ITK_INCLUDE_DIRS in both trees, so it covers the
installed case. It is not empty in a build tree: it accumulates the include
directories of every module find_package requested, so the response file
widens there too.
A consumer tree that includes UseITK gets ITK_<FACTORY>_FACTORY_REGISTER_MANAGER
on every target, including an external module's Python wrapping library.
Wrapped classes that include itkImageFileReader.h then reference the
register function of every IO factory the installed ITK enables, and the
wrapping library, which links only its module's dependencies, fails to
load with unresolved symbols.

Python registers the factories at import, so ITKModuleExternal asks the
wrapping directory to drop them and itk_wrap_module removes the ones
ITK_FACTORY_LIST names. The request uses a variable only itk_wrap_module
reads, so a scope added after the wrapping directory, such as a module's
examples, cannot lose its factory registration.

ITK's own build never includes UseITK, so in-tree wrapping is unchanged,
and a module's library, tests, and examples keep their managers.
With BUILD_TESTING=ON, ITK_USE_KWSTYLE defaults on, and when no KWStyle is
found the module build includes Utilities/KWStyle/BuildKWStyle.cmake to
build one, which exists only in a source tree; an external module then
fails to configure against an installed ITK. Build KWStyle only when the
source tree provides it, so an install-tree consumer falls through to the
existing branch that turns ITK_USE_KWSTYLE off.

The <Module>InDoxygenGroup test ran Utilities/Doxygen/mcdoc.py by a
source-relative path. Install the script into the package directory and
resolve it the way BuildHeaderTest.py is resolved.

The header test and the test driver pick up UseITK's IO factory register
manager through itkImageFileReader.h, which references every IO factory the
ITK enables. The header test linked only the module, and the test driver
linked the module's test dependencies and ITKTestKernel, which covers the
default IO modules but not the ones an ITK built with extra IO modules
enables; both failed with unresolved symbols. Link ITK_LIBRARIES for
external modules, as any consumer executable must.
@hjmjohnson
hjmjohnson force-pushed the wrapping-install-optin branch 2 times, most recently from 4a20b1c to 79cb0cc Compare September 23, 2026 17:35
@hjmjohnson

Copy link
Copy Markdown
Member Author

Two force-pushes again, one concern each: 5346910ab27..4a20b1c6662 is a plain rebase onto main (all five commits content-identical), and 4a20b1c6662..79cb0cc5d77 condenses the five multi-line in-source comments this PR added to one line each, per Documentation/AI/prose-budget.md.

@hjmjohnson

Copy link
Copy Markdown
Member Author

@greptileai review this draft before I make it official

@hjmjohnson

Copy link
Copy Markdown
Member Author

ARMBUILD-x86_64-rosetta failed on an ExternalData fetch, not on code: the single build error CDash counted is CMake Error at CMake/ExternalData.cmake:1169, which aborted the build before the test drivers linked, hence the 2414 "Not Run" tests. The Linux build of the same tree reported 0 errors and 0 test failures, and this job passed on the previous head, whose only difference is in-source comment text. Re-running the job.

@hjmjohnson hjmjohnson changed the title ENH: Wrap external modules against an installed ITK ENH: Build, test, and wrap external modules against an installed ITK Sep 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:Documentation Issues affecting the Documentation module type:Enhancement Improvement of existing methods or implementation type:Infrastructure Infrastructure/ecosystem related changes, such as CMake or buildbots

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants