ENH: Build, test, and wrap external modules against an installed ITK - #6891
hjmjohnson wants to merge 5 commits into
Conversation
38a9615 to
6cb9163
Compare
6cb9163 to
bcf8398
Compare
Doesn't this get taken by the remote module from the ITK_DIR it is pointed at? |
dzenanz
left a comment
There was a problem hiding this comment.
Thank you for pushing this Hans.
0e529fa to
5346910
Compare
|
Two force-pushes just now, on purpose: |
|
TANGENTAL COMMENT: Keeping ITK wrapping part of the ITK build process is useful for development and maintanance. However, some packaging seniors being able to build ITK Wrapping against an installed ITK is useful. For example in the condo-forge infrastructire. IMHO it would be a good feature to add to the wrapping directory. |
The conda-forge infrastructure for separate packaging was the motivation for this as a first step. There is followup work that would be needed for completing this work, and hopefully we will find funding to support that larger effort :). |
|
This comment has been minimized.
This comment has been minimized.
An external module starts with include(ITKModuleExternal), but that file and the ones it reaches are not installed, so configuring a module against an installed ITK fails immediately. Adds the transitive closure of that entry point over ITK-owned CMake files, the templates they configure, and BuildHeaderTest.py, which the header test runs by a path that only resolves in a source tree. About 115 KB.
The wrapping infrastructure and the generated SWIG type indices exist only
in a build tree, so an installed ITK cannot wrap an external module.
ITK_INSTALL_WRAPPING_DEVELOPMENT_FILES=ON installs both under the package
directory, where ITK_CMAKE_DIR and ITK_DIR already point for an installed
consumer. It defaults to OFF because the indices are tens of megabytes.
Also repairs the install-tree branch in ITKModuleExternal.cmake: it read
EXISTS"${ITK_CMAKE_DIR}/..." with no space, which CMake evaluates as false
even when the file is present, so the branch was unreachable.
Wrapping an external module against an installed ITK failed in castxml with "'itkCommand.h' file not found" although the header was installed: the generated response file carried only the module's own include directories. In a build tree the dependency targets supply ITK's; from an install tree they contribute nothing. ITKConfig.cmake sets ITK_INCLUDE_DIRS in both trees, so it covers the installed case. It is not empty in a build tree: it accumulates the include directories of every module find_package requested, so the response file widens there too.
A consumer tree that includes UseITK gets ITK_<FACTORY>_FACTORY_REGISTER_MANAGER on every target, including an external module's Python wrapping library. Wrapped classes that include itkImageFileReader.h then reference the register function of every IO factory the installed ITK enables, and the wrapping library, which links only its module's dependencies, fails to load with unresolved symbols. Python registers the factories at import, so ITKModuleExternal asks the wrapping directory to drop them and itk_wrap_module removes the ones ITK_FACTORY_LIST names. The request uses a variable only itk_wrap_module reads, so a scope added after the wrapping directory, such as a module's examples, cannot lose its factory registration. ITK's own build never includes UseITK, so in-tree wrapping is unchanged, and a module's library, tests, and examples keep their managers.
With BUILD_TESTING=ON, ITK_USE_KWSTYLE defaults on, and when no KWStyle is found the module build includes Utilities/KWStyle/BuildKWStyle.cmake to build one, which exists only in a source tree; an external module then fails to configure against an installed ITK. Build KWStyle only when the source tree provides it, so an install-tree consumer falls through to the existing branch that turns ITK_USE_KWSTYLE off. The <Module>InDoxygenGroup test ran Utilities/Doxygen/mcdoc.py by a source-relative path. Install the script into the package directory and resolve it the way BuildHeaderTest.py is resolved. The header test and the test driver pick up UseITK's IO factory register manager through itkImageFileReader.h, which references every IO factory the ITK enables. The header test linked only the module, and the test driver linked the module's test dependencies and ITKTestKernel, which covers the default IO modules but not the ones an ITK built with extra IO modules enables; both failed with unresolved symbols. Link ITK_LIBRARIES for external modules, as any consumer executable must.
4a20b1c to
79cb0cc
Compare
|
Two force-pushes again, one concern each: |
|
@greptileai review this draft before I make it official |
|
|
Let an external (remote) module be configured, built, and Python-wrapped against an installed ITK, not only a build tree. Opt-in via
ITK_INSTALL_WRAPPING_DEVELOPMENT_FILES(default OFF, tens of MB). Verified by building and wrapping 16 remote modules against an installed ITK 6 tree, and by configuring, building, and running module test suites (BUILD_TESTING=ON) against it; every fix the modules themselves needed is already in review upstream.Why this matters. Today a remote module can only be wrapped from an ITK build tree. That is the reason the ITK Python wheel cache (ITKPythonBuilds) ships whole build trees, and it is why nobody can wrap a module against the ITK a distribution or a
cmake --installprovides. With this PR an installed prefix is a complete wrapping SDK, which is the precondition for a smaller wheel cache and for remote-module wheels built from an installed ITK.Remote-module PRs needed for the verification below (GitHub shows each one's merge state):
None of them depends on this PR, and this PR does not depend on them: they are ITK 6 API updates and wrap files that never worked, needed to build the modules at all.
What was broken (five commits)
include(ITKModuleExternal); that file and its transitive includes were absent from the install tree, so configuring a module against an installed ITK failed at the first line. Installs the closure of that entry point, plusBuildHeaderTest.py, whichITKModuleHeaderTest.cmakeruns by a path that only resolves in a source tree (about 115 KB).ITK_INSTALL_WRAPPING_DEVELOPMENT_FILES=ONinstalls both under the package directory, whereITK_CMAKE_DIRandITK_DIRalready point for an installed consumer; the generated indices carry no absolute paths and relocate as-is. Also repairsITKModuleExternal.cmake's install-tree branch, which readEXISTS"${ITK_CMAKE_DIR}/..."with no space and therefore never evaluated true.'itkCommand.h' file not foundon an installed header. AddsITK_INCLUDE_DIRS, whichITKConfig.cmakesets in both trees. It is not empty in a build tree, where it accumulates the include directories of every modulefind_packagerequested, so the castxml response file widens there too.ITK_<FACTORY>_FACTORY_REGISTER_MANAGERto every target in a consumer tree. A wrapped class that includesitkImageFileReader.hthen references the register function of every IO factory the installed ITK enables, and the wrapping library, which links only its module's dependencies, fails to load (25 unresolved symbols in LesionSizingToolkit). Python registers factories at import (itk/support/base.py), soITKModuleExternal.cmakesets an ITK-privateITK_WRAPPING_NO_FACTORY_REGISTER_MANAGERbefore adding the wrapping directory, anditk_wrap_moduleremoves the definitions there withremove_definitions. Only the wrapping directory is affected: a module's C++ library, tests, and examples keep automatic factory registration (verified by symbol inspection of a module example executable, which still carries all 25 register references and passes its IO tests).BUILD_TESTING=ON,ITK_USE_KWSTYLEdefaults on and, with no KWStyle found, the module includesUtilities/KWStyle/BuildKWStyle.cmaketo build one, which exists only in a source tree, so configure fails. The<Module>InDoxygenGrouptest ranUtilities/Doxygen/mcdoc.pyby a source-relative path. The header test and the test driver both pick up the register manager from item 4 and linked too few IO libraries (the test driver'sITKTestKernelcovers the default IO modules, not the ones a maximal install enables). KWStyle is built only when the source tree provides it,mcdoc.pyis installed and resolved likeBuildHeaderTest.py, and both test executables linkITK_LIBRARIESfor external modules.Verification: 16 remote modules against an installed ITK 6
ITK at upstream
mainplus this branch (re-verified 2026-09-23 at the current head), configured withITK_WRAP_PYTHON=ON,ITK_INSTALL_WRAPPING_DEVELOPMENT_FILES=ON, and everyEXCLUDE_FROM_DEFAULTmodule enabled that builds (57 modules ON, 0 failed targets), thenninja install. Each remote module was configured withITK_DIR=<prefix>/lib/cmake/ITK-6.0,ITK_WRAP_PYTHON=ON, built, and its Python module built; LesionSizingToolkit was additionally installed and exercised from Python.main), TractographyTRX (tee-ar-ex/ITKTractographyTRX#30)With
BUILD_TESTING=ONagainst the same prefix, four modules configure, build their header test and test driver, and run: Dissolve 2/2, LesionSizingToolkit 42/42, RingArtifact 1/2, Ultrasound 37/48. All four resolve the header-test script to<prefix>/lib/cmake/ITK-6.0/BuildHeaderTest.py, the install-tree branch. The remaining failures are module-side and tracked on the module PRs: RingArtifact's baseline is FFT-backend dependent, Ultrasound has one truncated data object and ten Python tests blocked by thePYTHONPATHlimitation below. Running any of them needs #6894 orDYLD_LIBRARY_PATH.Companion ITK PRs:
itkTestDriverRPATH; needed to run module tests from an install tree)None of the module-side fixes is specific to an installed ITK: they are ITK 6 API changes (
ITK_DISALLOW_COPY_AND_MOVE,override, the VNL FFT removal), wrap files that never worked, and examples that only compiled inside an ITK source tree. ITK's own build never includesUseITK.cmake, so in-tree wrapping is unchanged. Two things do change for an external module built against a build tree: its castxml response file gainsITK_INCLUDE_DIRS(item 3), and its wrapping directory no longer carries the IO factory register managers (item 4).Known limitations, deliberately left for follow-up
<prefix>/lib/cmake/ITK-6.0/Wrapping/...) because the output directories come from the installed wrapping CMake. Modules therefore must be built serially against one prefix, and a read-only prefix will not work yet. Fixing this means routing the output directories through the consumer's binary dir; it is independent of the five changes here.itkTestDrivercarries no install RPATH (ITK sets none), so on macOS it cannot load@rpath/libITKTestKernelfrom<prefix>/libwithoutDYLD_LIBRARY_PATH; every module test that runs throughITK_TEST_DRIVERaborts until it is set. Pre-existing and independent of this PR; fixed by BUG: Give the installed itkTestDriver an RPATH to ITK's libraries #6894.itk_python_add_test) do not run from an install tree:ITKModuleTest.cmakeputs${ITK_DIR}/Wrapping/Generators/Pythonfirst onPYTHONPATH, which in an install tree is the installed wrapping infrastructure (a partialitkpackage without ITK's own module configurations), not ITK's Python package. Same root cause as the output-directory limitation above, and the same follow-up.