Bundled native libraries collide with same-named libs from a MIKE engine installation on LD_LIBRARY_PATH — import mikeio segfaults (proposal: auditwheel-style isolation)

[jsmariegaard]

Summary

import mikeio (and therefore mikecore) segfaults in any Python process that has a MIKE installation's bin directory on LD_LIBRARY_PATH (e.g. after sourcing mikevars.sh). This makes it impossible to use mikeio in the same environment — or the same process — as an installed MIKE engine.

The root cause is in how the mikecore Linux wheel bundles its native libraries, and it is fixable entirely inside the wheel build with standard tooling (auditwheel / patchelf). We have verified a fix experimentally: with a patched mikecore, mikeio and a running MIKE engine coexist happily in a single process.

The problem, at a high level

The mikecore wheel ships its own copies of DHI and Intel native libraries (libeum.so, libufs.so, libpfs2004.so, libMzCart.so, libimf.so, libiomp5.so, libsvml.so, libirng.so, libintlc.so.5, …), built around 2015. A MIKE engine installation ships newer libraries with exactly the same names (2025 builds in current releases).

Because the two library sets share names, the dynamic linker mixes them, in two independent ways:

1. LD_LIBRARY_PATH hijack. mikecore's bundled libs carry RUNPATH=$ORIGIN. On Linux, RUNPATH is searched after LD_LIBRARY_PATH — so once a MIKE bin dir is on LD_LIBRARY_PATH, mikecore's 2015 libeum.so gets its dependencies (libimf, libiomp5, libpfs2004, …) resolved to the engine's 2025 copies. Result: import mikeio → SIGSEGV, even if no engine library was ever loaded by Python.
2. soname dedup. Even with (1) fixed, glibc loads at most one library per soname. Whichever libimf.so / libiomp5.so / … loads first is silently reused by the other party. With the engine loaded first, mikecore aborts with free(): invalid size; with mikecore first, the engine fails with e.g. libpetsc.so.3.22: undefined symbol: __svml_ldexp2.

Nothing about this is specific to one MIKE release — any MIKE version that ships same-named libraries will collide with mikecore's bundle. Conversely, once the bundle is properly isolated, mikecore is compatible with every MIKE version, past and future, because its libraries no longer share names with anything outside the wheel.

What mikecore needs, concretely

Make the bundled native libraries fully self-contained and collision-free — exactly what auditwheel repair does for every manylinux wheel on PyPI:

1. RPATH instead of RUNPATH. Set DT_RPATH=$ORIGIN on every bundled .so (patchelf --force-rpath --set-rpath '$ORIGIN'). RPATH is searched before LD_LIBRARY_PATH, so the bundle always resolves to itself regardless of the caller's environment.
2. Unique sonames. Give every bundled library a name that cannot collide with a MIKE installation — e.g. a version/hash suffix (libeum-mc2015.so, or auditwheel's libeum-a1b2c3d4.so style): rename the files, patchelf --set-soname accordingly, and patchelf --replace-needed old new in every bundled lib that depends on them. The three libraries mikecore's Python code dlopens by absolute path (libeum.so, libufs.so, libMzCart.so) can keep their filenames — only their sonames and NEEDED entries need the treatment.

The simplest implementation is to run auditwheel repair as part of the Linux wheel build — it performs both steps (and the manylinux compliance check) automatically. If the bundle layout makes auditwheel awkward, the equivalent manual patchelf recipe is ~20 lines; we have a working script and can share it.

Windows likely deserves an audit too (DLL search order / same-basename shadowing when a MIKE bin is on PATH), but we have only verified the Linux side; the Windows failure mode — if any — may differ.

Evidence

All experiments on Linux (WSL2, Ubuntu), mikecore from PyPI, a current MIKE engine installation, mikevars.sh + oneAPI setvars.sh sourced:

Scenario	Stock mikecore	Patched mikecore (RPATH + unique sonames)
import mikeio + read dfsu	SIGSEGV (exit 139)	✅ works
mikeio first, then load an engine library	SIGSEGV	✅ works
engine library first, then mikeio	abort, free(): invalid size	⚠️ still fails¹
engine simulation + mikeio reading its output mid-run, one process	impossible	✅ works — mikeio values match the engine's in-memory state on all wet elements

¹ This residual ordering constraint is on the caller's side, not mikecore's: loading an engine library with RTLD_GLOBAL puts its symbols in the global scope where they override anything loaded later. Importing mikeio before the engine avoids it.

Ask

Could the mikecore Linux wheel be built (or post-processed) with auditwheel-style isolation as described above? Happy to share the patchelf script, the test matrix, and a live demo, or to help test a candidate wheel against MIKE engine installations.

🤖 Drafted with Claude Code