com4FlowPy: optional Numba compute engine (bit-identical, ~30-45x faster)

[jmasseysykes]

Summary

Adds an optional Numba (@njit) compute engine for com4FlowPy's per-cell BFS,
selected via a new engine config key (python | numba; default python).
The BFS is expressed as a single @njit function over flat arrays; tiling,
multiprocessing, forest handling, variable parameters, I/O and merging are
unchanged. infra/back-calculation, previewMode and relId outputs fall back to the
Python engine, as does a missing numba install — which is declared as an
optional dependency (pip install avaframe[numba]), so nothing changes for
existing installs.

Results

• Bit-identical to the Python engine on single-tile domains — including
  against an independent BFW com4FlowPy run's stored zDelta / fpTravelAngleMax.
• ~30–45× faster at 8 cores (BFW 10 m: 302 s → 6.7 s; 5 m Connaught extreme:
  95 min → 193 s), with peak memory on par with the Python engine (measured via PSS).
• New pytest (avaframe/tests/test_com4FlowPyNumba.py) asserts engine equivalence
  across forest modules and variable alpha/Umax; it skips automatically if numba
  is not installed.
• On fine-resolution flat runout terrain a small set of cells differ from the
  Python engine — this is inherent numerical sensitivity of the routing (the
  Python engine's own result shifts ~175 cells under a physically meaningless
  ±1 µm DEM perturbation), not an engine defect; both are valid IEEE-754 solutions
  of the same model.

Full methodology, benchmarks and characterization:
avaframe/com4FlowPy/benchmark/REPORT.md.

Notes / questions for maintainers

• New identifiers follow the surrounding com4FlowPy snake_case; happy to switch
  to lowerCamelCase per the style guide if you prefer.
• The benchmark harness + report live under com4FlowPy/benchmark/; happy to
  relocate (e.g. under tests/ or docs/).
• A single-precision (float32) variant was evaluated and rejected (no speed or
  memory benefit, larger error at high resolution) — details in the report.
• AI assistance: Level 5 per the project policy — AI-assisted via Claude Code,
  with every line human-reviewed and validated against the Python engine and BFW's
  stored outputs.

PR Checklist

• I ran pytest locally without fails — com4FlowPy suite 12/12 (full suite via CI)
• I added/updated tests where needed
• I updated documentation where needed — config documented in com4FlowPyCfg.ini; happy to add .rst docs
• Commits squashed/rebased onto latest master — 2 commits; will rebase/squash to your preference before merge
• Standardtest run without difference — com1DFA/cython paths are untouched by this change

🤖 Generated with Claude Code

[fso42]

@jmasseysykes : Thanks for your PR!

3 things from my side (i.e. purely maintainer-wise, not com4FlowPy specific):

• We can do the squash/rebase if you don't want to deal with it.

• In general: you don't have to worry about the Standardtest stuff at all because we handle it. :-) (I should probably make that clear in the automatic PR list)

• I would be interested in your experiences with the numba package: How much hassle is it to install? Is it working reliably on different platforms? Have you got experiences using it in a python package prepared for pypi installation?

[jmasseysykes]

Thanks @fso42 — really glad this is useful, and happy to answer. Quick honest caveat up front: I've done all my own testing on Ubuntu, and I'm newer to the packaging/cross-platform side, so I've flagged below where I'm relaying how the integration is designed vs. what I've personally verified.

Happy for you to do the final squash/rebase — I'll keep the branch current in the meantime. And good to know re: Standardtest.

On numba:

• Install (verified): frictionless on Ubuntu. pip install avaframe[numba] pulls a prebuilt wheel — llvmlite bundles its own LLVM, so no system compiler and no extra system deps. Andi saw the same in a separate pixi env.
• Cross-platform (by design, not personally tested): numba ships manylinux/macOS/Windows wheels with LLVM bundled, so it's normally toolchain-free — but I've only run it on Ubuntu x86-64 myself. The integration is deliberately low-risk regardless: numba is an optional extra, and if its import or JIT ever fails the code falls back to the pure-Python engine (with a warning), so a missing or broken numba never blocks a run.
• PyPI packaging: declared under [project.optional-dependencies], so the default pip install avaframe is unchanged — only avaframe[numba] adds it. I haven't shipped numba in a published package before, so no first-hand wheel-resolution experience there; I'd defer to yours.

Per your AI policy, this is a Level 5 contribution (AI-assisted via Claude Code, with every line human-reviewed and validated against the Python engine and BFW's stored outputs).

[fso42]

@jmasseysykes : thanks for your reply (and sorry for my delayed reply, I was on vacation:-) ).

[ahuber-bfw]

Thx again @jmasseysykes for the PR and sorry for the delayed review.

From my side everything looks good com4FlowPy-wise.
I did a rebase and added some minor edits.

general remarks:

• speed-up in the order of 30-45 times compared to python engine could be confirmed for selected test-cases in addition to the ones provided.
• bit-wise matching of numba and python engines is also confirmed for 3 additional benchmarks (com4_avaArzlerAlmForestNumba, com4_avaArzlerAlmNullNumba, com4_avaParabolaChannelPaperNullNumba)
• additional pytests in tests/test_com4FlowPyNumb.py provided along with the original PR also confirm matching between results of both engines.

Notes to @fso42 and @PaulaSp3 regarding maintenance / further steps:

• I updated the documentation in develop.rst and moduleCom4FlowPy.rst (please check if this is ok for you as well) -- I referenced this PR in the docu, so anyone interested can have a more detailed look.
• at this point I would suggest keeping numba as optional dependency and keep engine = python as default setting in the cfg (@fso42: please check the pyproject.toml, if this is ok with you)
• however, I would suggest working toward shifting to numba engine as the default, once we have more experience with it.
• I left the additional benchmark directory in com4FlowPy untouched for now, as it does not affect any existing code --> However, code and content should be refactored/moved and integrated with existing AvaFrame benchmark tests/data in a next step (I would suggest a separate PR for this) --> see: #1332
• code styling, formatting and docstrings: can be done at a later stage together with other open issues --> #1333
• @fso42: if additional squashing of commits is required I can do this

[fso42]

Thanks @jmasseysykes , this is now merged into master