Bring GA main up to date with main on anuga-community#296
Merged
Conversation
L1: Replace Logger (file-only) with TeeStream in prepare_data.py so
print() goes to both terminal and file. Remove the StringIO hack
from anuga_run_toml.py and run_model.py — plain print() now works.
L2: Change DefaultConsoleLogLevel from CRITICAL to INFO so log.info()
is visible on the terminal by default. Remove the clamping rule
that prevented file from logging more than console.
L3: Default log_filename=None (lazy). No file is created on import;
file logging only starts when set_logfile() is called. Eliminates
spurious timestamped log files in test directories.
L4: New set_logfile(path) function installs TeeStream on sys.stdout
and configures the logging.FileHandler simultaneously — one call
makes all print() and log.*() output go to both terminal and file.
Exported as anuga.set_logfile() and anuga.TeeStream in public API.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… output log.verbose(msg): file-only output for internal ANUGA chatter. Same level as log.debug() but named to guide contributors: use this instead of print() inside verbose=True code paths. set_logfile(..., verbose_to_screen=True): drops console threshold to DEBUG so verbose output also appears on screen — useful for debugging. setup_mesh.py: replace verbose print() calls with log.verbose() / log.info() and pass verbose=False to create_pmesh_from_regions and sequential_distribute_* so mesh construction no longer floods the terminal. Key stats (triangle count, extent) still appear on screen via log.info(). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_FileOnlyStream: file-only counterpart to TeeStream — writes to the log file handle without touching the terminal. file_only(): context manager that temporarily installs _FileOnlyStream as sys.stdout so any print() inside the block goes to the log file only. Restores original stdout on exit. If no logfile is active, output is discarded (StringIO). Exported as anuga.file_only(). setup_mesh.py: wrap create_pmesh_from_regions and sequential_distribute_* calls with file_only() and restore verbose=True so the full mesh construction output is captured in the log file but not shown on screen. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
New docs/source/setup_anuga_script/logging.rst covering: - Quick start with set_logfile() - Output destination table (print/info/verbose/debug) - file_only() context manager for capturing verbose internals - verbose_to_screen flag for debugging - Log level reference table - API autofunction blocks Add short docstrings to log.py shortcut functions so autofunction has content to render. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…eries When indices=[N] selects a single centroid, points[:,0] is a 1-element ndarray. pyproj dispatches to _transform_point (scalar path) which fails with numpy >= 1.25 when given a non-0d array. Fix: pass .tolist() to transformer.transform() so pyproj always takes the array code path, then convert the results back to numpy arrays for the affine transform multiplication. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Move sw_domain_cupy.py, sw_domain_cuda.py, cuda_anuga.cu and cupy_tests/ to archive/cupy_cuda/ to reduce clutter. These are not wired into meson or the public API; the active GPU work is the OpenMP-offloading path in sw_domain_openmp_ext.pyx. A README.md explains the contents and archive date. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
get_all_swwfiles() globs for 'datatest1*.sww' in CWD. Leftover datatest1<timestamp>.sww files from prior runs were being picked up, producing more rows than the 4 expected and causing IndexError. Fix: chdir into a fresh tempdir for the test body, set domain datadir to '.', and clean up the whole tmpdir on exit. Nullify self.sww before restoring CWD so tearDown doesn't try to remove an already-deleted file. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ks.py) Measures wall time and peak RSS for small/medium/large dam-break scenarios across multiprocessor_mode 0/1/2. Results saved as JSON for comparison across commits. compare_benchmarks.py shows ±% deltas for speed and memory. benchmarks/results/ is gitignored. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ethods) Merges scripts/benchmark_distribute.py and scripts/benchmark_distribute_mesh.py into benchmarks/distribute_benchmarks.py covering all four approaches: distribute(), collaborative(), distribute_basic_mesh(), dump()+load() Also moves run_benchmark_grid.py to benchmarks/ with updated paths and 4-column table parsing for the new distribute_basic_mesh() column. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…order When _neighbours had not yet been accessed (None), reorder() skipped the index-remapping step. Later, accessing mesh.neighbours triggered _build_neighbours() which copied _triangle_neighbours — a pre-reorder array whose indices no longer matched the new triangle numbering. This caused distribute_basic_mesh() to compute ~59% more ghost triangles than distribute() for the same mesh and scheme (measured: 9,002 vs 5,655 on a 1M-triangle metis/4-rank run). Fix: trigger _build_neighbours() at the start of reorder() so _neighbours is always populated and correctly remapped. Adds a regression test that verifies each neighbour pair shares exactly 2 nodes after a random reordering of a Basic_mesh. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Bumps the github-actions group with 1 update: [conda-incubator/setup-miniconda](https://github.com/conda-incubator/setup-miniconda). Updates `conda-incubator/setup-miniconda` from 3 to 4 - [Release notes](https://github.com/conda-incubator/setup-miniconda/releases) - [Changelog](https://github.com/conda-incubator/setup-miniconda/blob/main/CHANGELOG.md) - [Commits](conda-incubator/setup-miniconda@v3...v4) --- updated-dependencies: - dependency-name: conda-incubator/setup-miniconda dependency-version: '4' dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions ... Signed-off-by: dependabot[bot] <support@github.com>
…dabot/github_actions/github-actions-570f7a7cac Bump conda-incubator/setup-miniconda from 3 to 4 in the github-actions group
Bare `conda create python=X` does not include pip, causing `python -m pip install build` to fail with 'No module named pip'. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Build isolation creates a fresh venv that picks up the system pkg-config, which resolves to Ubuntu's system Python headers. On ubuntu-latest these headers require a split pyconfig.h that is not present, causing the Cython sanity check to fail with 'cannot compile programs'. Pre-install all build deps (meson-python, meson, ninja, cython, pybind11, numpy) into the conda env and pass --no-isolation to python -m build, so meson finds the conda env's Python headers. Mirrors the approach used in the conda-setup.yml test workflow. Also fix runner.os == 'linux' -> 'Linux' so the sdist step actually runs. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
conda's ninja package installs the binary but does not register a Python distribution, so python -m build --no-isolation fails its package check with 'Missing dependencies: ninja'. pip wheel --no-build-isolation uses whatever is on PATH (including conda-installed meson/ninja/cython) without a Python-package scan. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…elease Previously the workflow only triggered on release:published, so there was no way to verify wheels built successfully before tagging. New structure: - build-wheels: runs on every push/PR to main/develop (early feedback) and on release events. Builds one wheel per (python, os) combination using pip wheel --no-build-isolation to avoid the conda-ninja/pkg-config mismatch. Repairs Linux wheels with repairwheel. - build-sdist: builds source distribution once (Linux/py3.13). - publish: only runs on release:published, requires both build jobs to pass, then downloads all artifacts and uploads via trusted publishing. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
meson falls back to /usr/bin/pkg-config (system) when no pkg-config is specified in the native file. The system pkg-config finds Ubuntu's Python 3.12 headers which have a split pyconfig.h that doesn't exist, causing the Cython sanity check to fail. Two-pronged fix: - Add pkg-config to conda create so conda's pkg-config (aware of the conda env) shadows the system /usr/bin/pkg-config on PATH. - Export PKG_CONFIG_PATH=$CONDA_PREFIX/lib/pkgconfig:... before the build so meson finds the conda Python's .pc files first. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
python -m build --no-isolation checks that all build-system.requires entries are importable Python packages. conda's ninja binary is not a Python package, so the check fails with 'Missing dependencies: ninja'. For the sdist, drop --no-isolation so the build isolation venv installs PyPI's ninja (which does register as a package). PKG_CONFIG_PATH is still exported and is inherited by the isolated venv subprocess, so meson finds the conda env's Python headers rather than the broken system ones. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
After publishing to PyPI, upload all dist/* files to the GitHub Release using gh release upload. Requires contents:write permission on the publish job. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add vectorised epsg_to_ll() and ll_to_epsg() to redfearn.py using pyproj - Rewrite UTMtoLL and LLtoUTM as thin wrappers (public API preserved) - Replace utm.to_latlon() in quantity.py with epsg_to_ll via geo_reference.get_epsg() - Replace scalar UTMtoLL loop in geospatial_data.py with vectorised epsg_to_ll - Replace utm.from_latlon() in test_quantity.py with LLtoUTM - Remove unused `from affine import Affine` in tif2point_values.py - Move pyproj>=3.6 from optional [data] to required dependencies - Remove utm and affine from pyproject.toml Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds .devcontainer/devcontainer.json and a dedicated environment.yml so users can open ANUGA in a fully configured cloud environment by clicking 'Code → Open with Codespaces' on GitHub — no local install needed. Setup: - onCreateCommand: creates the conda env (cached by Codespaces prebuilds) - postCreateCommand: pip install --no-build-isolation -e . (needs source) - Activates anuga_env in every new terminal via postStartCommand - Default Python interpreter and Jupyter kernel point to anuga_env - Extensions: ms-python.python, ms-python.pylance, ms-toolsai.jupyter The devcontainer environment.yml is kept separate from the main environments/ files so it can evolve independently and stays clean (no utm, correct pyproj as hard dep, jupyter/ipykernel included). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Geo_reference with only a zone set defaults to hemisphere='undefined', so get_epsg() returns None. Use the isSouthHemisphere parameter to infer the UTM EPSG (32700+zone or 32600+zone) when no explicit EPSG is stored. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
main's redfearn.py was missing the e.item()/n.item() workaround that prevent pyproj's scalar path from calling float(ndim>0 array). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Collaborator
Author
|
@gareth-d-ga Hi Gareth, just tried to push our changes from anuga-community to the GA repository. I used to be able to do this, but it looks like I have lost permission. Can you point me to the correct person to OK this PR. One of the CI tests has failed, but someone just needs to rerun the failing test. Seems to be becoming quite common for random tests to fail because something has not downloaded. Cheers Steve |
get_gauges_from_file called gauge_get_from_file, which lives in abstract_2d_finite_volumes/gauge.py and was never imported into util, so every call raised NameError. Import it locally inside the function (gauge imports from util at module level, so a top-level import would be circular) and add a regression test that parses a small gauge file.
Collaborator
Author
|
@acarapetis Hi, I noticed that you are one of the 7 people who manage the Geoscience Australia github account. I used to be able to manage this repository An alternative might be to disable pull requests and the addition of issues and point people over to the Thanks for your help, @stoiver |
Member
|
@ga-autobots @Trivendra-cloud Hopefully someone can help Stephen out here? |
loc.strip(r'\n') only stripped newlines, leaving leading spaces from comma-separated fields (e.g. "308500, 6193000, gauge_a" → " gauge_a"). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…-297 Fix NameError in util.get_gauges_from_file
Previously only Linux wheels were repaired (auditwheel); macOS and Windows wheels were copied to dist/ unrepaired. The macOS build links conda's libomp via an @rpath into the build env, so `import anuga` from the published arm64 wheel fails on a plain pip install with "Library not loaded: @rpath/libomp.dylib". Run repairwheel for every platform — it dispatches to auditwheel (Linux), delocate (macOS, bundles libomp.dylib and rewrites the rpath) and delvewheel (Windows, bundles dependent DLLs) — so all published wheels are self-contained. Conda lib dirs that exist on the OS are passed via -l so dependent libs are found at repair time. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I dont seem to write access any more. So could you merge this update. thanks Steve