Create FastNetUK utils script - #2370
Open
James Warner (jwarner8) wants to merge 26 commits into
Open
Conversation
Contributor
Total coverage: 93% (HTML report)Name Stmts Miss Branch BrPart Cover --------------------------------------------------------------------------------------------------- src/CSET/__init__.py 105 0 14 0 100% src/CSET/_common.py 156 0 54 0 100% src/CSET/cset_workflow/app/fetch_fcst/bin/fetch_data.py 117 28 26 0 78% src/CSET/cset_workflow/app/fetch_nimrod/bin/fetch_nimrod.py 81 8 28 11 83% src/CSET/cset_workflow/app/finish_website/bin/finish_website.py 79 2 8 2 95% src/CSET/cset_workflow/app/parbake_recipes/bin/parbake.py 29 0 8 0 100% src/CSET/cset_workflow/app/send_email/bin/send_email.py 25 0 4 0 100% src/CSET/cset_workflow/lib/python/jinja_utils.py 17 0 6 0 100% src/CSET/extract_workflow.py 103 1 26 1 98% src/CSET/graph.py 44 0 14 0 100% src/CSET/operators/__init__.py 89 0 26 0 100% src/CSET/operators/_atmospheric_constants.py 9 0 0 0 100% src/CSET/operators/_colormaps.py 249 3 72 4 98% src/CSET/operators/_stash_to_lfric.py 3 0 0 0 100% src/CSET/operators/_utils.py 190 8 72 6 95% src/CSET/operators/ageofair.py 142 7 64 5 94% src/CSET/operators/aggregate.py 77 1 22 1 98% src/CSET/operators/aviation.py 61 0 18 0 100% src/CSET/operators/collapse.py 155 8 72 3 93% src/CSET/operators/constraints.py 113 7 50 3 93% src/CSET/operators/convection.py 38 4 10 2 88% src/CSET/operators/ensembles.py 27 0 14 0 100% src/CSET/operators/feature.py 44 0 10 0 100% src/CSET/operators/filters.py 67 2 30 0 98% src/CSET/operators/fluxes.py 41 0 10 0 100% src/CSET/operators/humidity.py 135 0 52 0 100% src/CSET/operators/imageprocessing.py 57 0 16 0 100% src/CSET/operators/mesoscale.py 18 0 2 0 100% src/CSET/operators/misc.py 172 1 72 3 98% src/CSET/operators/plot.py 1120 157 412 71 83% src/CSET/operators/power_spectrum.py 98 3 30 3 95% src/CSET/operators/precipitation.py 204 2 92 2 99% src/CSET/operators/pressure.py 41 0 12 0 100% src/CSET/operators/read.py 432 21 184 16 94% src/CSET/operators/regrid.py 147 1 70 3 98% src/CSET/operators/scoreswrappers.py 208 16 58 6 91% src/CSET/operators/temperature.py 121 0 32 0 100% src/CSET/operators/transect.py 63 0 24 0 100% src/CSET/operators/wind.py 46 3 10 2 91% src/CSET/operators/write.py 15 0 6 0 100% src/CSET/recipes/__init__.py 104 0 28 0 100% src/CSET/sample_data/__init__.py 0 0 0 0 100% --------------------------------------------------------------------------------------------------- TOTAL 5042 283 1758 144 93% |
Contributor
Author
|
Produces well formatted iris cubes 
|
Contributor
Author
|
Additional requirements;
|
Contributor
Author
Data fixing working nicely now for CSET ingestion...
|
James Warner (jwarner8)
marked this pull request as ready for review
August 14, 2026 09:57
| ---------- | ||
| cube : iris.cube.Cube | ||
| Original unstructured source cube, used for fixing metadata. | ||
| grid::: |
Contributor
There was a problem hiding this comment.
Should there be a description of grid here?
| units="hours", | ||
| ) | ||
|
|
||
| # Start with coordinates of just forecast_period. |
Contributor
There was a problem hiding this comment.
Is the description/comment correct here?
|
|
||
| out_cube.attributes = cube.attributes.copy() | ||
|
|
||
| # Delete fill value attribute, as this tends to be np.float64(nan), which causes iris merge/concat issues. |
Contributor
There was a problem hiding this comment.
This assumes fill_value always exists. Could make the code safer in case it is removed in the future?
| out_cube.attributes = cube.attributes.copy() | ||
|
|
||
| # Delete fill value attribute, as this tends to be np.float64(nan), which causes iris merge/concat issues. | ||
| del out_cube.attributes["fill_value"] |
Contributor
There was a problem hiding this comment.
Suggested change
| del out_cube.attributes["fill_value"] | |
| if "fill_value" in out_cube.attributes: | |
| del out_cube.attributes["fill_value"] |
| # Delete fill value attribute, as this tends to be np.float64(nan), which causes iris merge/concat issues. | ||
| del out_cube.attributes["fill_value"] | ||
|
|
||
| # Some unit corrections for specific variables. |
Contributor
There was a problem hiding this comment.
I think the geopotential_height_at_pressure levels is not just a unit correction is it?
| Run processing on FastNetUK data. | ||
|
|
||
| Process produces CSET-ready netCDF files for loading. | ||
| """ |
Contributor
There was a problem hiding this comment.
Suggested change
| """ | |
| """ | |
| # Define and parse input and output path arguments. |
| cubes = iris.load(file) | ||
|
|
||
| fixed_cubes = iris.cube.CubeList() | ||
| # For each cube, where ndim > 1 (excluding latitude/longitude array), do regridding |
Contributor
There was a problem hiding this comment.
How are latitude and longitude excluded? If they are not explicitly excluded, won't cube.ndim always be >1?
| dim_coords_and_dims=[(time_coord, 0)], | ||
| ) | ||
|
|
||
| cube.rename(name) |
Contributor
There was a problem hiding this comment.
Is this line necessary? The previous command sets long_name=name.
Contributor
|
This all looks good and the README file is very informative. There are a few comments and (small) suggested code changes. The files in utils are not covered by the coverage report so not totally clear that every part of the code is covered by a test (although tests look sensible). Happy for it to be merged into main after suggestions/comments addressed. |
8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This will also make the code transportable for inclusion as a postprocessing task in the FastNetUK inference.
This is largely a rehash of #2106 but retaining that branch as is given it is being used for FastNetUK evaluation.
Note we are not regridding the data in this util, just reshaping to the UKV grid.
Microsoft CoPilot was used to help generate the tests and documentation.
Contribution checklist
Aim to have all relevant checks ticked off before merging. See the developer's guide for more detail.