Skip to content

Add Cardington recipes #2128

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Simon Osborne (mo-sro) wants to merge 14 commits into
base: main
Choose a base branch
from
Open

Add Cardington recipes #2128

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
9242b0b
Add EIGHT Cardington recipe yaml files
mo-sro May 12, 2026
18c51ff
Change wind calculation operator to vector wind from list
mo-sro May 15, 2026
54462e2
Change to correct operator name wind.calculate_wind_vector
mo-sro May 15, 2026
aaeab54
Merge branch 'main' into cardington_recipes
mo-sro Jun 3, 2026
a5ddc0f
Merge branch 'main' into cardington_recipes
mo-sro Jun 10, 2026
bef83ea
Update Cardington variable names in YAML configuration
mo-sro Jun 12, 2026
7874d2c
Merge branch 'main' into cardington_recipes
mo-sro Jun 16, 2026
ed10559
Merge branch 'main' into cardington_recipes
mo-sro Jul 6, 2026
737416a
Update Cardington latent and sensible heat recipes
mo-sro Jul 6, 2026
5252560
Merge branch 'main' into cardington_recipes
mo-sro Jul 10, 2026
04d8c65
Modify wind gust recipe to better constrain load
mo-sro Jul 15, 2026
eab9104
Merge branch 'main' into cardington_recipes
mo-sro Jul 15, 2026
aa81619
Update all recipes for load constraints
mo-sro Jul 15, 2026
92e6e6c
Merge branch 'cardington_recipes' of github.com:/MetOffice/CSET into …
mo-sro Jul 15, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 63 additions & 0 deletions ...ET/recipes/surface_fields/cardington/cardington_latent_heat_single_point_time_series.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
category: Time series of latent heat flux at Cardington single point

@ukmo-huw-lewis ukmo-huw-lewis Jul 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
category: Time series of latent heat flux at Cardington single point
category: Time series at Cardington

Suggest category should be something relatively general that lots of outputs might sit in (e.g. generic_spatial_plots outputs all have category "Surface Spatial Plot".
The main aim of category is to offer some organisation/cataloguing of output plots.

Suggest you have same category wording for all Cardington time series recipes.

title: "Time series of $PLOT_VARNAME at lat_pt:52.10438, lon_pt:-0.42286 specific to Cardington gridpoint."

@ukmo-huw-lewis ukmo-huw-lewis Jul 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
title: "Time series of $PLOT_VARNAME at lat_pt:52.10438, lon_pt:-0.42286 specific to Cardington gridpoint."
title: "Time series of latent heat flux at Cardington [lat:52.10438, lon:-0.42286]"

Suggesting:

a) given current approach is a different recipe per variable type, be explicit on the variable name in recipe title. If we end up with more generic recipe template for all/many Cardington obs types, then suggest that time to introduce a $PLOT_VARNAME input. Otherwise sense these recipes require user to track lots of flavours of 'VARNAME'.

b) aim for shorter title as will improve plot readability / dimensions etc. Alternative suggested.

Implement similar changes across proposed recipes.

@ukmo-huw-lewis ukmo-huw-lewis Jul 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similarly, remove $PLOT_VARNAME from the description line below.

Alternative strategy would be to consolidate more of these recipes into generic Cardington time series recipe, but under impression that is not desirable (e.g. due to need for variable-specific processing).

description: "Plots a time series of $PLOT_VARNAME at a selected Cardington gridpoint."

@ukmo-huw-lewis ukmo-huw-lewis Jul 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here worth highlighting in the description that interpolation of gridded model data to observation location is based on nearest neighbour.


steps:
- operator: read.read_cubes
file_paths: $INPUT_PATHS
model_names:
- Cardington 30min

@ukmo-huw-lewis ukmo-huw-lewis Jul 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Cardington 30min
- OBS (30min)

Not wholly wedded to this, but for e.g. external publication, keeping legend titles as simple as possible may have value, and not obvious what "Cardington" is in plot label here.

Cardington as location is highlighted in the simpler plot title.

Consider implementing this approach across recipes.

- $MODEL_NAME
constraint: ['$VARNAME', '$UM_VARNAME']

@ukmo-huw-lewis ukmo-huw-lewis Jul 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
constraint: ['$VARNAME', '$UM_VARNAME']
constraint: ['$OBS_VARNAME', '$VARNAME']

Strongly suggest we need to keep "VARNAME" as consistent with general CSET use of "VARNAME" (typically standard_names, model output etc), and also that we should remove explicit referneces to 'UM' here.

Therefore instead, introduce $OBS_VARNAME to cater for any obs-specific naming conventions.

@ukmo-huw-lewis ukmo-huw-lewis Jul 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would need to make equivalent alternations further down in recipe where currently $VARNAME and $UM_VARNAME referenced.


- operator: filters.filter_multiple_cubes

Cardington:
operator: constraints.combine_constraints
varname_constraint:
operator: constraints.generate_var_constraint
varname: $VARNAME
cell_methods_constraint:
operator: constraints.generate_cell_methods_constraint
cell_methods: []
varname: $VARNAME
height_constraint:
operator: constraints.generate_attribute_constraint
attribute: 'height'
value: $HEIGHT

$MODEL_NAME:
operator: constraints.combine_constraints
varname_constraint:
operator: constraints.generate_var_constraint
varname: $UM_VARNAME
cell_methods_constraint:
operator: constraints.generate_cell_methods_constraint
cell_methods: []
varname: $UM_VARNAME
pressure_level_constraint:
operator: constraints.generate_level_constraint
levels: [ ]
coordinate: "pressure"
stash_constraint:
operator: constraints.generate_attribute_constraint
attribute: STASH

- operator: fluxes.latent_heat_units

@ukmo-huw-lewis ukmo-huw-lewis Jul 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider adding in call to filter fill_values (see example comment below) here for consistency with SH flux recipe.

- operator: misc.remove_attribute
attribute: [ 'time_coverage_end', 'time_coverage_start', 'valid_min', 'valid_max', 'date', 'date_created' ]

- operator: regrid.regrid_to_single_point
lat_pt: 52.10438
lon_pt: -0.42286
latlon_in_type: "realworld"
method: "Nearest"
boundary_margin: 0

# Make a single NetCDF with all the data inside it.
- operator: write.write_cube_to_nc
overwrite: True

# Plot the data.
- operator: plot.plot_line_series
45 changes: 45 additions & 0 deletions .../recipes/surface_fields/cardington/cardington_sensible_heat_single_point_time_series.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
category: Time series of sensible heat flux at Cardington single point

@ukmo-huw-lewis ukmo-huw-lewis Jul 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See detailed comments in latent_heat recipe for style/naming changes, to be applied consistently in relevant places to this recipe also.

title: "Time series of $PLOT_VARNAME at lat_pt:52.10438, lon_pt:-0.42286 specific to Cardington gridpoint."
description: "Plots a time series of $PLOT_VARNAME at a selected Cardington gridpoint."
steps:
- operator: read.read_cubes
file_paths: $INPUT_PATHS
model_names:
- Cardington 30min
- $MODEL_NAME

- operator: misc.mask_fill_values

@ukmo-huw-lewis ukmo-huw-lewis Jul 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After some trial/error, I was able to utilise existing CSET operator functionality to avoid introducing specific new code here as follows:

  - operator: filters.apply_mask
    mask:
       operator: filters.generate_mask
       condition: 'lt'
       value: 1.0e+10

This makes recipe slightly longer, but re-uses existing code. The logic is somewhat 'simple' as here just saying to preserve everything where data < 1e10 rather than setting specific values to NaN based on tolerances etc.

Would that approach be sufficient for most recipes (with some knowledge of 'gotcha' numbers).

My sense is the mask_fill_values is neat code, and maybe useful capability, but looking for simple implementation and avoiding too much new code where possible.


- operator: fluxes.sensible_heat_flux_from_covariance
HEIGHT: $HEIGHT
CARDINGTON_VARNAMES: $WT_VARNAMES

@ukmo-huw-lewis ukmo-huw-lewis Jul 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would want to see worked example
(e.g. cset_bake command) to understand input WT_VARNAMES etc.

This could be another example where if recipe is aiming to be bespoke to specific variable, coding known required varnames within recipe could be preferable to requiring users to be familiar with details of input needed.


- operator: filters.filter_multiple_cubes
Cardington:
operator: constraints.generate_var_constraint
varname: surface_upward_sensible_heat_flux

$MODEL_NAME:
operator: constraints.generate_var_constraint
varname: surface_upward_sensible_heat_flux

- operator: misc.remove_attribute
attribute:
- time_coverage_end
- time_coverage_start
- valid_min
- valid_max
- date
- date_created

- operator: regrid.regrid_to_single_point
lat_pt: 52.10438
lon_pt: -0.42286
latlon_in_type: "realworld"
method: "Nearest"
boundary_margin: 0

- operator: write.write_cube_to_nc
overwrite: True

- operator: plot.plot_line_series
64 changes: 64 additions & 0 deletions src/CSET/recipes/surface_fields/cardington/cardington_surface_pressure_time_series.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
category: Time series of pressure at Cardington single point

@ukmo-huw-lewis ukmo-huw-lewis Jul 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See detailed comments in previous recipes for areas to address on general approach, to be reflected in updates to this recipe also.

title: "Time series of $PLOT_VARNAME at lat_pt:52.10438, lon_pt:-0.42286 specific to Cardington gridpoint."
description: "Plots a time series of $PLOT_VARNAME at a selected Cardington gridpoint."

steps:
- operator: read.read_cubes
file_paths: $INPUT_PATHS
model_names:
- Cardington 05min
- Cardington 30min
- $MODEL_NAME
constraint: ['$VARNAME', '$UM_VARNAME']

- operator: filters.filter_multiple_cubes

Cardington:
operator: constraints.combine_constraints
varname_constraint:
operator: constraints.generate_var_constraint
varname: $VARNAME
cell_methods_constraint:
operator: constraints.generate_cell_methods_constraint
cell_methods: []
varname: $VARNAME


$MODEL_NAME:
operator: constraints.combine_constraints
varname_constraint:
operator: constraints.generate_var_constraint
varname: $UM_VARNAME
cell_methods_constraint:
operator: constraints.generate_cell_methods_constraint
cell_methods: []
varname: $UM_VARNAME
pressure_level_constraint:
operator: constraints.generate_level_constraint
levels: []
coordinate: "pressure"
stash_constraint:
operator: constraints.generate_attribute_constraint
attribute: STASH

- operator: misc.remove_attribute
attribute: [ 'time_coverage_end', 'time_coverage_start', 'valid_min', 'valid_max', 'date', 'date_created' ]

- operator: misc.convert_units
units: 'hPa'

- operator: regrid.regrid_to_single_point
lat_pt: 52.10438
lon_pt: -0.42286
latlon_in_type: "realworld"
method: "Nearest"
boundary_margin: 0

- operator: misc.mask_fill_values

@ukmo-huw-lewis ukmo-huw-lewis Jul 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See comment in sensible_heat recipe on proposal to adopt existing apply_mask and generate_mask to satisfy this requirement.

Reminder of comment to then apply consistently across Cardington recipes.


# Make a single NetCDF with all the data inside it.
- operator: write.write_cube_to_nc
overwrite: True

# Plot the data.
- operator: plot.plot_line_series
71 changes: 71 additions & 0 deletions src/CSET/recipes/surface_fields/cardington/cardington_surface_temperature_time_series.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
category: Time series of surface temperature at Cardington single point

@ukmo-huw-lewis ukmo-huw-lewis Jul 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See other comments for general style suggestions/requirements to be adopted consistently in this recipe.

title: "Time series of $PLOT_VARNAME at lat_pt:52.10438, lon_pt:-0.42286 specific to Cardington gridpoint."
description: "Plots a time series of $PLOT_VARNAME at a selected Cardington gridpoint."
steps:
# ------------------------------------------------------------

@ukmo-huw-lewis ukmo-huw-lewis Jul 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would remove, or simplify the docstrings from recipe file itself.

As above, avoid explicit reference to "UM" for example.

Suggest the call to read.read_cubes does not require specific docstrings in final version, so would remove.

# 1. Read data
# IMPORTANT:
# - Constrain the UM *at read time* so Iris only ever loads
# a single surface_temperature cube.
# ------------------------------------------------------------

- operator: read.read_cubes
file_paths: $INPUT_PATHS
model_names:
- Cardington 05min
- Cardington 30min
- $MODEL_NAME
constraint: ['$VARNAME', '$UM_VARNAME']

constraints:
$MODEL_NAME:
operator: constraints.combine_constraints
stash_constraint:
operator: constraints.generate_attribute_constraint
attribute: STASH
values:
- $UM_VARNAME
cell_methods_constraint:
operator: constraints.generate_cell_methods_constraint
cell_methods: [ 'maximum' ]

@ukmo-huw-lewis ukmo-huw-lewis Jul 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this recipe be updated to reference Tmax therefore?

Are there intended to be equivalents for Tmin, Tmean etc?

Could the required cell_method potentially be user-defined variable $CELL_METHOD if appetite/requirement to cover all options?

coord: 'time'
interval: '1 hour'
preprocessors:

@ukmo-huw-lewis ukmo-huw-lewis Jul 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Preprocessors is new to me.

Is it useful to adopt similar preprocessor step in previous recipes to remove unwanted attributes also?

This would seem to be neater approach than a 'random' call to these operators lower down in recipe.

Aim to adopt consistent approach where possible across the recipe set.

Alternatively, why are scalar_coords removed in this recipe, but seemingly not in other recipes? Is this still needed?

- operator: misc.remove_scalar_coords
coords:
- forecast_reference_time
- realization

# IMPORTANT: Global temperature-only prune.

@ukmo-huw-lewis ukmo-huw-lewis Jul 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this still valid/required if constrain by varname now done in read.read_cubes step?

# filter_multiple_cubes must have exactly ONE key here,
# otherwise non-temperature cubes will survive and break convert_units.
- operator: filters.filter_multiple_cubes
temperature:
operator: constraints.generate_var_constraint
varname: $VARNAME

- operator: misc.remove_attribute
attribute:
- time_coverage_end
- time_coverage_start
- valid_min
- valid_max
- date
- date_created

- operator: misc.convert_units
units: 'Celsius'

- operator: regrid.regrid_to_single_point
lat_pt: 52.10438
lon_pt: -0.42286
latlon_in_type: "realworld"
method: "Nearest"
boundary_margin: 0

- operator: misc.mask_fill_values

- operator: write.write_cube_to_nc
overwrite: True

- operator: plot.plot_line_series
52 changes: 34 additions & 18 deletions ...temperature_single_point_time_series.yaml → ...temperature_single_point_time_series.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,40 +1,54 @@
category: Time series of air temperature at Cardington single point
title: Time series of 'air_temperature' at lat_pt:52.10438, lon_pt:-0.42286 specific to Cardington gridpoint.
description: Plots a time series of the air temperature at a selected Cardington gridpoint.
title: "Time series of $PLOT_VARNAME at lat_pt:52.10438, lon_pt:-0.42286 specific to Cardington gridpoint."
description: "Plots a time series of $PLOT_VARNAME at a selected Cardington gridpoint."

steps:
- operator: read.read_cubes
file_paths: $INPUT_PATHS
model_names: ['Cardington','$MODEL_NAME']
constraint:
model_names:
- Cardington 05min
- Cardington 30min
- $MODEL_NAME
constraint: ['$VARNAME', '$UM_VARNAME']

- operator: filters.filter_multiple_cubes

Cardington:
operator: constraints.combine_constraints
varname_constraint:
operator: constraints.generate_var_constraint
varname: 'air_temperature'
varname: $VARNAME
cell_methods_constraint:
operator: constraints.generate_cell_methods_constraint
cell_methods: []
varname: 'air_temperature'
varname: $VARNAME
height_constraint:
operator: constraints.generate_attribute_constraint
attribute: 'height'
value: $HEIGHT

$MODEL_NAME:
operator: constraints.combine_constraints
varname_constraint:
operator: constraints.generate_var_constraint
varname: $UM_VARNAME
cell_methods_constraint:
operator: constraints.generate_cell_methods_constraint
cell_methods: []
varname: $UM_VARNAME
pressure_level_constraint:
operator: constraints.generate_level_constraint
levels: []
coordinate: "pressure"

- operator: misc.remove_attribute
attribute: ['time_coverage_end', 'time_coverage_start', 'valid_min', 'valid_max', 'date', 'date_created']

- operator: filters.filter_multiple_cubes
cardington:
stash_constraint:
operator: constraints.generate_attribute_constraint
attribute: 'height'
value: 2.0
model:
operator: constraints.generate_attribute_constraint
# This currently only works for UM. Maybe we need another operator to make this nicer?
attribute: STASH

- operator: misc.remove_attribute
attribute: [ 'time_coverage_end', 'time_coverage_start', 'valid_min', 'valid_max', 'date', 'date_created' ]

- operator: misc.convert_units
units: 'Celsius'
units: 'celsius'

- operator: regrid.regrid_to_single_point
lat_pt: 52.10438
Expand All @@ -43,6 +57,8 @@ steps:
method: "Nearest"
boundary_margin: 0

- operator: misc.mask_fill_values

# Make a single NetCDF with all the data inside it.
- operator: write.write_cube_to_nc
overwrite: True
Expand Down
52 changes: 52 additions & 0 deletions ...SET/recipes/surface_fields/cardington/cardington_visibility_single_point_time_series.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
category: Time series of visibility at Cardington single point
title: "Time series of $PLOTNAME at lat_pt:52.10438, lon_pt:-0.42286 specific to Cardington gridpoint."
description: "Plots a time series of $PLOTNAME at a selected Cardington gridpoint."

steps:
- operator: read.read_cubes
file_paths: $INPUT_PATHS
model_names:
- $CARDINGTON_05_LABEL
- $CARDINGTON_30_LABEL
- $MODEL_NAME

- operator: filters.filter_multiple_cubes
Cardington:
operator: constraints.generate_var_constraint
varname: $VARNAME

$MODEL_NAME:
operator: constraints.combine_constraints
varname_constraint:
operator: constraints.generate_var_constraint
varname: $UM_VARNAME
pressure_level_constraint:
operator: constraints.generate_level_constraint
levels: []
coordinate: "pressure"
stash_constraint:
operator: constraints.generate_attribute_constraint
attribute: STASH

- operator: misc.remove_attribute
attribute:
- time_coverage_end
- time_coverage_start
- valid_min
- valid_max
- date
- date_created

- operator: misc.convert_visibility_to_km

@ukmo-huw-lewis ukmo-huw-lewis Jul 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check if this required, or if
read._convert_cube_units_callback() in read.py doesn't already cover this? And if not, why not?

That function includes test of varnames for "visibility" and then converts units to km.


- operator: regrid.regrid_to_single_point
lat_pt: 52.10438
lon_pt: -0.42286
latlon_in_type: "realworld"
method: "Nearest"
boundary_margin: 0

- operator: write.write_cube_to_nc
overwrite: True

- operator: plot.plot_line_series
Loading