Skip to content

Extract external metadata - #279

Merged
Sam Clarke-Green (t00sa) merged 46 commits into
MetOffice:mainfrom
james-bruten-mo:lfric_jules_metadata
Apr 27, 2026
Merged

Extract external metadata#279
Sam Clarke-Green (t00sa) merged 46 commits into
MetOffice:mainfrom
james-bruten-mo:lfric_jules_metadata

Conversation

@james-bruten-mo

@james-bruten-mo James Bruten (james-bruten-mo) commented Feb 23, 2026

Copy link
Copy Markdown
Collaborator

PR Summary

Sci/Tech Reviewer: Matthew Hambley (@MatthewHambley)
Code Reviewer: Sam Clarke-Green (@t00sa)

jules-shared is currently duplicated between LFRic Apps and Jules, so that it's available to the lfric build system configuration step. Ideally we'd remove this duplication. This PR, along with MetOffice/lfric_core#285, will edit the code extraction and configuration steps so that external metadata can be used. Then a follow up Apps and Jules PR will remove the duplicated metadata.

Code Quality Checklist

  • I have performed a self-review of my own code
  • My code follows the project's style guidelines
  • Comments have been included that aid understanding and enhance the readability of the code
  • My changes generate no new warnings
  • All automated checks in the CI pipeline have completed successfully

Testing

  • I have tested this change locally, using the LFRic Apps rose-stem suite
  • If any tests fail (rose-stem or CI) the reason is understood and acceptable (e.g. kgo changes)
  • I have added tests to cover new functionality as appropriate (e.g. system tests, unit tests, etc.)
  • Any new tests have been assigned an appropriate amount of compute resource and have been allocated to an appropriate testing group (i.e. the developer tests are for jobs which use a small amount of compute resource and complete in a matter of minutes)

trac.log

Test Suite Results - lfric_apps - external_meta_apps/run2

Suite Information

Item Value
Suite Name external_meta_apps/run2
Suite User james.bruten
Workflow Start 2026-03-16T14:09:17
Groups Run developer
Dependency Reference Main Like
casim MetOffice/casim@2026.03.2 True
jules MetOffice/jules@2026.03.2 True
lfric_apps james-bruten-mo/lfric_apps@lfric_jules_metadata False
lfric_core james-bruten-mo/lfric_core@lfric_jules_metadata True
moci MetOffice/moci@2026.03.2 True
SimSys_Scripts MetOffice/SimSys_Scripts@2026.03.2 True
socrates MetOffice/socrates@2026.03.2 True
socrates-spectral MetOffice/socrates-spectral@2026.03.2 True
ukca MetOffice/ukca@2026.03.2 True

Task Information

✅ succeeded tasks - 1164

Security Considerations

  • I have reviewed my changes for potential security issues
  • Sensitive data is properly handled (if applicable)
  • Authentication and authorisation are properly implemented (if applicable)

Performance Impact

  • Performance of the code has been considered and, if applicable, suitable performance measurements have been conducted

AI Assistance and Attribution

  • Some of the content of this change has been produced with the assistance of Generative AI tool name (e.g., Met Office Github Copilot Enterprise, Github Copilot Personal, ChatGPT GPT-4, etc) and I have followed the Simulation Systems AI policy (including attribution labels)

Documentation

  • Where appropriate I have updated documentation related to this change and confirmed that it builds correctly

PSyclone Approval

  • If you have edited any PSyclone-related code (e.g. PSyKAl-lite, Kernel interface, optimisation scripts, LFRic data structure code) then please contact the TCD Team

Sci/Tech Review

  • I understand this area of code and the changes being added
  • The proposed changes correspond to the pull request description
  • Documentation is sufficient (do documentation papers need updating)
  • Sufficient testing has been completed

(Please alert the code reviewer via a tag when you have approved the SR)

Code Review

  • All dependencies have been resolved
  • Related Issues have been properly linked and addressed
  • CLA compliance has been confirmed
  • Code quality standards have been met
  • Tests are adequate and have passed
  • Documentation is complete and accurate
  • Security considerations have been addressed
  • Performance impact is acceptable

Co-authored-by: Matthew Hambley <MatthewHambley@users.noreply.github.com>
Co-authored-by: Sam Clarke-Green <74185251+t00sa@users.noreply.github.com>
 All conflicts fixed but you are still merging.
@github-actions github-actions Bot added the cla-modified The CLA has been modified as part of this PR - added by GA label Mar 19, 2026

@t00sa Sam Clarke-Green (t00sa) left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trivial tweak to function docstring, but looks good otherwise

Comment thread build/extract/extract_science.py Outdated
Co-authored-by: Sam Clarke-Green <74185251+t00sa@users.noreply.github.com>
@t00sa

Copy link
Copy Markdown
Collaborator

Sorry Matthew Hambley (@MatthewHambley) - I thought you'd SRed both this and the core ticket

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple of comments to consider.

Comment thread build/extract/extract_science.py Outdated
Comment thread build/extract/extract_science.py

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Matthew, comments addressed below

Comment thread build/extract/extract_science.py Outdated
Comment thread build/extract/extract_science.py

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This whole issue will be revisited as we migrate build systems but in the mean time, this seems to fulfil the requirements.

@t00sa Sam Clarke-Green (t00sa) left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

@github-actions github-actions Bot removed the cla-modified The CLA has been modified as part of this PR - added by GA label Apr 27, 2026

@t00sa Sam Clarke-Green (t00sa) left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approving after dependencies.yaml update

@t00sa
Sam Clarke-Green (t00sa) merged commit 9ee7884 into MetOffice:main Apr 27, 2026
7 checks passed
@matthewrmshin

Copy link
Copy Markdown

This change is breaking our daily JEDI build at the moment while building jedi_lfric_tests. The build did not use to fetch JULES before, but now it attempts to do so and fails.

@james-bruten-mo

Copy link
Copy Markdown
Collaborator Author

This change is breaking our daily JEDI build at the moment while building jedi_lfric_tests. The build did not use to fetch JULES before, but now it attempts to do so and fails.

Yes, this will now require Jules access. This PR is a pre-requisite for another one which will remove the lfric-jules-shared metadata as this is an exact copy of the jules-shared metadata stored in the Jules repo. We're removing this as maitaining 2 copies is obviously not great. Do you need a shared account being granted access to jules?

@matthewrmshin

Copy link
Copy Markdown

This change is breaking our daily JEDI build at the moment while building jedi_lfric_tests. The build did not use to fetch JULES before, but now it attempts to do so and fails.

Yes, this will now require Jules access. This PR is a pre-requisite for another one which will remove the lfric-jules-shared metadata as this is an exact copy of the jules-shared metadata stored in the Jules repo. We're removing this as maitaining 2 copies is obviously not great. Do you need a shared account being granted access to jules?

Yes, if the easiest option is to grant read-only access to a shared account to fetch JULES. Please grant access to modarth in which case.

@matthewrmshin

Copy link
Copy Markdown

(As long as it is possible to pre-fetch JULES and then bypass Git fetch in build time.)

@james-bruten-mo

Copy link
Copy Markdown
Collaborator Author

Yes, using a local copy of Jules should be possible. I've added modarth with read access to Jules

@ss421

Copy link
Copy Markdown
Contributor

This change caused a failure in the jedi build and we now have a fix with help from Matt Shin (@matthewrmshin) Yaswant Pradhan (@yaswant) James Bruten (@james-bruten-mo) David Simonin (@DavidSimonin). I see that the change did update the Makefile in: jedi_lfric_tests but that did not trigger codeowners. Is it understood why and can it be rectified so we get notified when any source file is modified?

@yaswant

Copy link
Copy Markdown
Collaborator

Steven Sandbach (@ss421) I suspect this PR was pointing to an older version of CODEOWNERS file, hence you were not automatically notified.

@ss421

Copy link
Copy Markdown
Contributor

Steven Sandbach (Steven Sandbach (@ss421)) I suspect this PR was pointing to an older version of CODEOWNERS file, hence you were not automatically notified.

I see. So we were just unlucky! :-(

@james-bruten-mo

Copy link
Copy Markdown
Collaborator Author

Ah no, this is because the makefiles have their own owner. But we can modify the owners file so that the jedi makefiles are jointly owned by jedi and the other makefile owner - both will get notifications.

@ss421

Copy link
Copy Markdown
Contributor

Ah no, this is because the makefiles have their own owner. But we can modify the owners file so that the jedi makefiles are jointly owned by jedi and the other makefile owner - both will get notifications.

Thanks :-) Are there any other files in that category or is it just Makefiles?

@yaswant

Copy link
Copy Markdown
Collaborator

Ah I forgot to take notice of the order...

At the moment changes to any Makefile, and .mk files have overriding rules

**/*Makefile  @MatthewHambley @hiker
**/*.mk          @MatthewHambley @hiker

Please take a look at the applications and interface sections for jedi where these files might exist. We can then add new entries at the bottom of the CODEOWNERS file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants