Skip to content

Implement lazy directives for Ramble - #1756

Open
linsword13 wants to merge 3 commits into
Ramble-Project:developfrom
linsword13:lazy-directives
Open

Implement lazy directives for Ramble#1756
linsword13 wants to merge 3 commits into
Ramble-Project:developfrom
linsword13:lazy-directives

Conversation

@linsword13

Copy link
Copy Markdown
Collaborator

This is inspired by Spack's lazy directive (spack/spack#51881).

  • Add in the dict descriptor (as a non-data descriptor) to support the lazy initiailization
  • Remove the convert_class_attributes since now the lazy execution allows us to perform copy on first access
  • Update the various directives to support the new semantics. Specifically, it now requires listing out all mutated dicts of a directive
  • Also unify the various language meta directives, so now they are all essentially DirectiveMeta
  • Update the mirror test since before it implicitly relied on the mutation happening both at the class and the instance level

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors Ramble's directive system to support lazy evaluation of directives upon first access using a new DirectiveDictDescriptor descriptor. It removes the need for explicit class-to-instance attribute conversion and updates various base classes to use DirectiveMeta directly as their metaclass. Additionally, comprehensive tests are added to verify lazy evaluation, inheritance, and attribute isolation. The review feedback highlights a critical issue where instance-level modifications to these lazily evaluated directive dictionaries would be lost during cloning or copying. It is recommended to dynamically copy all evaluated directive dictionaries from self.__dict__ to the cloned or copied instances in both ApplicationBase.clone() and ObjectMixin.copy().

Comment thread var/ramble/repos/builtin/base_classes/application-base/base_class.py Outdated
Comment thread var/ramble/repos/builtin/base_classes/object-mixin/base_class.py
@ramble-pr-bot

ramble-pr-bot Bot commented Sep 1, 2026

Copy link
Copy Markdown

Ramble Performance Test Metrics

Results produced with commit: 2b398d1

Test Name Outcome Duration (s) Most Recent Run (s) Last 5 Avg (s)
test_analyze_large_file passed 2.0712 2.0776 (2d42cdb) 2.0633
test_large_template_expansion passed 1.2171 1.1994 (2d42cdb) 1.1989
test_many_experiments passed 30.6699 23.2386 (2d42cdb) 23.6468
test_many_objects_defaults passed 16.5683 12.1959 (2d42cdb) 12.5012
test_matrix_filter_perf passed 1.1091 1.0590 (2d42cdb) 1.0757

@codecov

codecov Bot commented Sep 1, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.78870% with 9 lines in your changes missing coverage. Please review.
✅ Project coverage is 94.08%. Comparing base (6d999da) to head (e9a802e).
⚠️ Report is 3 commits behind head on develop.

Files with missing lines Patch % Lines
lib/ramble/ramble/language/language_base.py 94.23% 9 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #1756      +/-   ##
===========================================
+ Coverage    94.07%   94.08%   +0.01%     
===========================================
  Files          370      370              
  Lines        37399    37554     +155     
===========================================
+ Hits         35182    35333     +151     
- Misses        2217     2221       +4     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@linsword13

Copy link
Copy Markdown
Collaborator Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors Ramble's directive system to support lazy evaluation on first access, introducing the DirectiveDictDescriptor class and removing the eager conversion of class attributes to instance attributes across all base classes. A comprehensive test suite has been added to verify lazy evaluation, inheritance, and isolation. The review feedback highlights critical inheritance issues where using getattr on classes or objects can mistakenly retrieve parent class attributes (such as evaluated directive dictionaries or preferred versions) instead of checking the subclass's own __dict__ directly, which would bypass subclass directive execution or raise incorrect errors.

Comment thread lib/ramble/ramble/language/language_base.py Outdated
Comment thread lib/ramble/ramble/language/language_base.py Outdated
Comment thread lib/ramble/ramble/language/shared_language.py Outdated
@linsword13

Copy link
Copy Markdown
Collaborator Author

/gemini review

This is inspired by Spack's lazy directive (spack/spack#51881).

* Add in the dict descriptor (as a non-data descriptor) to support the lazy initiailization
* Remove the `convert_class_attributes` since now the lazy execution allows us to perform copy on first access
* Update the various directives to support the new semantics. Specifically, it now requires listing out all mutated dicts of a directive
* Also unify the various language meta directives, so now they are all essentially DirectiveMeta
* Update the mirror test since before it implicitly relied on the mutation happening both at the class and the instance level

Signed-off-by: Lin Guo <linsword13@gmail.com>
Signed-off-by: Lin Guo <linsword13@gmail.com>
Signed-off-by: Lin Guo <linsword13@gmail.com>
@linsword13

Copy link
Copy Markdown
Collaborator Author

/gemini review

@linsword13
linsword13 marked this pull request as ready for review September 4, 2026 00:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant