Skip to content

fix[next]: Add build configuration to the compilation fingerprint#2650

Open
edopao wants to merge 21 commits into
GridTools:mainfrom
edopao:dace-compilation-fingerprint
Open

fix[next]: Add build configuration to the compilation fingerprint#2650
edopao wants to merge 21 commits into
GridTools:mainfrom
edopao:dace-compilation-fingerprint

Conversation

@edopao

@edopao edopao commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Description
The compiled artifacts (build folders, SDFGs) were keyed only by a hash of the program source. As a result, changing the backend configuration — in particular DaCe's dace.Config or the builder factory for gtfnbackends — did not invalidate the cache, so a stale artifact could be reused even though the configuration that produced it had changed.

This PR includes the relevant build-time configuration into the cache-folder fingerprint so artifacts are correctly rebuilt when the configuration changes.

To make it possible, we added an optional build_context_id to get_cache_folder() which allows to distinguish different build contexts that produce different artifacts for the same extension source.

Changes to DaCe backend
DaCeCompiler captures the non-default dace.Config values at construction and includes them in the build context fingerprint, so an SDFG is rebuilt when the user changes the backend configuration.

Changes to GTFN backend
The builder_factory is fingerprinted into the above-mentioned build_context_id. Note that builder_factory specifies the cmake configuration.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the DaCe backend compilation workflow to incorporate DaCe configuration into the compilation fingerprint, so that different DaCe/compiler flag configurations produce separate build subfolders (avoiding cache/build-folder clashes).

Changes:

  • Add a DaCe config snapshot (dace_config_nondefaults) to the DaCeCompiler stage data used for fingerprinting.
  • Append a per-compiler fingerprint subfolder to the existing cache folder when setting sdfg.build_folder.
  • Add a unit test intended to verify that changing CXXFLAGS/CUDAFLAGS/HIPFLAGS results in a different build folder.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
src/gt4py/next/program_processors/runners/dace/workflow/compilation.py Adds DaCe config snapshotting to compiler “identity” and uses a fingerprint-named subfolder for builds.
tests/next_tests/unit_tests/program_processor_tests/runners_tests/dace_tests/test_dace_compilation.py Adds a regression test asserting compiler-flag changes lead to different build folders.

Comment thread src/gt4py/next/program_processors/runners/dace/workflow/compilation.py Outdated
@edopao edopao force-pushed the dace-compilation-fingerprint branch 2 times, most recently from 20af1f4 to 261d78a Compare June 16, 2026 10:28
@edopao edopao force-pushed the dace-compilation-fingerprint branch from 58e3273 to b7ab4d0 Compare June 19, 2026 10:59
@edopao edopao requested a review from egparedes June 19, 2026 13:19
edopao and others added 5 commits June 22, 2026 10:11
…lder API

Propagate the CompilableProject -> ExtensionSource rename and the new
get_cache_folder signature (build_context_id) through docstrings, comments,
user docs and tests. Also fix compiledb's prototype-source mismatch: the
locate and create paths must share one ExtensionSource so the new folder
naming (binding source folded into slug + fingerprint) resolves to the same
cache folder.
Comment thread src/gt4py/next/otf/compilation/cache.py
@edopao edopao changed the title fix[next-dace]: Add dace configuration to compilation fingerprint fix[next]: Add step configuration to compilation fingerprint Jun 26, 2026
@edopao edopao changed the title fix[next]: Add step configuration to compilation fingerprint fix[next]: Add fingerprint of the build configuration to cache folder name Jun 26, 2026
@edopao edopao changed the title fix[next]: Add fingerprint of the build configuration to cache folder name fix[next]: Add build configuration to the compilation fingerprint Jun 26, 2026

@egparedes egparedes left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Just a nitpick and the failing test, otherwise looks good.

Comment thread src/gt4py/next/otf/compilation/cache.py Outdated
edopao and others added 2 commits June 26, 2026 23:20
Co-authored-by: Enrique González Paredes <enriqueg@cscs.ch>
@edopao edopao force-pushed the dace-compilation-fingerprint branch from 96224ef to 36b5212 Compare June 26, 2026 21:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

next: meta-issue related to all toolchain caching erros in gt4py.next

3 participants