Conversation
…xercised
Upstream v0.14.0 plus PR 2194 has no sliding_window support, so a
gemma4-unified decode cannot be driven end to end in CI. Point the Windows
OGA build at the fork branch that already carries both the AMDGPU MorphiZen
integration 2194 supplied and sliding_window.
Rebased onto main: the original three commits edited the monolithic
.github/workflows/windows-build.yml, which main has since split into
windows-{deps,build-mock,build-real,gpu-test}.yml. The OGA pin now lives in
windows-deps.yml, so the change is applied there. The cache key keeps the mm3
bump and the trailing -2 from the pre-rebase history, since test_0_3 has moved
under the same ref.
Windows only, matching the original scope; linux-build.yml keeps its own
OGA_VERSION and still builds upstream.
The fork branch's build.py imports tools/python/util, whose dependency_resolver imports requests at module scope. Upstream v0.14.0 had no such import, so the oga prebuild job never needed it, and the Python 3.14 environment actions/setup-python provides does not carry it. The job therefore died with ModuleNotFoundError before configuring anything. Passing --ort_home already suppresses every download_dependencies call, so only the import has to resolve; nothing is fetched at build time. Co-Authored-By: Cursor <cursoragent@cursor.com> Made-with: Cursor Co-authored-by: Cursor <cursoragent@cursor.com>
|
Thanks for opening a PR! This project follows LLVM's incremental-development and AI-tool-use Before requesting review, please check that:
Reviewers are assigned through |
L2 Accuracy Results (EP vs CPU)
Threshold: 0.01 | Run: 6517 - Commit: |
Binary sizesBaseline: run 35413032867 -
MorphiZen EP Performance Results
EPContext Export Performance
EPContext Import Performance
OGA Benchmark Results
OGA Wheel Smoke (Python benchmark_e2e.py)
VLM Benchmark Results
Run: 6517 - Commit: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
#1068 (the #961 reland) plus #599, so CI builds OGA from the AMDGPU fork and can
actually exercise the workload the allocator change exists for. On its own,
#1068's CI cannot reach Gemma-4 at all.
Stacked on
perf/allocator-pool-large-reland, so this PR's diff is only #599'ssingle-file change to
.github/workflows/windows-deps.yml, byte-identical to#599's. The allocator commits belong to #1068 and should be reviewed there.
Related issue or design
Carries #1068 (relands #961, reverted by #1056) and #599.
Why
#961 optimises the path a decoder takes when
past_present_share_buffer=false:the KV tensors are reallocated every decode step, and above a 4096-token context
each one is over the 16 MB pool cutoff, so every step used to page-pin ~3.4 GiB
afresh. The models that exercise this are the Gemma-4 pair, because they package
max_length=262144and cannot use a shared KV buffer.CI does not currently build an OGA that can run them.
windows-deps.ymlpinsOGA 0.14.0 plus upstream PR patch 2194, which lacks the
gemma4-unifiedfeatures. #599 points it at
amd-genmingz/onnxruntime-genai @ test_0_3, the forkbranch carrying the AMDGPU MorphiZen integration and
sliding_windowsupport.So #1068 alone can be reviewed and its EPCert regression evidence checked, but
its headline result — Gemma-4 decode throughput — cannot be reproduced by CI
until #599 lands. This PR is the combination, run together, so that evidence
exists in CI rather than only on my bench.
What
Two cherry-picks from #599, authorship preserved, on top of #1068's branch:
test: build OGA from the AMDGPU fork so gemma4-unified features are exercised(Zhong, GenMing)ci(windows): install requests before building OGA from the forkNothing is modified relative to either PR.
git diffagainst #599's head forwindows-deps.ymlis empty.Test plan
Local measurements for the allocator change are in #1068 and not repeated here.
What this PR adds is CI coverage of the Gemma-4 path, which is the point of
combining them, so the test plan is the pipeline itself:
windows-depsbuilds OGA from the forkbuild-windowsgreenwindows-gpu-testgreen, includingonnxruntime_perf_testFor reference, the numbers this is meant to let CI confirm, from #1068 — gfx1151,
Gemma-4 26B-A4B, 8192-token prompt, 128 generated, only
hipgpu.dlldiffering,arms interleaved and order-reversed with a private
TEMPeach:pre-commit run --from-ref <base> --to-ref HEADpasses on this branch.Notes for reviewers
Do not merge this ahead of #599. It duplicates #599's commits so CI can run
the combination; #599 is the right place to review and land that change, and
this PR should be closed or rebased once #599 merges.
Review split. The allocator change is #1068's. This PR exists for CI
evidence. If you would rather not carry a duplicate PR, the alternative is to
land #599 first and let #1068's CI pick it up; I raised this because #1068's
Gemma-4 claim is otherwise unverifiable in CI, and I would rather the combination
be exercised before either lands than after.
Scope caveat inherited from #1068. My Gemma-4 measurement is at an 8192-token
context, not #961's 16384, because a 16K run exceeds my box's commit limit on
mainas well as on this branch. CI on this PR is the opportunity to get the 16Kfigure on hardware that can hold it.
Checklist
AI assistance
The branch assembly and this description were produced with Cursor (Claude Opus
5). The two carried commits are unmodified cherry-picks from #599 and verified
byte-identical to it. Disclosure for the allocator work is in #1068.