perf(et): enable uberkernel by default#23
Closed
CodeDoes wants to merge 0 commit into
Closed
Conversation
CodeDoes
force-pushed
the
uberkernel-default-on
branch
from
July 21, 2026 17:38
5b8f50c to
c118878
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The ET backend previously required
GGML_ET_UBERKERNEL=1(or any non-empty, non-"0"value) to turn the uberkernel on. When the variable was unset, the uberkernel stayed off.This PR flips the default so the uberkernel is enabled unless the user explicitly disables it with
GGML_ET_UBERKERNEL=0.Why the uberkernel matters
On ET-SoC1 the dominant cost isn't the matmul itself — it's the per-kernel launch ceremony:
evict_region_past_l2()+WAIT_CACHEOPSRMS_NORM→MUL, orMUL_MAT→ADD) and runs them in one ET launch with a singleet_barrier_global(32ULL)at the endPR #15 (
Q4_K MUL_MAT (scalar + matrix engine) + uberkernel perf recovery) and PR #16 (K-quant MUL_MAT series, "plus uberkernel support") were both built and bench-tested with the uberkernel on. The board numbers in the July 17–18 sessions showed best Q8_0 performance only when the uberkernel + runtime opts were enabled (~170 prompt / ~16 genfor SmolVLM-256M versus~94 / ~11baseline).Why make it the default now
GGML_ET_UBERKERNEL=0restores old behavior)Tests / validation
uberkernel.slots,ggml_et_uberkernel_begin/end_graph) already handles abort/abort paths; callers that disable it explicitly see identical behaviorDepends on: #20, #21 (N-split / alignment fix lives in those PRs; this PR only changes the default-enable flag)