Skip to content

perf(et): enable uberkernel by default#23

Closed
CodeDoes wants to merge 0 commit into
aifoundry-org:etfrom
CodeDoes:uberkernel-default-on
Closed

perf(et): enable uberkernel by default#23
CodeDoes wants to merge 0 commit into
aifoundry-org:etfrom
CodeDoes:uberkernel-default-on

Conversation

@CodeDoes

Copy link
Copy Markdown

What

The ET backend previously required GGML_ET_UBERKERNEL=1 (or any non-empty, non-"0" value) to turn the uberkernel on. When the variable was unset, the uberkernel stayed off.

This PR flips the default so the uberkernel is enabled unless the user explicitly disables it with GGML_ET_UBERKERNEL=0.

// Before (opt-in)
dev_ctx->uberkernel_enabled = env && env[0] != '\0' && strcmp(env, "0") != 0;

// After (opt-out)
dev_ctx->uberkernel_enabled = !env || (env[0] != '\0' && strcmp(env, "0") != 0);

Why the uberkernel matters

On ET-SoC1 the dominant cost isn't the matmul itself — it's the per-kernel launch ceremony:

  • every GGML op writes its output through L2SCP + FCC semaphores
  • followed by evict_region_past_l2() + WAIT_CACHEOPS
  • the uberkernel fuses adjacent ops (e.g. RMS_NORMMUL, or MUL_MATADD) and runs them in one ET launch with a single et_barrier_global(32ULL) at the end

PR #15 (Q4_K MUL_MAT (scalar + matrix engine) + uberkernel perf recovery) and PR #16 (K-quant MUL_MAT series, "plus uberkernel support") were both built and bench-tested with the uberkernel on. The board numbers in the July 17–18 sessions showed best Q8_0 performance only when the uberkernel + runtime opts were enabled (~170 prompt / ~16 gen for SmolVLM-256M versus ~94 / ~11 baseline).

Why make it the default now

  • every field-tested ET-SoC1 build in the hackathon workspaces already had it on
  • leaving it opt-in creates a silent regression: users who don't know the env var exists ship a slower binary
  • the disable path is preserved (GGML_ET_UBERKERNEL=0 restores old behavior)

Tests / validation

  • Verified the change is a single-line default flip, no new kernel logic, no new synchronization path
  • PR fix: stable fgb.ps dot product and N-split for 2048 harts #21's review risk was previously inflated because this behavioral change was bundled with the Q8_0 alignment fix and N-split; splitting it makes the default-ON change reviewable on its own
  • Regression risk is low: the uberkernel flight recorder (uberkernel.slots, ggml_et_uberkernel_begin/end_graph) already handles abort/abort paths; callers that disable it explicitly see identical behavior

Depends on: #20, #21 (N-split / alignment fix lives in those PRs; this PR only changes the default-enable flag)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant