docs(qwen3.8-flash-next): ubatch measurements with experts resident - #1

Open

mrmagidev wants to merge 39 commits into

mihailescu2m:masterfrom

mrmagidev:docs-ubatch-resident

Commits on Sep 2, 2026

metal : promote mul_mm batched dst offsets to 64-bit
mihailescu2m
committed
bf1a61b
metal : fix half-idle simdgroup in kernel_mul_mv_iq3_xxs_f32 for ne00 < 1024

masterFoad
authored and
mihailescu2m
committed
ce65b77
CUDA + ggml: add sparse flash attention

am17an
authored and
mihailescu2m
committed
34a822c
fix PDL issues

am17an
authored and
mihailescu2m
committed
c2dc045
metal : support n_kv_max sparse mask hint in flash attention vec kernel

ggerganov
authored and
mihailescu2m
committed
f9b1a36
metal : fix sparse flash attention row addressing

ggerganov
authored and
mihailescu2m
committed
aa33814
cont : use sparse vec FA for prefill

ggerganov
authored and
mihailescu2m
committed
3cd91a0
metal : single-pass flash attention sparse index compaction

ggerganov
authored and
mihailescu2m
committed
7ebf049
tests : add perf cases for sparse flash attention prefill

ggerganov
authored and
mihailescu2m
committed
305d1bb
qwen4 : enable sparse attention

ggerganov
authored and
mihailescu2m
committed
559b5ef
cont : adjust nsg

ggerganov
authored and
mihailescu2m
committed
1a12258
ggml: add ggml_backend_tensor_get_host_ptr
mihailescu2m
committed
f7c7a2e
metal: CPU-side shared event signal and notification
mihailescu2m
committed
2fe57ad
metal: opt-in GPU profiling and capture tooling
mihailescu2m
committed
5c557c5
metal: cap the flash-attention unroll, and index MXFP4 by byte
mihailescu2m
committed
5b94c9b
qwen: align q8_0 KV dequant loads on Metal
mihailescu2m
committed
efdd557
moe-stream: stream MoE routed experts from SSD
mihailescu2m
committed
ef6ed8f
moe-stream: run the expert GEMMs over a partitioned pair list
mihailescu2m
committed
e379505
moe-stream: raise I/O queue depth and fix the eviction clock
mihailescu2m
committed
2327bf0
moe-stream : prefetch experts ahead of the layer that needs them
mihailescu2m
committed
9d9aff4
llama-bench: support MoE expert streaming
mihailescu2m
committed
0d8fa27
moe-stream: resolve expert cache slots on the GPU
mihailescu2m
committed
34e7f7f
moe-stream: stream a large gather table's rows from disk
mihailescu2m
committed
4739972
moe-stream: enable pair partitioning by default
mihailescu2m
committed
0c799d4
moe-stream: guarantee a pair-split receiver, and let WAVE_CAP win
mihailescu2m
committed
fed95d4
metal: harden opt-in profiling and capture tools
mihailescu2m
committed
8724404
metal: parse the opt-in profiling env vars by value
mihailescu2m
committed
411c5ef
deepseek: union-8 sparse attention
mihailescu2m
committed
67cc44c
deepseek: lift the union-8 64k row cap
mihailescu2m
committed
bbf27d6
qwen4exp: indexer and routing optimisations
mihailescu2m
committed
d2a07a3
qwen4exp: keep indexer keys in f16
mihailescu2m
committed
0d74c2e
qwen4exp: native NextN/MTP draft head
mihailescu2m
committed
cb640db
qwen4exp: map block top-k selections through blk_cells
mihailescu2m
committed
778a648
qwen4exp: bounded speculation and persistent slot checkpoints
mihailescu2m
committed
22e37f9
server: fix checkpoint warning build
mihailescu2m
committed
55c7ab4
server: bound checkpoint sidecar blob reads by the file size
mihailescu2m
committed
a3fd0a4
tests: expert-GEMM perf cases at both target models' shapes
mihailescu2m
committed
cf8e62b
docs: checkpoint composition and kernel surveys
mihailescu2m
committed
8e0e527

Commits on Sep 4, 2026

docs(qwen3.8-flash-next): ubatch measurements with experts resident
mrmagidev
committed
6cb7e4c