Skip to content

tests: add MoE MMQ benchmark with routing-distribution generator#62

Draft
roberteg16 wants to merge 1 commit into
masterfrom
rogarcia.moe-mmq-benchmark
Draft

tests: add MoE MMQ benchmark with routing-distribution generator#62
roberteg16 wants to merge 1 commit into
masterfrom
rogarcia.moe-mmq-benchmark

Conversation

@roberteg16

Copy link
Copy Markdown

Summary

  • Adds benchmark-moe-mmq, a standalone benchmark for the MMQ mul_mat_id kernel used by MoE.
  • Generates controlled expert-routing distributions so the per-expert token load (which drives MMQ tile sizing) can be swept independently of everything else:
    • uniform - tokens spread evenly across all experts
    • single - all tokens concentrated on one expert (worst case)
    • zipf - power-law imbalance (--alpha)
    • concentration - sweep of tokens-per-expert aligned to 16 (active set shrinking from n_experts down to top_k)
  • Reports tokens-per-expert stats and a roofline per row (arithmetic intensity, % of roofline ceiling, mem/comp bound); writes CSV.
  • Defaults match Qwen3.6-35B-A3B (256 experts, top_k 8, Q4_K). Cycles distinct weight copies (--rotate, auto-sized from L2/MALL) so measurements reflect DRAM traffic rather than cache residency.
  • Registered as a build-only target (llama_build), not a ctest, since it is a manual perf tool.

Test plan

  • cmake --build build --target benchmark-moe-mmq
  • ./bin/benchmark-moe-mmq --legend prints column/distribution help
  • ./bin/benchmark-moe-mmq --dist all --proj both --m 512,2048 runs on the target backend and emits the table + CSV

Standalone benchmark (benchmark-moe-mmq) for the MMQ mul_mat_id kernel used
by MoE. Generates controlled expert-routing distributions (uniform, single,
zipf, concentration) so the per-expert token load can be swept independently,
which is what drives the MMQ tile sizing. Reports tokens-per-expert stats plus
a roofline (AI, %roof, mem/comp) and writes CSV. Defaults match Qwen3.6-35B-A3B
(256 experts, top_k 8, Q4_K). Cycles distinct weight copies to defeat L2/MALL
caching so measurements reflect DRAM traffic.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant