feat(qwen4_exp): load-time per-tensor FP8 dense projections (W8A8 via _scaled_mm) - #389

Closed

gdevenyi wants to merge 16 commits into

FlashML-org:mainfrom

gdevenyi:feat/qwen4-exp-fp8-dense

Commits on Sep 10, 2026

feat(qwen4_exp): tensor parallelism for Qwen3.8-Flash-Next (offload backend)

gdevenyi
and
claude
committed
c3d0f97
fix(qwen4_exp): load the HF config for sharding only when TP > 1

gdevenyi
and
claude
committed
96e190c
test(qwen4_exp): TP shard test against the new expert-piece stream
gdevenyi
committed
a99433c
feat(qwen4_exp): load-time per-tensor FP8 dense projections (W8A8 via _scaled_mm)

gdevenyi
and
claude
committed
be41dea
chore(qwen4_exp): log the load-time FP8 dense mode once at weight load

gdevenyi
and
claude
committed
0968c24
fix(qwen4_exp): return the FP8 quantization slack to the allocator before the cache planner runs

gdevenyi
and
claude
committed
8d37231
fix(qwen4_exp): give in_proj_ba its own storage under FP8 dense

gdevenyi
and
claude
committed
1a6f504
perf(fp8): quantize activations with a block-parallel amax

gdevenyi
and
claude
committed
74ccbea
feat(layers): optional per-tensor FP8 lm_head (FREETOKEN_FP8_LMHEAD=1)

gdevenyi
and
claude
committed
85c4b60
fix(qwen4_exp): forward prefix through Qwen4ExpMoE, and test the CPU build

gdevenyi
and
claude
committed
62a9a8c
Merge branch 'rb/tp' into rb/fp8
gdevenyi
committed
339c312
feat(quant): let the triton NVFP4 expert kernel serve a tensor-parallel shard

gdevenyi
and
claude
committed
d2ad39b
Merge branch 'rb/tp' into rb/fp8
gdevenyi
committed
72e018e
fix(qwen4_exp): one predicate decides the FP8 lm_head, for builder and loader alike

gdevenyi
and
claude
committed
1da3c6c
fix(qwen4_exp): import fp8_lmhead_enabled, which use_fp8_lmhead calls

gdevenyi
and
claude
committed
d377a20
fix(qwen4_exp): restore the local head counts the GDN forward reads

gdevenyi
and
claude
committed
bd106c9

Navigation Menu

feat(qwen4_exp): load-time per-tensor FP8 dense projections (W8A8 via _scaled_mm) - #389

feat(qwen4_exp): load-time per-tensor FP8 dense projections (W8A8 via _scaled_mm)#389
gdevenyi wants to merge 16 commits into
FlashML-org:mainfrom
gdevenyi:feat/qwen4-exp-fp8-dense

Commits on Sep 10, 2026

feat(qwen4_exp): tensor parallelism for Qwen3.8-Flash-Next (offload backend)

fix(qwen4_exp): load the HF config for sharding only when TP > 1

test(qwen4_exp): TP shard test against the new expert-piece stream

feat(qwen4_exp): load-time per-tensor FP8 dense projections (W8A8 via _scaled_mm)

chore(qwen4_exp): log the load-time FP8 dense mode once at weight load

fix(qwen4_exp): return the FP8 quantization slack to the allocator before the cache planner runs

fix(qwen4_exp): give in_proj_ba its own storage under FP8 dense

perf(fp8): quantize activations with a block-parallel amax

feat(layers): optional per-tensor FP8 lm_head (FREETOKEN_FP8_LMHEAD=1)

fix(qwen4_exp): forward prefix through Qwen4ExpMoE, and test the CPU build

Merge branch 'rb/tp' into rb/fp8

feat(quant): let the triton NVFP4 expert kernel serve a tensor-parallel shard

Merge branch 'rb/tp' into rb/fp8

fix(qwen4_exp): one predicate decides the FP8 lm_head, for builder and loader alike

fix(qwen4_exp): import fp8_lmhead_enabled, which use_fp8_lmhead calls

fix(qwen4_exp): restore the local head counts the GDN forward reads