hip: add flash attention kernel A/B override

[TheTom]

Apollo isolated a large Qwen3.5 prefill regression on gfx1201 with ROCm 7.2. This adds one HIP-only diagnostic override so we can compare the tile and MMA flash-attention paths from the same build. Default dispatch is unchanged.

I reproduced the same model and command shape on the RX 9070 XT available here, but only with Vulkan. Warm-cache results do not reproduce the regression:

• f974005: 794.92 pp512
• shared base 15586e2: 799.37 pp512
• current head 407f323: 796.93 pp512
• f974005 with -fa 0: 788.39 pp512
• shared base with -fa 0: 788.82 pp512

The first cold Vulkan run was slower and recovered after pipeline compilation, so these are repeated warm-cache samples. Apollo's ROCm result remains the relevant oracle.

Apollo, please build this head for gfx1201 and run these three cases against each affected model:

env -u GGML_HIP_FA_KERNEL ./llama-bench -m <model> -ngl 99 -fa 1 -p 512 -n 128 -r 5 -o json
GGML_HIP_FA_KERNEL=tile ./llama-bench -m <model> -ngl 99 -fa 1 -p 512 -n 128 -r 5 -o json
GGML_HIP_FA_KERNEL=mma ./llama-bench -m <model> -ngl 99 -fa 1 -p 512 -n 128 -r 5 -o json

Please post the build commit and the five individual pp512 samples for each case. The 9B result is the most sensitive one, so start there. If one forced path recovers the upstream result, I will turn that into the narrow default-dispatch fix and keep the other path available as a fallback.

Local validation: git diff --check. HIP compile CI and the requested gfx1201 performance A/B are pending.

[TheTom]

@apollo-mg thanks for doing the isolation work and providing the full environment and comparison data. The 9B result is the best discriminator here. Please start with the three commands in the PR body after a fresh gfx1201 build, then post the individual samples so we can distinguish a stable kernel gap from warmup noise.

[apollo-mg]

gfx1201 A/B on cc1a79194 — the MMA path is not compiled for RDNA4, so there is nothing to switch to

(Posting as @apollo-mg's agent.)

RX 9070 XT (gfx1201), ROCm 7.2.53211, build cc1a79194, -DGGML_HIP=ON -DAMDGPU_TARGETS=gfx1201 -DGGML_HIP_NO_VMM=ON -DCMAKE_BUILD_TYPE=Release.

Qwen3.5-9B UD-Q2_K_XL (the sensitive one)

case	pp512 avg	five samples
env -u GGML_HIP_FA_KERNEL	314.27 ± 3.86	313.62, 315.27, 307.92, 317.46, 317.06
GGML_HIP_FA_KERNEL=tile	315.81 ± 0.93	316.59, 316.65, 314.70, 316.19, 314.91
GGML_HIP_FA_KERNEL=mma	run fails	see below

tg128: default 87.24 ± 0.15, tile 87.35 ± 0.18.

Qwen3.8-27B GSQ-RCO IQ3_XXS

case	pp512 avg	five samples
env -u GGML_HIP_FA_KERNEL	366.18 ± 1.62	366.03, 368.36, 363.85, 365.98, 366.67
GGML_HIP_FA_KERNEL=tile	378.06 ± 2.30	374.45, 377.25, 379.47, 380.30, 378.82
GGML_HIP_FA_KERNEL=mma	run fails	see below

tg128: default 29.20 ± 0.06, tile 29.24 ± 0.07.

Why mma produces no result

fattn-mma-f16.cuh:2121: ERROR: HIP kernel flash_attn_ext_f16 has no device code
compatible with HIP arch 1300.

Repeated for every launch, no output rows. The MMA flash-attention kernel has no gfx1201 device code in this build, so BEST_FATTN_KERNEL_MMA_F16 is not a reachable target on RDNA4 — the override returns it, the launch finds nothing compiled, and the run dies.

Reading

default and tile are the same path. 314.27 vs 315.81 on the 9B, 366.18 vs 378.06 on the 27B — the second is ~3% and I would not read anything into it. Default dispatch was already choosing TILE, because on this arch there is nothing else to choose.

So this is not a kernel-selection regression. On gfx1201 the fork has exactly one usable FA prefill path and it is the slow one. The question becomes why TILE here is ~2.6x (27B) to ~7.7x (9B) slower than upstream/buun's prefill on the same card and model, rather than which of two paths gets picked.

Happy to run anything else — including a build with the MMA instances compiled in for gfx1201, if that is a cmake flag rather than a code change. That would make the A/B you designed actually answerable.

Scope: one card, one backend, default flags, no turbo/VBR.