llama : add an attention split separate from the tensor split

[Piggidragon]

Overview

Stacked on #5. This PR targets that branch, so the diff below is only its own change,
llama : add an attention split separate from the tensor split. Merge #5 first.

Adds --attn-split (-as), the fraction of the attention heads each GPU gets under
--split-mode tensor.

That share decides two things: how much of a host-resident KV cache a device receives, and how much
attention work it does. Neither has to follow the memory split. On an asymmetric pair the device with
the slow host link is exactly the one that should hold less of the cache, while the memory split still
wants to fill both cards.

Unset, it follows --tensor-split and nothing changes. Only the attention tensors and the cache move;
the rest of the model still follows --tensor-split. A linear-attention layer is a different
mechanism and keeps --tensor-split. The share is rounded to whole heads by the granularity rules
that already keep GQA consistent, so a ratio the head count cannot express lands on the nearest one it
can.

Stacked on the host-cache correctness PR, which is what makes a host-resident cache work under
-sm tensor at all.

This is a new CLI and public API addition, so per CONTRIBUTING it carries a higher bar. The argument
for it over the existing mechanism: --tensor-split cannot express it. With a device-resident cache
both devices run concurrently inside a layer and the collective is a barrier, so --tensor-split
wants the compute ratio; with a host-resident cache the cache placement wants the bandwidth ratio.
One knob cannot serve both, and the measurements below show the second one is worth several times more
than the first on this pair.

Testing

Two GPUs, RTX 4070 (gen4 x16) + RTX 3060 (gen3 x4), CUDA, NCCL, stock clocks.
Qwen3.8-27B-UD-Q5_K_M, a 56935-token prompt from this repository's docs, -c 65536 -n 128 -ngl 99,
q8_0 cache, -sm tensor -ts 50,50 -nkvo --kv-cpu-pinned --recurrent-state-offload. The model has
n_head_kv = 4, so the whole-head shares available are 2,2 / 3,1 / 4,0.

--attn-split	rounds to	pp	tg
(follows -ts)	2,2	433.33	2.47
0.75,0.25	3,1	475.52	3.55
3,1	3,1	475.69	3.56
1,0	4,0	501.97	7.49

3.03x generation from moving the cache off the slow card, and +16% prefill. 0.75,0.25 and an
explicit 3,1 agree to within noise, which is the rounding working: a ratio the head count cannot
express takes the nearest one it can, and it lands strictly between its two neighbours.

Setting --attn-split when it is not used costs nothing. llama-bench -ngl 99 -ts 50/50 -r 3,
all 16 rows against the parent PR without this commit:

model	sm	nkvo	pp512 parent	pp512 this	tg128 parent	tg128 this
Qwen3.8-27B-UD-IQ2_M	layer	0	755.87	752.83	26.38	26.34
	layer	1	532.98	532.78	8.68	8.66
	tensor	0	610.71	609.89	30.96	30.91
	tensor	1	588.78	589.17	25.61	25.64
gemma-4-26B-A4B	layer	0	2732.69	2733.83	103.83	103.90
	layer	1	1941.45	1986.41	42.87	42.98
	tensor	0	1685.83	1688.82	85.84	86.09
	tensor	1	1339.58	1343.70	34.97	35.37

tests/test-llama-archs.cpp runs the tensor-split matrix a third time with all attention heads on the
first device (attn_split = 1,0) and the cache in host memory, checking the logits against CPU.

test-llama-archs -s 1 at 1, 2, 3 and 4 virtual CUDA devices: passes. test-arg-parser: passes.
Built with -DLLAMA_FATAL_WARNINGS=ON.

Not done here

• Deriving the split automatically from the measured bandwidth.
• -as for llama-bench, which only takes -sm, -ts, -ub, -ngl and -fa; the sweeps below had
  to be run through llama-completion.

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES - implemented by an agent on my instruction, see the Assisted-by: commit trailers.

🤖 Generated with Claude Code