cuda: stream TurboQuant MoE experts from pinned host memory

[jasstrong]

Three patches letting a TurboQuant mixture-of-experts model keep its expert stack
in pinned host memory with the hot experts cached in VRAM, so a model larger than
the card runs without spilling everything to the CPU.

An expert stack is the one weight worth doing this to. A token routes to a
handful of experts out of hundreds, so the traffic is per-expert rather than the
whole tensor, and at that granularity the bus is fast enough that the copies
overlap compute.

1. Route expert addresses through an optional table. The MoE kernels find an
expert by arithmetic on the base pointer, which requires every expert to be
resident and contiguous. The table is built on the device (reading it back would
be illegal during a graph capture) and initially reproduces that layout exactly,
so it is a no-op. The point is that an entry can then be pointed elsewhere.

2. Cache the table per expert tensor, rather than rebuilding it per call. Raw
device memory, never freed early, retired on replacement, released at teardown,
for the same reason as the pre-rotation caches next to it.

3. The residency cache. Per tensor, a slab of n_slots expert-sized slots plus
bookkeeping that stays on the device so the policy runs inside the graph. Each
decode probes the routed experts against the slots, awards a slot to each miss,
copies those in and repoints the table. Anything that loses a slot contest reads
in place from host memory, which is correct if slower.

Measured on an MI210 with a 91 GB TQ3 Qwen3.8-Flash-Next, four expert layers in
host memory:

	generation
all experts in VRAM (ceiling)	34.2 t/s
four layers in host, paging off	22.9 t/s
paging on, 16 slots	31.7 t/s
paging on, 64 slots	32.6 t/s
paging on, 128 slots	32.8 t/s

43% over reading in place, and 96% of what the same layers cost when they never
left the card.

Correctness was checked by perplexity rather than by reading the output, because
the failure mode here is a wrong expert, which produces fluent text that is
quietly wrong. On 273 KB of prose, perplexity is 3.3453 for all of: experts in
VRAM, experts in host with paging off, and paging on at 64, 16 and 4 slots. Four
slots against ten routed experts per token forces a collision on essentially
every call, which is the condition that matters: the probe reserves a slot it
hits with an atomicMin so a miss mapping to the same slot cannot claim it and
overwrite the contents mid-call. An earlier version of this without the
reservation passed a read-the-output check and was wrong.

Everything is opt-in: GGML_MOE_HOST_EXPERTS to admit host-resident experts,
GGML_MOE_EXPERT_TABLE for the indirection, GGML_MOE_PAGE_IN with
GGML_MOE_PAGE_SLOTS for the paging. With them unset the kernels take the original
contiguous path and the generated code is unchanged.

One known rough edge: the pool allocator half-succeeds when it cannot get what it
asked for, rather than sizing to available VRAM or failing outright. The slot
count behaves monotonically here (16 through 128 improve smoothly), though on a
different kernel I have seen the direct-mapped modulo make it erratic, so a
mixing hash or set-associativity may be worth it later.

TurboQuant-specific, admitted only for the TQ expert path and only on opt-in, so
no upstream question here.

[giveen]

@jasstrong have you checked out --moe-cache auto

[jasstrong]

Thanks, I had not. I have now, and it does not cover this case.

Two hard blockers first. The cache only sees experts assigned to the CPU backend on the regular MUL_MAT_ID path, and these sit in ROCm_Host behind an explicit override, so there is nothing for it to observe. And moe_cache_type_supported() enumerates 22 types from Q1_0 to IQ4_XS with no TQ entry, so TQ3_1S experts are declined regardless of placement.

The deeper difference is that moe-cache caches and this pages. moe-cache wants the canonical experts CPU-resident in full and copies the hot ones into a VRAM pool. That assumes host RAM can hold every expert. On the machine I am running, the experts are about 57.6 GiB across 48 layers, the n-gram table is another 27 GiB, and the box has 60 GiB. Canonical CPU experts do not fit, so that placement is not a slower option, it is not an option. What this PR does instead is keep most experts resident and page the spill: the routed experts are copied into a bounded slot pool, and an expert that loses a slot collision is simply read in place through the address table. Host residency is sized to the spill, currently 8.4 GiB, not to the model.

On teaching moe-cache about TQ rather than landing this: I do not think that is as cheap as it sounds. moe-cache dispatches through mmvq.cuh and holds cached experts in a pooled slab at a uniform per-expert stride. The TQ MoE kernels are 2905 lines across four variants selected per architecture, fp16 MFMA multi-column on CDNA, dp4a on RDNA2 and RDNA3, and a scalar fallback. They are the survivors of 32 commits of measurement on real hardware, including one that deleted the decode paths that turned out slower than what replaced them. Reconciling them with the slab layout means rewriting the innermost addressing of every variant, and that tuning was established empirically rather than by construction, so it would have to be re-established the same way.

Which is an argument for the shape of this PR rather than against it. The address table is exactly the seam that integration needs: once expert addresses come from a table instead of being computed from the tensor base, the same kernels can read an expert from a slot, from host memory, or in principle from a moe-cache slab. I would rather land the indirection and wire it up afterwards than rewrite four tuned kernels against a layout before anything has measured whether the combination wins.

[jasstrong]

CI on this branch has been red since it opened, for two reasons that never show up on HIP. Fixed in b6d7fd5 and e04fd47.

nvcc

error #177-D: declared but never referenced, twice, and both were genuine leftovers from the rework rather than anything subtle:

• tq_page_in_experts, the original bulk kernel. tq_page_in_misses superseded it once the residency plan started marking which experts actually need copying, and nothing has called it since. Removed. I moved its comment onto the kernel that replaced it rather than deleting it, because it carries the measurement the whole approach rests on: 27.6 GB/s for a coalesced copy against 2.7 GB/s for the matmul reading the same host memory in place.
• n_expert_all, a width computed and never used.

MUSA

The non-HIP branch of the host-memory check did not compile there at all: no cudaPointerAttributes, cudaPointerGetAttributes, cudaMemoryTypeHost or cudaMemoryTypeUnregistered.

Paging is only worth anything when the expert stack is in host memory, and that query is how it is established, so the feature is now off on MUSA. I want to be explicit that this is off by omission and not by measurement:

• vendors/musa.h maps a good deal of the memory API (cudaHostRegister, cudaMemcpy and friends) but nothing for pointer attributes.
• Upstream llama.cpp has no reference to a musaPointerGetAttributes anywhere, so there is no spelling to copy. A guessed symbol would just break the build again on hardware I cannot test.

Rather than leave that as a silent consequence of the compile fix, the gate now says page_in_on = false on MUSA with the reasoning next to it, and names the four mappings to add to vendors/musa.h to turn it back on. If someone with MUSA headers can confirm those symbols exist, enabling it is a four-line change plus deleting two blocks.

Worth flagging for whoever reviews: I have no MUSA or NVIDIA hardware, so both of these are build-correctness only. CI is the test, not me.