CUDA: radix top-k for large row counts

[praneshgo]

Overview

Replaces CUB's per-row DeviceTopKKernel with a grid-over-rows radix select, gated on GGML_CUDA_TOPK_RADIX_MIN_ROWS.
Current implementation of top_k finding is sequential over rows (internal to a row, it is parallel). There is a for loop that goes over all the rows. The upstream top-k uses CPU processing over rows to loop over, but GPU processing internal to the row. This PR introduces radix select that is a single call at the op level that processes on the GPU completely and has a positive impact on the prefill perf.

Also, the optimization applies to any model that calls ggml_top_k, with some conditionals on row and column count. Some examples that call ggml_top_k include deepseek32, deepseek4, minimax-m3.
top_k calls are used after MOE router, with backend sampling, DFlash and Sparse-attention indexers.

Performance with various -lzm options on Qwen 3.8 Flash Next over DGX Spark

Six runs per arm over two rounds. Cold cache before every arm. Prompts from a 2.09M-word prose corpus so no two requests share n-grams — filler from a small vocabulary saturates the reachable n-gram set after one request and inflates the demand-paged arm. On/off order flipped between rounds.

The on arm is I/O-bound, demand-paging a 26.8 GiB embedding table one fault at a time, so removing launches buys nothing there — hence its runs interleaving. An independent 2×2 at pp8192 agrees: 675.2 → 800.8 resident, 182.4 → 193.6 faulting.

Decode is unchanged in every arm (~17.9–22.3 tok/s): ggml_top_k is not on this model's decode path at batch 1.

--lazy-mode auto (resolves to OFF on this device, so the table is read resident)

run	prompt tok	radix on	radix off
1	34,865	798.60	639.90
2	34,661	804.00	644.60
3	34,798	800.90	643.60
4	34,812	805.40	658.80
5	34,742	803.40	662.60
6	34,733	804.00	662.40
	mean	802.72 ± 2.50	651.98 ± 10.38
	median	803.70	651.70

--lazy-mode on-direct (from #28136 )

run	prompt tok	radix on	radix off
1	34,865	728.90	619.30
2	34,661	737.00	622.10
3	34,798	741.80	624.10
4	34,812	733.90	616.40
5	34,742	742.70	621.60
6	34,733	748.10	625.70
	mean	738.73 ± 6.87	621.53 ± 3.34
	median	739.40	621.85

--lazy-mode on (demand-paged mmap)

run	prompt tok	radix on	radix off
1	34,865	214.40	196.80
2	34,661	216.30	216.70
3	34,798	247.40	233.50
4	34,812	199.60	196.30
5	34,742	227.60	218.00
6	34,733	225.60	217.90
	mean	221.82 ± 16.01	213.20 ± 14.31
	median	220.95	217.30

Summary:

--lazy-mode	radix on	radix off	effect of radix
auto (resident)	802.72 ± 2.50	651.98 ± 10.38	+23.1%
on-direct	738.73 ± 6.87	621.53 ± 3.34	+18.9%
on (demand-paged)	221.82 ± 16.01	213.20 ± 14.31	+4.0%, inside noise

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: AI was used to partially assist while making the code edits and to understand the contexts of the code base.