CUDA: support PrismML Bonsai 2 (prism.hadamard weights + PQ2_0/PTQ1_0 types)

[Andgihat]

What

Makes PrismML's Bonsai 2 models run on BeeLlama. Two independent pieces are needed, one per commit:

1. prism.hadamard folded weights. Bonsai 2 ships weights pre-folded with a normalized Sylvester–Walsh–Hadamard rotation, so the activation has to be rotated by the same transform before every folded matmul. Without it the model loads happily and produces garbage.
2. The PQ2_0 / PTQ1_0 quant types. The ggufs on the main PrismML repo use two types that are not in upstream ggml: PQ2_0 (id 142, group 128, 2.13 bpw) and PTQ1_0 (id 143, group 128, 1.75 bpw, trits packed base-3 five per byte). Before this only the dev-repo Q2_0 file (id 42, group 64, supported since CUDA: add Q2_0 (ternary, group-64) weight support #96) would load.

Provenance

Ported from PrismML's own llama.cpp fork, the same way #96 was done. Kernel and format design credit is theirs; this adapts it to the v0.4.6 tree.

Changes

Hadamard (126fa16)

• llama-model: parse and validate the prism.hadamard.* metadata block; build one F32 rotation per (block size, buffer type) plus the per-width sign vectors, in the same buffer as the folded weights.
• llama-graph: apply sign flip and rotation to the activation in build_lora_mm / build_lora_mm_id, memoized per (activation, rotation); apply the inverse after the token-embedding lookup.
• llama-context: a one-time graph check that every folded weight is consumed through its transform — turns a silent wrong-math path into a load error.

Quant types (dd614c2)

• Block layouts, traits, reference quantize/dequantize.
• CPU vec_dot: generic path plus the x86 VNNI and ARM SIMD variants for PQ2_0, with arch-fallback aliases for every other target. PTQ1_0 has no SIMD variant on any arch in the source fork either, so it aliases the generic one everywhere.
• CUDA: dequantize, convert, get_rows, MMVQ and MMQ kernels — including the PTQ1_0 multi-column path that shares one 128-trit decode across up to three columns — plus the Ampere MMQ tuning table and the two template instances.
• ftype plumbing: gguf, model loader, quantize tool, gguf-py.
• gguf.cpp also recognises a Q2_0 file that is really in the legacy group-128 layout and points at the PQ2_0 build, instead of failing with a bare tensor-offset mismatch.

Deliberately not ported: the DGX Spark (GB10) tile double-buffering and L2-prefetch paths, which have no counterpart in this tree, and the Metal / Vulkan / CDNA / RDNA backends. CUDA only, same scope as #96.

Verification

RTX 5060 Ti (sm_120), CUDA 12.8, -DGGML_CUDA_FA_ALL_QUANTS=ON -DGGML_BACKEND_DL=ON -DGGML_CPU_ALL_VARIANTS=ON.

• test-backend-ops -o MUL_MAT: 94 OK / 0 FAIL for the two new types against the CPU reference. The remaining cases are type_b=f16, reported not supported by both CUDA and CPU — the same split the existing Q2_0 shows (47 OK / 33 not supported per type).
• test-backend-ops -o GET_ROWS: 8/8 OK.
• Ternary-Bonsai-2-27B-PQ2_0.gguf (7.21 GB, 402 tensors of type 142): loads as PQ2_0 - 2.13 bpw (group 128), loaded 402 Hadamard-folded weight(s), answers correctly at 302 t/s prefill / 47.3 t/s generation.
• Ternary-Bonsai-2-27B-PTQ1_0.gguf (5.95 GB, 402 tensors of type 143): loads as PTQ1_0 - 1.75 bpw (group 128), answers correctly on both a short prompt (171 t/s prefill / 42.4 t/s generation, MMVQ path) and a ~380-token one (357 t/s prefill / 41.1 t/s generation, MMQ path, which this type takes at every batch size).
• No regression on the existing type-42 group-64 file: same answer, 45.5 t/s before and after.
• For scale, the same τ²-bench run against a non-ternary 27B on the same machine, split and simulator: Qwen3.8-27B-UD-Q3_K_XL scores 0.950 vs 0.750 here. The gap sits almost entirely in mms_issue (5/6 vs 2/6); both are 6/6 on service_issue and within one task on mobile_data_issue. That is the quantisation/model trade, not a porting defect — kernels match the CPU reference exactly (see above).
• Note on the τ²-bench figures above: they are from the small split, where every task has one fault per task. The splits behind published leaderboard numbers are much harder (base averages 4.25 faults per task, test 4.08), so these are useful for comparing the two local models under identical conditions, not for ranking against the leaderboard.

MTP fix (477afefd)

• The MTP graph does its own embedding row lookup and never restores the primal basis, while the trunk does, so a Bonsai 2 gguf with an embedded MTP block fails the graph check with Hadamard-latent table 'token_embd.weight' is read without the inverse transform. The inverse is factored out of build_inp_embd into build_hadamard_inverse_embd and called from both paths, against whichever table was actually read. Same change as qwen35: apply the Hadamard inverse to the MTP token-embedding lookup PrismML-Eng/llama.cpp#205. No-op for models without folded weights.
• With it, ProCreations/Ternary-Bonsai-2-27B-MTP loads and decode goes 37.4 → 71.1 t/s (--spec-draft-n-max 8, acceptance 0.247, mean draft length 2.98). The upstream launcher's default of 2 gives only 61.7 t/s here.

[dwizzle204]

Tested #169 on an RTX 5060 Ti 16 GB with Ternary-Bonsai-2-27B-PQ2_0.gguf (3907dc16…62ec1) using -ngl 99 -fa 1 -ctk q8_0 -ctv q8_0.

Bee: dd614c290
Prism: 7dffb158d

Early results are solid.

Correctness

• PQ2_0: 47 OK / 0 FAIL
• PTQ1_0: 47 OK / 0 FAIL
• merge_intervals: 5/5
• V2-A contracts: 44/44
• V2-B: 4/7, same T4/T6/T7 failures as Prism
• long-context needle: PASS

No quality regression observed from the port.

Performance

Back-to-back, same GGUF and flags:

	Prism	Bee #169
pp512	947.2 t/s	800.2 t/s
tg128	45.81 t/s	41.86 t/s
VRAM @ 131K	14,469 MiB	14,012 MiB

Bee is ~15% slower on short prefill and ~9% slower on decode, but uses ~457 MiB less VRAM.

Reasoning control

This is the main gain for my workload.

With unbounded reasoning, Bonsai had several cases that consumed very large reasoning budgets without returning a usable final answer.

Under Bee with an 8K reasoning budget, the same 50-question Spatial/Zebra set finished 50/50:

• Spatial: 100%
• Zebra: 100%

The prior unbounded Bonsai result on the same set was 98.0%.

One representative case went from ~414s without a usable answer to ~66s with a correct answer.

So far #169 looks like a clean Bonsai 2 port, with the main practical benefit being Bee's reasoning controls rather than raw throughput.