perf(speculative): recover MTP throughput on Apple GPU generation 15+ without giving up temperature-0 byte-identity

[inureyes]

Problem / Background

The exactness probe from #1186 fails closed on Apple GPU generation 15 and newer, which is exactly where MTP pays off.

Measured on an M5 Max 128 GB with the Qwen3.8-27B 4-bit target plus its bf16 MTP drafter: block 3 is 1.19x classic decode (39.37 against 32.96 tok/s, medians of 5 interleaved repetitions with 90 s cooldowns), and the target's own multi-token forward amortizes at T=4/T=1 = 1.29. On an M1 Ultra the same pairing is 0.70x.

So the honest position after the gate change in #1186 is that byte-identity holds where MTP is slow and fails where MTP is fast, and declining is a real throughput loss on the newer hardware. The decision recorded for this project is to keep the contract and research the speed, not to restate the contract to accommodate the slowness.

Mechanism to Attack

The divergence on generation 15+ comes from qmv_wide, which MLX selects for M >= 2 affine quantized matmuls when arch_gen >= 15 (see https://github.com/ml-explore/mlx/blob/main/mlx/backend/metal/quantized.cpp). qmv and qmv_wide reduce over K with different lane counts, so their results differ in the last ulp. The verify block is the only place that runs M >= 2.

Proposed Solution

Directions worth measuring, in rough order of cost:

1. Measure what qmv_wide actually buys on the verify shapes. If forcing the plain qmv reduction at M = K costs only a few percent on a memory-bound 27B target, exactness is recoverable almost for free on generation 15+. Note that dispatch_qmv already skips qmv_wide when a global scale is present, which is one existing lever, and an upstream switch to force qmv would be another. Report the measured verify-forward delta at block 3 and 4.
2. If a forced-qmv verify is viable, wire it behind the probe. Engage it only where the probe would otherwise decline, and re-run the probe to confirm the arms converge.
3. Account for the drafter. Independently of exactness, the drafter is now the binding constraint on this hardware (see perf(speculative): cut the MTP drafter's per-step cost, now the binding constraint on M5-class hardware #1185): about 10.5 ms per drafter step against a 30.3 ms full target decode step, roughly 149 GB/s effective against the target's 530 GB/s, and about 26% of the round is drafter overhead. Whatever recovers exactness should be measured against a round whose drafter cost has already been addressed, or the two efforts will each look smaller than they are.

Acceptance Criteria

Follow the repository's "Performance issue completion criteria" in the contributor guidelines:

• Measured through the production call path rather than the op.
• The two arms are proven to differ before any number is reported (counter, metric, or log line that can only move in one arm).
• Medians and dispersion reported over repeated runs, with the machine load recorded.
• What was not measured is named explicitly, in the closing comment and in the benchmark record.
• The benchmark record is written to docs/benchmark_results/<feature>-<hw>-<date>.md and linked from this issue.
• Whatever the outcome, the fix(speculative): MTP temperature-0 byte-identity fails on Apple GPU generation 15+ and at block widths above the qmv batch limit #1186 gate still fails closed by default.

A measured "no, forcing the exact kernel costs too much" is a complete answer and closes this issue, provided the number is recorded and the gate keeps failing closed.

[inureyes]

Closed by #1199. MTP now engages on Apple GPU generation 15 and later with the temperature-0 byte-identity contract intact, by re-probing with qmv_wide disabled and keeping it off when that is what makes the verify block exact. Measured 1.04x classic decode on M5 Max against 1.00x today and 1.16x with the contract forfeited. Two follow-ups named in the PR: scoping the switch to the verify forward so a co-resident server does not pay it on batched decode, and #1200, which currently prevents confirming the contract by diffing generated text.