ggml-quants: vectorize make_qkx2_quants scale search with AVX2/FMA

[codspeed-hq]

Summary

quantize_chunk[q4_k] is the most expensive micro-benchmark in the ggml suite, and the flamegraph shows ~90% of its time is spent in make_qkx2_quants — specifically the scale-search sweep that runs nstep+1 (21) times per 32-element block, recomputing nearest_int(iscale*(x[i]-min)) and three weighted sums for every element. It is ~99% instruction-bound and was entirely scalar.

This change adds an AVX2/FMA vectorized implementation of that inner sweep (make_qkx2_sweep_avx2), processing 8 elements per iteration and accumulating sum_l, sum_l2, sum_xl in vector registers before a horizontal reduction. _mm256_cvtps_epi32 uses round-half-to-even, matching nearest_int() over the clamped range, so it produces the same quantized levels.

Key implementation detail

ggml-quants.c is compiled into ggml-base without -mavx2/-mfma (those flags are only on ggml-cpu), so a plain #if defined(__AVX2__) guard would compile to nothing. The helper is therefore isolated with #pragma GCC target("avx2,fma") + __attribute__((target("avx2,fma"))) so the intrinsics compile regardless of the TU baseline ISA, and it is only entered after a runtime __builtin_cpu_supports("avx2") check. Non-x86 and pre-AVX2 CPUs keep the untouched scalar fallback, so portability is preserved.

Correctness

• test-quantize-fns passes for all quant types (round-trip error within thresholds).
• A standalone bit-identity harness comparing the scalar and AVX2 sweeps over 10,000 randomized blocks (n = 16/32, nmax ∈ {3, 15, 31}) reported 0 level mismatches — the vector path produces bit-identical quantized levels to the scalar path.
• The three weighted sums differ only in floating-point summation order, which can produce last-bit differences in the heuristic scale search for some quant types (numerically equivalent, no quality change). Q4_K output is bit-identical.

Measured impact (CodSpeed simulation)

Benchmark	Base	Head	Change
quantize_chunk[q4_k]	10.4 ms	6.1 ms	+69.2%

No regressions across the other benchmarks (q4_0, q4_1, q5_0, q6_k, q8_0 unchanged).

Base and head CodSpeed runs executed on different CI runners; the ~69% gain is far larger than any environment-induced variance.

[codspeed-hq]

Merging this PR will improve performance by 64.88%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 1 improved benchmark
✅ 27 untouched benchmarks

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
⚡	Simulation	quantize_chunk[q4_k]	10.4 ms	6.3 ms	+64.88%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing codspeed-optim-vectorize-the-make-qkx2-quants-k-quant-scale-searc-1785161336020 (7496180) with master (46819c9)}