Optimize AVX2 ggml_vec_dot_q8_0_q8_0: hardware F16C scale conversion + 2-accumulator unroll

[codspeed-hq]

Summary

Optimizes the AVX2 path of ggml_vec_dot_q8_0_q8_0 (ggml/src/ggml-cpu/arch/x86/quants.c) — the q8_0×q8_0 dot product that drives every q8_0 matrix multiplication (attention/FFN projections and the LM head).

Two changes in one kernel:

1. Hardware F16C scale conversion. Each block's two FP16 scales (x.d, y.d) were converted to FP32 through the generic 64K-entry lookup table (ggml_lookup_fp16_to_fp32). Flamegraph analysis of the macro benchmarks showed this table access as a hot spot inside the dot product. On x86 with F16C the conversion is a single vcvtph2ps instruction, so the kernel now uses it directly (scoped, local #define, falling back to the lookup table when F16C is unavailable).

2. Two-accumulator unroll. The main loop processed one block per iteration with a single accumulator, so the per-block _mm256_fmadd_ps formed one serial dependency chain (latency-bound). It now processes two blocks per iteration into two independent accumulators, summed once at the end, hiding FMA latency on out-of-order cores. A scalar tail handles an odd trailing block.

Correctness

The vcvtph2ps conversion produces the exact IEEE-754 half→single value (identical to the lookup-table entries for finite inputs), and splitting the float accumulation into two chains matches the reordering already used by the sibling AVX kernel in the same file. Verified with the repository's test-quantize-fns suite (built with -DGGML_NATIVE=ON, i.e. exercising the AVX2/F16C/FMA path) — all quantization types, including q8_0, pass with no errors.

Performance (CodSpeed, CPU simulation)

Measured in-sandbox on the q8_0 macro benchmarks with matched base/head builds, two runs each. Under CPU simulation the cache-miss metric is the most stable signal; instruction counts have some run-to-run variance because the graph compute is multi-threaded, so values below are averaged over two runs.

Benchmark	cpuTotal (base → head)	Cache-miss (base → head)
prompt_layer[q8_0]	1.0208 s → 0.9848 s (−3.5%)	0.09242 s → 0.08312 s (−10.1%)
lm_head[q8_0]	0.1776 s → 0.1691 s (−4.8%)	0.01421 s → 0.01302 s (−8.4%)
decode_layer[q8_0]	0.1702 s → 0.1710 s (within noise)	~unchanged

decode_layer[q8_0] sits within run-to-run noise: it is a single-token decode where the dot product is only ~21% of the benchmark and dominated by the (unchanged) qs loads, so this kernel change has little leverage there. No regressions on the other 12 (non-q8_0) benchmarks.

The 2-accumulator unroll additionally targets FMA latency, which the CPU-simulation instrument does not model — its benefit is expected to show up on the walltime macro runners in this PR's CI.

[codspeed-hq]

Merging this PR will not alter performance

✅ 28 untouched benchmarks

---

_{Comparing codspeed-optim-optimize-avx2-ggml-vec-dot-q8-0-q8-0-hardware-f16c-1785222663860 (5e42378) with master (46819c9)}