Vectorize ggml_vec_max_f32 with AVX2 for the soft-max row-max scan

[codspeed-hq]

Summary

Vectorizes ggml_vec_max_f32 (in ggml/src/ggml-cpu/vec.h), the row-maximum reduction used by scaled masked soft-max (ggml_compute_forward_soft_max_f32) — the normalization step at the heart of attention.

Analysis

Flamegraph analysis of the soft_max benchmark showed ggml_vec_max_f32 accounting for ~12% of self time, at ~100% instructions. The scalar reduction max = MAX(max, x[i]) is a serial dependency that the compiler does not auto-vectorize (the a>b?a:b form has NaN/signed-zero semantics that block a plain fmax reduction), so it processes one element per iteration over every attention-score row.

Change

Reduce eight lanes at a time with a running _mm256_max_ps accumulator, then fold to a scalar with a short SSE reduction, keeping a scalar tail for the remainder. A scalar fallback is retained for non-AVX2 targets and the Accelerate path is unchanged.

Correctness

For the finite inputs soft-max operates on (the code already asserts !isnan(wp[i])), the vectorized reduction returns a bit-identical maximum. Verified with a standalone A/B harness over 100,000 rows of varying lengths: 0 mismatches vs. the original scalar loop.

Performance (CodSpeed, CPU simulation)

Measured with the repository's benchmarks/ggml-benchmarks suite in CPU-simulation mode:

Benchmark	Base	Head	Change
soft_max	1.83 ms	1.57 ms	+16.78%

The other 14 benchmarks are unchanged — no regressions.

[codspeed-hq]

Merging this PR will improve performance by 16.81%

⚡ 1 improved benchmark
✅ 14 untouched benchmarks

Performance Changes

	Benchmark	BASE	HEAD	Efficiency
⚡	soft_max	1.8 ms	1.6 ms	+16.81%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing codspeed-optim-vectorize-ggml-vec-max-f32-with-avx2-for-the-soft-1785131252404 (4854863) with master (048bb65)}