ggml-cpu: vectorize ggml_vec_max_f32 across SIMD targets (soft-max row-max scan)

[codspeed-hq]

Summary

The attention soft-max row pass computes a per-row maximum via ggml_vec_max_f32, which in the baseline is a scalar max = MAX(max, x[i]) loop. This reduction carries a serial dependency the compiler does not auto-vectorize, so it dominated a large slice of the soft-max kernel.

This change replaces it with a running vector max folded to a scalar at the end, with explicit paths for x86 AVX512 / AVX2 / AVX / SSE2 and ARM NEON / SVE, a scalar tail for leftovers, and the scalar fallback for other targets. The GGML_USE_ACCELERATE path (vDSP_maxv) is preserved unchanged.

Why this matters here

The repository's macro (walltime) benchmarks run on aarch64 CodSpeed macro runners, where soft-max is on the attention hot path. This optimization is written through per-architecture SIMD so it accelerates both the x86 simulation benchmarks (where it was measured) and the ARM NEON/SVE walltime path in CI — not a single architecture.

Measurement (CodSpeed, CPU simulation)

Measured locally through the CodSpeed CLI on the soft_max benchmark:

Benchmark	Base	Head	Change
soft_max	1.8 ms	1.6 ms	+16.78%

No regressions in co-measured benchmarks.

Correctness

Integer/float max is exact, so the vectorized reduction is bit-identical to the scalar loop (no floating-point rounding differences from reassociation). Verified against the scalar reference for every length 0..199 — covering the SIMD body, the scalar tail, and the n < vector-width short-input case — on the AVX2, SSE2, and scalar builds; all match. The SOFT_MAX cases in test-backend-ops build and run cleanly.

[codspeed-hq]

Merging this PR will improve performance by 16.81%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 1 improved benchmark
✅ 27 untouched benchmarks

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
⚡	Simulation	soft_max	1.8 ms	1.6 ms	+16.81%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing codspeed-optim-vectorize-ggml-vec-max-f32-across-all-simd-targets-1785205186774 (92efc6f) with master (46819c9)}