Remove the OptimizeLittleEndian unsafe paths - #33
Conversation
|
Measured the
The "+2%" above is in instruction counts; on this wide core most of the nop cost hides in the front end, so the wall-clock gap vs the accessor form is about a point on the pure rows (1 MiB: −5.3% → −4.5%). Narrower decoders should see more of it. |
|
This change is quite a large diff to remove a small amount of unsafe. Can you talk about your motivation more? |
|
This one turned into a classic game of whack-a-mole. Originally I just wanted to remove the punning and the tag-selected big-endian fallbacks, nothing more. The lucky part: a very small change did it, and the tests stayed green. The unlucky part: it left a byte/word conversion behind on every seam, and that showed up as about 13% more instructions on a 64-byte Sum256 and 2-8% slower on the pure-Go build. Compress takes words — That is how the diff got its size, and most of it is signature changes plus the mechanical fallout across the three backends. I would argue the project comes out simpler — the five |
This continues 50f20c8 ("reduce/modernize some unsafe"): it removes the
OptimizeLittleEndianpunning and the big-endian fallbacks. Every platform now runs the same code, and moving between bytes and words no longer involvesunsafe. All of it is internal — the public API is untouched.Deleting just the punning would be a small diff but a slow one: Compress takes
*[16]uint32, so byte-holding callers would convert sixteen words per block — the very work the punning existed to skip.Instead, the boundaries move. Blocks, outputs, chaining values and keys cross the internal ABI as little-endian bytes (
*[64]byte,*[32]byte), and decoding happens once, inside the compression function. The spec already defines these byte forms — a parent's block is the two child chaining values serialized — so callers mostly just copy.That is also why the change grew past the gates themselves: every word/byte seam left on a hot path benchmarked as a conversion or a stall, and each one pushed the byte form a level deeper. The assembly is untouched; only its generated signature comments change.
Tested: the full suite with and without
BLAKE3_PUREGO, on go1.27 and tip; a real big-endian run under qemu (GOARCH=ppc64); 100k randomized asm-vs-pure and default-vs-purego comparisons plus 75s of differential fuzzing, all clean. apidiff reports no public API change, and regenerating the three avo backends reproduces the checked-in assembly byte for byte.Benchmarks on an idle m8i.large (Xeon 6975P-C), the repository's
BenchmarkBLAKE3, benchstat n=6:BLAKE3_PUREGO, EntireThe pure-Go rows track instruction counts. The kernel executes fewer real instructions than master; what grew is inline-mark nops — Go emits one per inlined call, and each
m.w(i)is two inlined calls deep (the accessor, thenbinary.LittleEndian). Dropping a level halves the delta to about +2%, whether by calling the stdlib directly:or by writing the accessor bodies as plain shifts. Hand-inlining the shifts at every use goes the other way — the loads stop merging and it measures 3× worse. I kept the accessors for the short call sites and the read-only/write-only types; happy to switch if you'd rather have the 2%.