perf(decode): prefetch the decode index eight tokens ahead

[ArthurZucker]

Stacked on #2387. Marginal — english +3.1%, japanese flat. Opened so the measurement exists as an artifact; close it if 3% is not worth ~35 lines of arch-gated inline asm in a hot path.

What it does

After #2387's id-ordered index, decode's entire remaining cost is one random access per token into decode_off, indexed by a token id and therefore with no spatial locality to exploit. A decoder already holds every id it is about to need, so the load can be issued ahead of use: prfm pldl1keep on aarch64, _mm_prefetch on x86_64, nothing elsewhere. A hint only — it cannot fault, an out-of-range id is filtered before it, and the portable build emits nothing.

Eight ahead covers an L2 hit at ~9 ns/token without running so far ahead that the line is evicted before use.

Why only 3%

The iterations were already independent, so the out-of-order engine was overlapping these loads without help. Decoding one chunk repeatedly, so the tables collapse into L1, runs the identical loop at 4.08 ns/token against 9.08 — but that 2.2× is a cache capacity effect across the whole working set, not per-access latency sitting idle waiting to be hidden. Prefetching cannot shrink a working set.

That is also the honest ceiling statement for this loop: the remaining headroom is the size of a table indexed by the id space, and ids cannot be renumbered — they are the model contract.

Rejected on the way, with numbers

Inlining short tokens into the index (8-byte entry: tag byte + 7 payload bytes, longer tokens spilling to the slab). Flat on both corpora — and not because the path was cold:

	≤ 7 bytes (inline-eligible)	≥ 8 (spilled)
english	88.5%	11.5%
japanese	95.0%	5.0%

It does remove the second dependent load, but that load was already cheap — the slab is id-ordered, so it tracks text locality — while the entry grew 4 → 8 bytes and doubled the footprint of the one table that does miss. English touches 31,395 distinct ids, so the live index went ~126 kB → ~251 kB. The index wants to be small, not informative.

SIMD UTF-8 validation. Measured share of decode time:

	from_utf8	alloc	gather + copy
english	1.1% (0.02 ns/B)	0.8%	98.2%
japanese	27.0% (0.54 ns/B)	0.8%	72.3%

std's ASCII fast path already runs at 0.02 ns/byte, so a SIMD validator is worth nothing on Latin text. It is worth revisiting for non-Latin, where a 4× validator would buy roughly +25% on japanese.

Tests

cargo test -p tk-encode green (176 + 2). clippy and fmt --check clean. No behaviour change: the prefetch is a hint, and the sparse-id equivalence test from #2387 still gates the lookup itself.

[ArthurZucker]

Closing: measured +3.1% on english and flat on japanese, which is not worth arch-gated inline asm in a hot path. The analysis stands in the commit message and in #2387 — the remaining headroom is cache capacity on a table sized by the id space, not latency that a prefetch can hide.