dasLLAMA: M5 pp deep-dense - board v8 with every metal cell at or ahead

LAWS.md

@@ -26,3 +26,4 @@ compacted, or cited as rules.
 | 2026-08-27 | .claude/agents/archivist.md (new), utils/lint (LINT027 line gate) | ARCHITECTURE.md cleanup: "1. size - needs split by sections past 300 lines (u pick the limit...) 2. some are bunch of unrelated prose. that stuff must just go. so its an extra agent for cleanup" + "we do both this PR. i'd like to not drop anchors" - archivist agent (belonging test deletes unrelated prose, census-driven splits, anchors never dropped); 300-line lint gate on rule docs in tagged folders; both arch docs groomed in this PR |
 | 2026-08-27 | skills/internal/make_pr.md | "should we update make_pr skill?" - the comment-drain row becomes the diff-scoped harvest row (the ruled harvest regime, trial passed on dasllama_image.das); rescue-bot references retired from the flow; the format row states no folder strips comments; the tree-frozen-during-preflight rule lands from this session's lib/-hiding incident |
 | 2026-08-27 | .claude/agents/ (rescue-bot, rescue-sweep-bot -> history/agents/), strip-advisory strings, .claude/hooks/README.md | "yes, lets retire both. they go to /history/agents i guess. this is evolution" - both rescue bots archived; the dormant strip advisory names the harvester as successor |
+| 2026-08-27 | REVIEW_COMMON.md (the [arch] audit triangle) | "1. if function with [arch(section)] is modified, other functions which link that section are audited. 2. if [arch(foo#section)] is modified, section gets audited to not go stale. 3. if foo#section is modified, all linked functions are audited." + on scope: "repo-wide from day one." - three constitutional rules after the ships-with-tests clause; one pass per anchor discharges all duties, no cascade |

REVIEW_COMMON.md

@@ -27,6 +27,19 @@ reachable branch ships a test that fails without it; a diff that adds a branch n
 distinguishes is a defect. The audit procedure - including how to settle "would this test fail
 without the change?" - is `skills/tdd_audit.md`.
 
+**A diff that changes a function carrying `[arch(at="<doc>#<anchor>")]` - beyond comments -
+audits the anchor's other citing functions and the cited section: read the section, check it
+still describes the code and each citer still conforms, verdict per function.** The citers of
+one anchor share one mechanism; MCP `arch_sites` lists them.
+
+**A diff that adds, removes, or retargets an `[arch(at=...)]` citation audits the cited
+section against the code - both sections on a retarget.** The citation claims the section
+describes this function; verify it does.
+
+**A diff that changes an anchored section audits every function citing that anchor.** One
+audit pass over an anchor's section text and citer set discharges every audit duty the diff
+triggers on that anchor - the duties never cascade.
+
 **A rule that a test, a lint, or the folder's `REVIEW.das` enforces is deleted.** Automation
 replaces the rule; the checklist keeps at most "weakening that check is a defect." A rule
 that COULD be automated is a lint or `REVIEW.das` candidate - say so in the review round.

modules/dasLLAMA/ARCHITECTURE.md

@@ -13,7 +13,7 @@ Shipped-arc plan docs live in `history/dasLLAMA/` (the archive log is `history/R
 a `*_plan.md` in this folder is an ACTIVE arc's working plan. Passages retired from this
 document are archived in `history/dasLLAMA/architecture_retired_notes.md`.
 
-Seven companions carry this document's sections, each keeping its sections' numbers. The
+The companions carry this document's sections, each keeping its sections' numbers. The
 routing block under each numbered heading below is the index: it names every companion that
 section runs to, and the sections that companion holds.
 
@@ -38,8 +38,10 @@ re-transcoding `$LCPP/src/unicode-data.cpp`).
 
 - `ARCHITECTURE_IMAGE.md` - sec.2.1-2.1g: the prepared-image rail.
 - `ARCHITECTURE_GPU.md` - sec.2.2b: the tensor-GEMM and fused-attention shapes that measured out.
-- `ARCHITECTURE_RUNTIME.md` - sec.2.2-2.4, 2.6-2.9, 2.11, 2.12: kernel shape, caches, lint
-  policy, knobs, coverage, the GPU ramp.
+- `ARCHITECTURE_GPU_PREFILL.md` - sec.2.2c-2.2i: the Metal prefill driver's GEMM form ladder,
+  dev-W knee map, attention slab, MoE bucket rail, and chunked submission.
+- `ARCHITECTURE_RUNTIME.md` - sec.2.2, 2.3, 2.3a, 2.4, 2.6-2.9, 2.11, 2.12: kernel shape,
+  caches, lint policy, knobs, coverage, the GPU ramp.
 - `ARCHITECTURE_MEASUREMENT.md` - sec.2.5, 2.10: the benchmark rig, the tune gate, and the
   sanctioned instrumentation rails.
 

modules/dasLLAMA/ARCHITECTURE_GPU.md

@@ -175,12 +175,11 @@ sync duty is `REVIEW_GPU.md`'s.
   3.5x slower - the mixed-int combinations exist in MPP's type lists but lower off the NAX
   fast path. W8A8 claims from other stacks do not transfer through MPP.
 
-**Sanctioned float-A stamps** (the licensed set of the rule that a `matmul2d` operand reaches
-the op as `float` only in a class stamped `[metal_kernel(float_a_ok=true)]`, each carrying
-that stamp): every tensor template's `XT = float` stamp - the live fallback wherever the half
-panel is absent (below the convert row floor, panel does not fit, half-X pinned off) - and the
+**Sanctioned float-A stamps** - the kernel classes stamped `[metal_kernel(float_a_ok=true)]`:
+every tensor template's `XT = float` stamp - the live fallback wherever the half panel is absent
+(below the convert row floor, panel does not fit, half-X pinned off) - and the
 batch-decode/classifier `MetalQ8GemmTensorT` family, whose half-X extension is an open ledger
-item. A float A operand anywhere else is the defect the rule names.
+item.
 - **Fused single-kernel attention (scores in threadgroup, online softmax):** loses 10-80% to
   the pipelined three-pass at real shapes (`benchmarks/attn/bench_metal_pf_fused_attn.das`) -
   Metal's cross-kernel pipelining plus full-width softmax beat tg-scope fusion.
@@ -189,4 +188,3 @@ The positive laws these races established - half operands, stage-only-to-transfo
 consecutive staging runs, relaxed_precision always - are `REVIEW_GPU.md` rules and the
 `modules/dasMetal/REVIEW.das` descriptor gate; this section keeps only the refuted shapes
 and why they lose.
-

modules/dasLLAMA/ARCHITECTURE_GPU_PREFILL.md

@@ -0,0 +1,172 @@
+# dasLLAMA Architecture - the Metal prefill driver
+
+Companion to `ARCHITECTURE.md`; section numbers are that document's.
+
+### 2.2c The prefill GEMM form ladder {#prefill-gemm-ladder}
+
+Every weight GEMM in `dasllama/dasllama_metal_prefill.das` splits its rows across two decisions.
+First the GEMV tail peel (sec.2.2e) takes up to `MM_TAIL_MAX` remainder rows off the padded tile.
+The rows that remain pick one of four forms, in this order, per site per forward:
+
+1. **tall in-kernel-dequant (K-quant deep class)** - a K-quant site whose f16 panel would
+   reach `TALLKQ_MIN_PANEL` (96 MiB) is DRAM-resident, and there the `KqMulMm*TH128` stamp
+   wins: it dequants its own W tile in threadgroup memory and reads the quant plane
+   (~0.56 B/element) once per 128-row tile, where a materialized f16 panel writes
+   2 B/element and re-streams them per tile. The knee is panel SIZE, not tile count - a
+   118 MiB panel wins 15-25% on the tall stamp while an 84 MiB one measures a small loss
+   (`benchmarks/lcpp_bench.das -p 512 -n 0 -r 5 --ngl 99` A/B with the gate forced per run,
+   gemma-4-12B ffn and Qwen3.8-27B qkv carriers, m5).
+   The 32-row TH stamp covers the row remainder at its X/y offsets.
+2. **dev-W all-device** - the weight plane is dequantized into a device f16 panel and multiplied
+   half x half. No threadgroup staging and no barriers, so the staged-operand tax is gone; the
+   dequant pass is paid once per site per forward against a GEMM that re-reads the f16 W panel
+   once per M row tile (`mp/128` on the tall stamp, `mp/32` on the 32-row stamp, where `mp =
+   ceil32(npos)` is the padded row count). sec.2.2d carries the panel size rules.
+3. **tall 128-row M-tile** - the stamp streams W `M/128` times over a 128-row tile, taken on the
+   row count's 128-floor with the 32-row stamp on the remainder. The remainder arm strides X by
+   `kdim`, so a caller that passes no `kdim` takes the tall stamp only when its row count is
+   already a multiple of 128, and otherwise stays whole-dispatch 32-tile.
+4. **32-row tile** - the default stamp.
+
+**Half operands ride an f16 activation panel.** One pass converts the f32 panel; the GEMM then
+re-reads it at half the bytes `d/64` times, so the convert amortizes above a row floor
+(`CVT_MIN_ROWS`, default 256) and loses under it. Two panels alternate, so one site's GEMM reads
+panel A while the next site's convert writes panel B. A site whose producer kernel writes the
+f16 copy alongside its f32 output takes its panel from `pf_twin_panel` directly instead of
+`pf_cvt_panel`, skipping the row floor entirely, because there is no convert pass to amortize.
+
+**The bf16-A stamp widens without dequantizing** - a bf16 row widens by an exact bit shift and
+rounds to the f16 tile - so the E-series `per_layer_model_proj` GEMM serves straight off the
+kept-bf16 blob with no resident f32 copy.
+
+**The occupancy floor guards the tall stamp.** A tall grid is taken only when
+`rows/128 * (d/64) >= TALL_OCC_FLOOR` (default 64, a sidecar knob). An under-occupied tall grid
+starves the GPU and small prompts regress hard without the floor.
+
+### 2.2d The dev-W panel knee map {#devw-panel-knees}
+
+Panel SIZE dominates the dev-W decision, not shape - each threshold is a knee, the panel size at
+which the dev-W win flips sign. A panel at or under `DEVW_SMALL_PANEL` (32 MiB) engages at every
+row count while its output width is at or under `DEVW_WIDE_N` (4096), and from
+`DEVW_WIDE_N_ROWS` (512 rows) when wider; a panel up to `DEVW_BIG_PANEL` (48 MiB) engages from
+`DEVW_BIG_ROWS` (2048 rows); above that the panel either splits into N-column tiles or the site
+declines. The mechanism: the f16 panel's W stream is 2 bytes an element against the q8 blob's
+1.06 once it leaves cache, and a wide-N panel costs more dequant work, which only a large row
+count repays. The raced evidence
+(32 MiB wins everywhere, 47 MiB only from 2048 rows, 112 MiB loses everywhere) is
+`benchmarks/matmul/bench_metal_nax_probe.das`'s grid.
+
+Clauses the isolated grid cannot see, because it races one site while production overlaps
+sites:
+
+- **A long-K (down-projection) dequant serializes** on the panel pair behind the up/gate GEMM
+  chain at small M, so it loses end to end there despite winning its isolated site race. The
+  long-K floor is 1024 rows on q8; the same tiling that wins its isolated site race at 512
+  rows measures zero to negative end to end (`../benchmarks/lcpp_bench.das -p 512 -n 0` A/B
+  with the form forced per run).
+- **An over-knee panel runs as N-column TILES**, each under the small-panel knee, with the tile
+  count bounded by `DEVW_MAX_TILES` (32), divisibility, the small-panel knee and the pool.
+  Narrow tiles measure fine in the same A/B. A K-quant site whose panel reaches
+  `TALLKQ_MIN_PANEL` leaves dev-W entirely for the tall in-kernel-dequant stamp (sec.2.2c
+  form 1).
+- **A k-quant tg fallback is slower than the q8 half-panel form**, so a k-quant site lowers the
+  over-knee bar, the tiled-rows floor, and the long-K floor (1024 rows to 512): a tiled read
+  still beats THAT fallback.
+
+`CVT_MIN_ROWS`, `TALL_OCC_FLOOR` and `DEVW_SMALL_PANEL` are box-raced and cached at init from
+the sidecar (`metal_cvt_min_rows`, `metal_tall_floor`, `metal_devw_small_panel_mb`); the other
+knees are fixed.
+
+### 2.2e The GEMV tail peel {#gemv-tail-peel}
+
+A prefill panel pads to `mp` rows, so `npos % 32` rows of every GEMM are padding. Up to
+`MM_TAIL_MAX` (8) remainder rows peel off the padded tile onto the fixed-B mv family instead;
+above that the padded tile is cheaper than three or more weight streams. One peeled row rides the
+reduction-split GEMV, two or more ride the b4 form only - the reduction-split GEMV walks per
+block and needs `kdim % 32`, while the b4 form - the batched fixed-B mv stamp, up to four rows a
+dispatch (`enc_mv_b4_c`) - reads whole 128-quant rounds and needs `kdim % 128`.
+
+### 2.2f The prefill attention slab {#prefill-attn-slab}
+
+Prefill attention is a three-kernel pipeline over one per-head f16 score slab padded to
+`np32 = ceil32(npos)` columns: QK writes the raw scores half once, rowstat mints each row's max
+and reciprocal sum, and AV applies `exp` while it stages P. The slab is written once and read
+once, and no separate softmax pass touches it. Every stage and every stamp of every stage binds
+ONE `AttnArgs` value, derived once per layer, and ignores the fields it does not read. Two forms
+serve it: the tiled QK/AV GEMM pair (the default, needing `head_size % 64` on BOTH attention
+classes) and the scalar 32x32 trio, which serves when that gate fails or `DASLLAMA_METAL_ATTN=0`
+pins it.
+
+- **A pad row of K or V stages as 0.** `pf_p_weight` zeroes P columns past each row's live
+  length exactly, so the P side needs no guard; the K and V sides do.
+- **The PADFREE stamps drop that guard**; an encoder picks one only where the whole WALK stays
+  inside live rows - `qoff + qrows == npos` as well as the divisibility. A padded query chunk
+  walks pad-query tiles past `npos`, where a real row's read poisons the tile.
+- **A block skip lifts to `uend` only where the tile holds span rows.** A causal-only tile above
+  the uniform span keeps its short causal walk, and a tile below every row's sliding window is
+  skipped whole because `pf_p_weight` zeroes P over the skipped region.
+- **The softmax scale rides the staged K side, not Q**, because `(q*s).k == q.(k*s)` and Q streams
+  from device raw.
+
+**Two attention classes.** The config fields describe the GLOBAL class and the `_swa` twins the
+SLIDING class; the loader guarantees at most two. Buffers size to the class maxima and each layer
+binds its own class's uniform set - a uniform model leaves the sliding twins null and binds the
+base set everywhere.
+
+### 2.2g The prefill MoE bucket rail {#prefill-moe-buckets}
+
+Routing is atomics-free: a router GEMV and a select pass, then a per-expert count kernel, then
+one bucket kernel that computes the padded prefix and fills the buckets. Each expert's bucket
+PADS to a whole 32-row tile, every threadgroup computes the same padded prefix, and threadgroup
+`e` publishes `basep[e]` for the mm and activation consumers. The bucket fill splits the entry
+range into contiguous ascending per-lane chunks and scans the chunk counts, which reproduces the
+serial entry order exactly, so the ordered weighted reduce is bit-stable against the CPU path
+that parks each routed expert's rows and reduces them in entry order. The selection is read
+GPU-side by the kernels; nothing reads back to the CPU, so encode-ahead and speculation stay
+compatible.
+
+Pad rows inside each expert's padded bucket carry a stamped sentinel and the reduce never
+references them; rows past the last expert's stamped tail are unstamped stale pool bytes, which
+is why validity tests compare the per-row entry against the live count, never the sentinel. The gather-X pass
+copies the bucket's token rows into a CONTIGUOUS f16 panel with pad rows zeroed, which lets the up
+and gate sites ride the contiguous tensor twins instead of the in-kernel gather form; the panel is
+minted once per layer and shared by both sites. An X read through the bucket index can never
+form a tensor view, which is why every tensor twin of the MoE family serves contiguous rows
+only.
+
+**The staging form that wins inside the gathered mul_mm kernels is per format, not universal.**
+The gathered q8 form carries its scale and quant pointers across k-blocks; the stateless index
+form measures 3.4-3.6% slower (`benchmarks/matmul/bench_metal_moe_lab.das`, gmm8 section). The
+gathered Q6_K is the opposite: a superblock-scalar cache measures 2.4% slower per mm than
+reloading per k-block (same lab, gmm6 section), so its stage is stateless.
+
+### 2.2h Pad rows and cooperative-op constraints {#prefill-pad-rows-and-coop}
+
+Activation panels size to `mp` rows because every kernel's M grid divides `mp` by
+its tile height - 32 for the default stamp, 128 for the tall stamps; a 64-row pad would bill a
+dead 32-row GEMM block on every short prefill. The GEMM has no edge
+masking, so pad rows are written with whatever the tile computes. That is safe because C-block
+rows are independent and no pad row is read back: the norm, rope, attention and elementwise
+kernels all bound at `npos`, and rowstat writes `[0, npos)` only.
+
+A continuation chunk (`start_pos > 0`) attends the session's existing rows: the K/V panels hold
+`[0, start_pos)` gathered rows plus the chunk at the `start_pos` offset, keys pad to the QK key
+grid's 64-tile, and the score slabs widen to `nk64` columns - the key rows padded to that
+64-tile - while the rows stay the `mp` queries.
+
+Cooperative matmul ops shape the kernel bodies: the accumulate loop is spelled ROLLED over
+matrix arrays with pointer tile bumps, because the hand-unrolled spelling hoists sixteen tile
+addresses into loop-lifetime registers and costs an occupancy tier (measured `max_threads` 704).
+
+### 2.2i Chunked submission and interleaved readback {#prefill-chunked-submit}
+
+A prefill encodes into `DASLLAMA_METAL_NCB` command buffers (default about four layers each) and
+commits each chunk as soon as it is encoded, so the scheduler analyzes chunk k while chunk k-1
+executes; `DASLLAMA_METAL_UNRETAINED=1` drops per-dispatch retain/release on top. Inside a chunk
+the `kn_*` calls record a step graph that `graph_flush_sched` replays on a CONCURRENT encoder,
+which auto-schedules and inserts barriers only at real hazards (`DASLLAMA_METAL_SCHED=0` keeps
+capture order, `DASLLAMA_METAL_PF_CAPTURE=0` is the serial-encode rollback).
+
+Completion and readback interleave: chunks complete in commit order and each completed chunk's
+roped-K and raw-V rows stream into the CPU K/V codec while later chunks keep the GPU busy. The
+residual-stream copy waits for the LAST chunk.

modules/dasLLAMA/PERF_LEDGER.md

@@ -1066,3 +1066,10 @@ group; wording kept.
   paste funnel; the server replaces that funnel.
 - **OPEN - v2: select-two compare widget (maybe)** on the leaderboard, and publishing the
   depth/batch test variants (`pp512@d4096`, `tg128@b4`, ...) that stay internal-only under v1.
+
+### From the prefill-driver harvest (2026-08-27)
+
+- **OPEN - MoE k-quant twin tune families.** The moe k4/k5/k6/q51 tensor twins are gated by
+  the DENSE k-quant crowns rather than raced on their own; give each its own
+  `metal_tensor_race` family and crown so the gate stops proxying. Instrument:
+  `metal_tensor_race` on an M5 box, one q8 and one k-quant MoE model.