Skip to content

dasLLAMA vulkan: cm2 tiles by default, fused feeds, merged k|v, device embed gather; [arch] citations go mandatory - #3896

Merged
borisbat merged 20 commits into
masterfrom
bbatkin/vkclass
Aug 28, 2026
Merged

dasLLAMA vulkan: cm2 tiles by default, fused feeds, merged k|v, device embed gather; [arch] citations go mandatory#3896
borisbat merged 20 commits into
masterfrom
bbatkin/vkclass

Conversation

@borisbat

@borisbat borisbat commented Aug 28, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Behavior change: on a device with NV_cooperative_matrix2 the Vulkan resident prefill now runs the cm2 tiles by default (DASLLAMA_COOPMAT=mm restores the old path); every vulkan .dlim baked on such a device re-mints, because its flavor tag moves from m3 to m4; the 8/16-bit SSBO storage gate now also requires core shaderInt16; DASLLAMA_VERSION is 12, so the committed sidecars are for the previous roster; DASLLAMA_VK_KVM is spelled DASLLAMA_VK_KV_MERGE. Repo tooling: the [docs] enforce_arch lint key is gone - LINT026's reverse pass (an anchor no [arch] cites) and LINT027's 300-line gate now run in every folder, and the lint walkers skip any subdirectory holding a CMakeCache.txt; an [arch] citation must name a document in the citing file's own folder tree (a far mechanism is restated in prose in the citer's tree); the PR lint's changed set now includes .md files, which arm their folder's document rules with no .das compiled.

The Vulkan prefill was well behind the reference exe on this box. The cause was not the chain shape but two spelling defects in the cm2 tiles: coopmatClamp's hand-emitted loop carried LoopMerge None, which drops the accumulator out of tensor registers for the whole kernel, and the q8 decode callback read its quants as 32-bit words where the driver's block-load path pattern-matches only the 16-bit form. With both fixed the cm2 tile beats the mm tile on every serving shape, so cm2 becomes the default where the hardware has it, and the levers only the mm path had are ported to it: the fused add+rms twins (q8 and f16 forms), an f16-out flash-attention stamp, the merged k|v GEMM, and Q4_K / Q6_K decode-in-load tiles. Three prefill levers land on both paths: chunked submits so the GPU starts while the CPU still records, a device-side token-embedding gather, and the merged k|v GEMM. Every Q8 requant writer now stores quants as bytes through the 8/16-bit store path, which this PR gives golden and live-device coverage; the hand-rolled word packing is gone.

Measured on the 5060 Ti against the reference exe, same box, back to back: llama-3.2-3B q8 prefill 6645 -> 7737 tokens/s (100.6% of the reference), tinyllama 20986 (~103%), Qwen3-4B Q4_K_M 1626 -> 3188; decode unchanged.

Where to look: resolve_coopmat_mode (the default flip); pf_run in dasllama_vulkan_prefill.das (the chunk ring, the fused-feed helper pf_qkv_feed_fused, the merged k|v arm); resident_upload / resident_plan / vulkan_embed_gpu_gate in dasllama_gpu_resident.das (the embed-gather arms mirror embed_row's ladder, the f32 table rides only the plan's headroom); the K4/K6 decode callbacks in dasllama_vulkan_classes.das; the LoopMerge line in spirv_emit.das; the K/V host-DMA placement at the end of record_at_cmd (a driver-defect mitigation, ledgered in ARCHITECTURE_GPU_VULKAN.md sec.2.2j).

The harvest's other half: every mechanism the arc filed into an architecture document is now cited from the code that implements it ([arch(at=...)], ~130 citations across dasLLAMA, dasSpirv, dasVulkan and daslib), and the comments those sections replaced are deleted. ARCHITECTURE_GPU.md split its Vulkan sections into ARCHITECTURE_GPU_VULKAN.md (2.2j-2.2p) under the 300-line gate; modules/dasVulkan/CLAUDE.md split into CLAUDE.md / ARCHITECTURE.md / REVIEW.md. REVIEW_COMMON.md gains the duty that makes this hold from now on - a fact filed from a comment lands with its anchor and its citation in the same diff - and the harvester agent's FACT verdict names the citer.

Validation, claims, ledger

Validation

  • Local-only, on the RTX 5060 Ti (driver 610.74), default path = cm2: test_vulkan_kernels.das 62/62, test_vulkan_tier.das 36/36 (both again under -jit inside the model-free suite), modules/dasVulkan/tests/integration/test_compute_features.das 6/6 on the device, tests/spirv 376/376 with spirv-val.
  • Vulkan parity with DASLLAMA_GPU=1 (log shows resident driver armed): llama-3.2-3B q8 token-exact over 40 (embed gather on the tied cls plane + merged k/v armed), Qwen3-4B Q4_K_M token-exact over 40 (the kq route).
  • The SmolLM2 vulkan dry bake (run.das --suite image-vulkan --arm vulkan) 4/4 - the settling run for the dry-bake guards.
  • dasLLAMA model-free suite under -jit: 94 of 97 files green; the three reds are box artifacts with controls - test_exchange_schema and test_site_records fail on this Windows checkout because core.autocrlf=true checks the sidecars and bench_records.json out as CRLF (an LF checkout of the same tip passes 21/21 and 2/2), and test_ple_modes passes every cell but its E2B CPU chat turn took ~55 min on the loaded box and tripped the 1200 s watchdog.
  • Full AOT sweep (test_aot -use-aot ... --test tests): 13092 tests, 13085 passed, 0 failed, 7 skipped (fixture-gated).
  • The sequence and imgui preflight gates were skipped (the preflight host pins the runtime DLL); neither is in this PR's blast radius (no type-system, generic-binding or dasImgui change).
  • Two external codex rounds, every finding verified or rejected by a fresh read.
  • A checklist audit round over the whole arc after the relink (five review-md auditors, two dragons): 10 VIOLATED verdicts landed as fixes (a section false for one citer, three dropped facts, three missing override announces, the attribution slips, the CRLF-committed kernel file renormalized to LF); 25 checklist-text defects fixed in the same batch; LINT026/027 green over all three module roots (454 files) and tree-wide (3902 files).

Claims - stated, not tested

  • The three chain-shape predicates (fused feed, fa f16-out stamp, merged k|v) were adopted on same-build two-run A/Bs - the only instrument a chain-shape change has; the probe times kernels, not chain predicates.

  • In-body tile constants (the 256/128 column widths, BK = 64, PF_CHUNK_MAX = 8) are readable in the SPIR-V dump (DASLLAMA_VK_SPV_DUMP); no dump was taken for this PR.

  • harness/vk_gemm_probe.das exits 0 on its decline paths (no device, no cm2); its rows enter no record store and nothing parses them.

  • A changed repo-root .md in a PR arms the whole tree's document rules in the changed-set lint lane - intended, a few seconds with nothing to compile.

  • The default flip's basis is the one-process interleaved rate probe (harness/vk_gemm_probe.das run_shape: cm2l 62.2/64.7 TFLOP/s vs mm 53.7/54.9 on the gate/down shapes); the e2e pairs are confirmation.

  • The device-side embed gather carries no work-size gate: a short prompt pays one gather dispatch instead of an x-panel upload. Settling run: benchmarks/lcpp_bench.das -p 8 with DASLLAMA_VK_GPU_EMBED=1 vs =0.

  • DASLLAMA_VK_FULLSG and DASLLAMA_VK_SPV_DUMP are instruments, not overrides: the first changes pipeline-creation flags, the second writes files; neither moves a served bit. DASLLAMA_VK_OVERLAP changes only when work happens.

  • Every desktop, MoltenVK and lavapipe driver reports core shaderInt16; a driver reporting the four old bits without it now loses the Vulkan tier, and the tier-disabled line names the set.

  • Mode 4 feeds the dense prefill GEMMs raw f16 activations where mode 3 fed per-block-scaled int8; post-norm activations sit far below the f16 ceiling, and the parity runs above are token-exact.

  • A name-parsed coopmat tile shape (coopmatWgAcc_f16_64x128) is not a kernel-model capability, so no dasMetal ledger line is owed.

  • The per-turn numbers in followup_vulkan.md are direction-grade (debug-jit, DASLLAMA_VK_REBAR=0, the reference build pinned); no board cell exists for the vulkan backend yet (see Not done).

Not done

  • The four K4/K6 cm2 tiles (and the pre-existing Q8 pair) are hand-stamped l/m twins; REVIEW_GPU.md asks for one class template - ruled a follow-up PR (followup_vulkan.md item 24). Riding with it: @role = "weight" on the Q8 pair's weight planes (the K4/K6 twins carry it), and a tests/spirv/ fixture for the coopmatWgAcc_f16_64x128 marker (device-tested today only through the dasLLAMA kernel suite).
  • The [arch] section-injection hook (.claude/hooks/arch_inject.py) lands here untried through the live harness beyond one session's synthetic payloads; the next session is its trial.
  • No vulkan board cell exists (gen_bench_records.das has no vulkan leg), so the served numbers above enter no record store and no row was re-minted under the flipped defaults.
  • The .dlim flavor tag carries the coopmat mode though no bake consumer reads it; the old m3 image is never reclaimed (a different lane).
  • followup_vulkan.md names the reference exe's symbols in seven lines; REVIEW.md's attribution carve-out lists three documents the ledger is not among.
  • Checklist-text defects the auditors filed against rule TEXT (semantic, not fixed here) are listed in the PR conversation.

🤖 Generated with Claude Code

https://claude.ai/code/session_015ZQUVVRi3EUqiZePdhVug7

borisbat and others added 18 commits August 28, 2026 09:12
@borisbat @claude
dasLLAMA vulkan: prefill overlap + device-side embed gather + merged …
c70c339 
…k|v GEMM

Three prefill levers on the resident driver, measured same-box vs llama-bench b10659
(REBAR=0 protocol after the mapped-heap residency incident, followup item 17):

- chunked submits (DASLLAMA_VK_OVERLAP): the window chain submits on a 1,2,4,8-layer
  ramp through a cmd ring, fence only on the last chunk - the GPU executes while the
  CPU records (llama.cpp's incremental-submit shape). tinyllama pp +2.2%, 3B +0.7%.
- device-side embed gather (DASLLAMA_VK_GPU_EMBED): the ids-form prefill seam
  (install_rdec_prefill_ids) + the engine embed gate with fail-closed CPU backfill;
  EmbGather dequants the q8 embd plane (tied models reuse the cls plane, untied q8
  places its own), EmbGatherF32 serves the untied fblob table (512MB cap, budget-
  guarded). 3B pp +6.0%, tinyllama +6.8%; tg untouched.
- merged k|v GEMM (DASLLAMA_VK_KVM): one dispatch over the adjacent k+v planes writes
  pf_kv row-interleaved (rope_kv_b/qk_rms read by kstride). A perf WASH end-to-end -
  the q,k,v level already overlaps on-device - kept for the record shortening
  (28 dispatches + 28 copies + a barrier per window gone).

vk_gemm_probe.das joins harness/ (per-shape isolated mm rates; the ngfx capture
vehicle - counters read unelevated now). Gates: kernels 52/52 (embed-gather oracle
pair added), tier 36/36, env registry 14/14, CPU==GPU token-exact on tinyllama + 3B
across every arm pair. Day-end: tinyllama 19527/293.0 vs 20277/291.4 (96.3% pp, tg
ahead); 3B 7062/105.4 vs 7691/110.0 (91.8% pp).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015ZQUVVRi3EUqiZePdhVug7
@borisbat @claude
dasLLAMA vulkan: cm2 kernel parity - coopmatClamp Unroll + the 16-bit…
6159327 
… unpack8 decode spelling

The cm2 l-tile ran 28-35 TFLOP/s where llama.cpp's identical geometry ran 62-66.
Two causes, both spelling-level:

- spirv_emit: coopmatClamp's hand-emitted per-element loop carried LoopMerge None;
  unrolled (glslang spells it [[unroll]]), the driver keeps the wg-scope accumulator
  in tensor-register form - left rolled, the dynamic per-element index demotes the
  coopmat to addressable storage for the WHOLE kernel. One word: 34.2 -> 57.8 TFLOP/s.
- The decode callback must be llama.cpp's EXACT 16-bit form: VkQ8Blk becomes
  int16[16] and decode_q8 is unpack8(qs[(cib.y&30)>>1])[cib.y&1]. Probe ladder on
  their kernel, our two-plane data: 16-bit unpack8 62.7 (their stock 63.3, the
  external scale plane ~1%); 32-bit word + shifts 55.4; 32-bit unpack8 + dynamic
  4-lane select 20.3 (VectorExtractDynamic on v4char kills the block-load path).
  The das storage-type surface already existed golden-tested (test_storage_8_16);
  the only additions are the unpack8(int16/uint16) -> byte2/ubyte2 lingua-franca
  overloads (zero emitter change) and the core shaderInt16 device feature across
  vulkan_boost's storage_8_16 creator family + gate (the Int16 capability rode
  driver leniency before).

Results (zen2 / 5060 Ti, driver 610.74, debug-jit, REBAR=0): cm2l drain-free
62.2/64.7 gate/down = par with their blob, +15-18% over mm; 3B mode-4 e2e pp
7293.9 +/- 47 BEATS mm (7058 +/- 28) = 94.8% of their cm2 build, new best; tg 104.4
(decode decoupled). tinyllama mode-4 18102 vs mm 19754 - the small shapes starve
the 128x256 l grids, so the mode-4 default flip waits on the l/m/split re-tune.

Rides along: REQUIRE_FULL_SUBGROUPS instrument (vk_fullsg, default OFF - measured
-9% on the wide mm shape), DASLLAMA_VK_SPV_DUMP (the SPV override's capture half),
vk_gemm_probe cm2/cm2x/their arms (decode bisect + llama.cpp-blob-in-our-harness
vehicles), roped-k forensics in the tier attention test (an intermittent readback
flake predating this work - prints bad-row window alignment when it fires).

Gates: kernels 54/54 oracle-exact both modes, tests/spirv 374/374, tier 36/36,
env registry 14/14.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015ZQUVVRi3EUqiZePdhVug7
@borisbat @claude
dasLLAMA vulkan: attention-chain K/V host DMA moves to the end of the…
83e0b80 
… chain

The tier attention test's roped-k readback flaked (~1-in-12): the mid-chain
K->host vkCmdCopyBuffer captured a ~70-workgroup partial prefix of at_prep's
output - with a correctly recorded, spec-valid global memory barrier
(compute|transfer -> compute|transfer, shader_write -> transfer_read) standing
between the dispatch and the copy, on one queue in one command buffer. A second
identical copy recorded at the END of the same command buffer read the same
buffer correct in the same failing run, and attention's own reads were never
affected. Driver 610.74 / RTX 5060 Ti; the in-cmd compute->transfer barrier is
observed unhonored under a race window.

Mitigation: the K/V host DMAs record after the wo GEMM instead of beside the
preps - the intervening attention/rq/wo work closes the window. The device-side
v append stays in place (attention consumes it through the rail as before).
16/16 tier-suite runs green with the move (1-3 fails per 12 before).

Rides along: roped-k failure forensics in the tier test (bad-row window
alignment, prints only on failure) and tinyllama role shapes in vk_gemm_probe
(arg "tl" - the small-model l/m tile-pick data).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015ZQUVVRi3EUqiZePdhVug7
@borisbat @claude
dasLLAMA vulkan: wave-efficiency cm2 tile pick + cm2 becomes the coop…
b14bd9a 
…mat2 default

cm2_tile_cols drops the prefer-large rule for a wave-efficiency comparison
(occupied wg slots over allocated wave slots, cross-multiplied; m only on a
strict win - ties go to l's bigger tile). Probe-fit on the 3B + tinyllama role
shapes; the pick flips gate-5632 and down/q-3072 to m and keeps every measured
l winner. tinyllama mode-4 pp jumps 18102 -> 20159 (+11%) on this alone - the
underfilled l grids hurt far more in-chain than isolated rates suggested.

With that, mode 4 beats mm back-to-back on BOTH serving llama models, so
resolve_coopmat_mode now defaults to cm2 where the hardware has it (the item
11(c) gate Boris set: cm2 must first beat mm). DASLLAMA_COOPMAT still
overrides the whole ladder.

Default-path board (zen2 / 5060 Ti, debug-jit, REBAR=0, vs llama-bench b10659):
3B 7406.8 +/- 310 pp / 105.7 tg = 96.3% / 96.1%; tinyllama 20188.1 +/- 185 pp
/ 294.4 tg = 99.6% pp (inside their row noise), tg AHEAD. Still unported to
mode 4 (headroom): the ar+rq fusion, the kvm merge, the wg_blk0 push-constant
base.

Gates: kernels 54/54, tier 36/36, env registry 14/14 under the new default.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015ZQUVVRi3EUqiZePdhVug7
@borisbat @claude
dasLLAMA: followup_vulkan item 11 - the (b)/(c) closure entry
5210d56 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015ZQUVVRi3EUqiZePdhVug7
@borisbat @claude
dasLLAMA vulkan: mode-4 ar fusion - the fused add+rms twin grows an f…
230f26a 
…16 form

The cm2 chain paid a split pair at every ar site (cls_ar writing f32 xb, then
f16cvt rereading it); the q8 chain has fused those since the ar_rq twin landed.
cls_ar_f16_b is the same fusion for the f16 feed: add+rms, the normed row lands
f16 out of the row stash, bit-identical to the split pair (gated in
test_vkd_ar_f16_fused). The consumer's feed picks the twin per site.

Back-to-back vk_fuse A/B, 5060 Ti: 3B pp 7463->7584 (+1.6%), tg 102.3->104.9;
tinyllama pp 19978->20374 (+2.0%), tg 278.6->289.9.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015ZQUVVRi3EUqiZePdhVug7
@borisbat @claude
dasLLAMA vulkan: the wg_blk0 push-constant lever closes as measured-no
9cc6def 
The cm2x probe grows a push variant (scale base off a push-constant field);
it lands slowest of the three base spellings, and lit no longer beats the
shipped shared-mem form either - the old 9% delta predates the 16-bit decode
respelling. Banked in followup item 11 with the ar-fusion port numbers.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015ZQUVVRi3EUqiZePdhVug7
@borisbat @claude
dasLLAMA vulkan: the merged k|v prefill GEMM serves mode 4 too
47b60f3 
pf_kv_merged only excluded cm2 out of caution - pf_gemm_enc is parametric in
(d, blk), so the merged d = 2*kvd call rides the cm2 tiles unchanged, doubling
the starved k/v grid and deleting the v GEMM + copy. vk_kvm A/B on cm2:
3B pp 7419 -> 7633 (+2.9%), tinyllama 20599 -> 20694 (+0.5%).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015ZQUVVRi3EUqiZePdhVug7
@borisbat @claude
dasLLAMA vulkan: the cm2 fa tile grows an f16-out stamp - mode 4 skip…
b78bdb2 
…s the b+6 convert

FaCm2H64/H128 restructure into templates (OUT16 + typedef OT, the RopeKvStoreT
pattern); the f16 stamp converts the O accumulator in-kernel and lands the wo
feed directly, so the per-layer attn->f16 convert dispatch never encodes. The
gate proves the twin bit-exact against the split pair's own on-device f16cvt
(CPU float16() differs on rounding ties - the device converts agree with each
other). spirv_builtins grows the coopmatWgAcc_f16_64x128 marker.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015ZQUVVRi3EUqiZePdhVug7
@borisbat @claude
dasLLAMA: followup_vulkan item 11 - the mode-4 chain-thinning day banked
5d5c3ca 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015ZQUVVRi3EUqiZePdhVug7
@borisbat @claude
dasSpirv/dasLLAMA: small-int SSBO stores - the requant writers drop t…
362e777 
…heir word packing

The 8/16-bit STORE half gets its coverage: a golden fixture (narrowing
OpSConvert/OpUConvert + stores through 8/16-bit access chains, spirv-val
clean) and a live-device cell (exact bytes for int16/uint16 elements and
int8/f16 struct members). On that foundation every Q8 requant writer stores
quants as bytes: q8_pack4 and the q8k butterfly (2 subgroup shuffles per
element) are deleted, outq members go array<int8>. Same bytes by the
bit-exact gates (kernels 56/56, tier 36/36).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015ZQUVVRi3EUqiZePdhVug7
@borisbat @claude
dasLLAMA: followup item 11 - the small-int store ledger closes
70cf928 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015ZQUVVRi3EUqiZePdhVug7
@borisbat @claude
dasLLAMA vulkan: Q4_K joins the cm2 prefill - decode-in-load over the…
ac89ad9 
… superblock planes

K4Cm2LBatch/K4Cm2MBatch: the Q8 tile geometry with a Q4_K decode callback -
nibble + per-32-group scale/min off the repacked planes (d*sc*q - dmin*mn),
(1, 256) layout blocks. Oracle-gated (0 of 89600 off, l and m). The probe
gains a k4 arg: 35.8-38.2 TF/s vs the serving kq tile at 12.0-12.7 on every
Qwen3-4B role shape.

Wiring: pf_f16_feed admits k4; the feed flags become GROUP-wide ANDs
(pf_qkv6/pf_gu6) and pf_gemm_enc takes the caller's feed flag, so a k6
sibling pins its whole group to the kq-tile route instead of feeding a
quant kernel f16 rows. Qwen3-4B Q4_K_M mode-3/4 pair: pp 1626 -> 2654
(+63%), tg equal; prefill kq parity arm token-exact on the default path.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015ZQUVVRi3EUqiZePdhVug7
@borisbat @claude
dasLLAMA: followup item 11 - the Q4_K cm2 entry
6dad289 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015ZQUVVRi3EUqiZePdhVug7
@borisbat @claude
dasLLAMA vulkan: Q6_K cm2 tiles land oracle-gated but stay PINNED off…
5e3be08 
… the feed

K6Cm2LBatch/K6Cm2MBatch: the 6-bit compose decode (nibble | qh 2 bits - 32,
per-16 signed sub-scale) over the repacked 192-byte planes. Oracle 0-off on
both tiles; the probe gains a k6 arg. But the rate collapsed: 9.3-13.4 TF/s
vs the kq tile at 11.9 (the two-plane ql/qh compose hits a driver cliff the
k4 spelling avoids), and unpinned Q4_K_M e2e regressed 2654 -> 2436. So
pf_f16_feed keeps k6 on the kq route; the classes, gates, wiring arms and
probe stand ready for the k6 decode-spelling chase. Q4_K_M confirmed back
at 2669 pp.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015ZQUVVRi3EUqiZePdhVug7
@borisbat @claude
dasLLAMA: followup item 11 - the Q6_K pinned entry
68161b3 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015ZQUVVRi3EUqiZePdhVug7
@borisbat @claude
dasLLAMA vulkan: the k6 cliff was ONE byte4 dynamic select - Q6_K unp…
4f15f3d 
…ins at 3188 pp

The k6x decode bisect (nil/flat/ql/pair/pair16/sfix probe variants over the
shipped l-tile body) pinned the collapse: the full two-plane 6-bit compose
costs only ~33% (flat 39.7 vs nil 59.6 TF/s) - the killer was the sub-scale
extract unpack8(word)[sidx & 3], a byte4 DYNAMIC select, the same death shape
the Q8 chase found (byte2 [i&1] selects are fine). Respelled as shift +
arithmetic-shift sign extension: 12.8 -> 32.9 TF/s (gate), 8.6 -> 25.0 (down).
k6 unpins from the kq route; Qwen3-4B Q4_K_M pp 2669 -> 3188 (+19%; 1626 at
mode 3 - +96% on the day). Oracle 0-off, tier 36/36, parity token-exact.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015ZQUVVRi3EUqiZePdhVug7
@borisbat @claude
dasLLAMA: followup item 11 - the k6 cliff closure + the decode select…
04779d3 
… rule

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015ZQUVVRi3EUqiZePdhVug7
Copilot AI lite review requested due to automatic review settings August 28, 2026 20:57
Copilot AI reviewed Aug 28, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR advances the dasLLAMA Vulkan backend (cm2 tiles default where supported; expanded prefill optimizations like fused feeds, merged k|v, and device-side embedding gather) and makes [arch(at=...)] citations mandatory/checked everywhere by strengthening LINT026/027 and updating docs/tooling to match.

Changes:

  • Vulkan backend: switch default coopmat path to cm2 where available; add/extend GPU prefill knobs and resident embed-gather plumbing.
  • SPIR-V / Vulkan feature gating: fix coopmatClamp loop control, expand 8/16-bit SSBO storage requirements, and add corresponding SPIR-V + live-device tests.
  • Repo tooling/docs: enforce [arch] reverse pass and rule-doc line caps without opt-in tags; introduce/refresh architecture + review docs and an editor hook to surface cited sections.

Reviewed changes

Copilot reviewed 75 out of 75 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
utils/REVIEW.md Clarifies review routing for shared utils libraries.
utils/mcp/tools/arch_of.das Updates arch citation reporting to include scope failures.
utils/mcp/REVIEW.md Adds install-list duty for top-level utils/mcp .das files.
utils/lint/tests/lint027_rule_doc_lines.das Updates LINT027 tests for “always armed” behavior.
utils/lint/tests/lint026_arch_citations.das Extends LINT026 tests (scope + build-tree skipping + .md positional).
utils/lint/REVIEW.md Minor wording/line-wrap correction in checklist.
utils/lint/README.md Documents new LINT026/027 always-on semantics.
utils/lint/main.das Implements always-on LINT026 reverse + LINT027 folder arming; skips build trees.
utils/common/arch_citations.das Adds anchor indexing + subtree-scope validation for citations.
tests/spirv/test_storage_8_16.das Adds SPIR-V validation tests for 8/16-bit stores + unpack16.
tests/spirv/test_coopmat2.das Asserts coopmatClamp loop uses LoopMerge Unroll.
tests/spirv/test_census.das Includes new fixtures in opcode census input set.
tests/spirv/_spirv_common.das Adds SPIR-V fixtures for q8 store + 16-bit-lane unpack8.
skills/mcp_tools.md Updates [arch] tool semantics and scope rule documentation.
skills/internal/make_pr.md Strengthens harvest-row duty: FACT requires anchor + citer + citation.
REVIEW_COMMON.md Adds same-diff “FACT => section+anchor+citation” duty.
modules/dasVulkan/utils/vulkan2rst.das Adds new helpers to documentation grouping regexes.
modules/dasVulkan/tutorials/REVIEW.md Points tutorials checklist to ARCHITECTURE.md (not CLAUDE.md).
modules/dasVulkan/tests/integration/test_compute_features.das Adds GPU store-path test kernel + assertions.
modules/dasVulkan/REVIEW.md Adds new Vulkan-specific review duties (features, docs grouping, ownership, portability).
modules/dasVulkan/generator/vk_emit_boost.das Replaces inline comment with [arch] citation for handle ABI rationale.
modules/dasVulkan/generator/REVIEW.md Updates generator checklist to point at ARCHITECTURE.md.
modules/dasVulkan/daslib/vulkan_window.das Adds ownership [arch] citations to finalizers.
modules/dasVulkan/daslib/vulkan_runtime.das Moves weak_copy rationale into ARCHITECTURE.md via citation.
modules/dasVulkan/daslib/vulkan_boost.das Expands storage gate + adds full-subgroups probe and pipeline creator; adds [arch] citations.
modules/dasVulkan/CLAUDE.md Splits present-tense module truths into ARCHITECTURE.md; keeps agent-facing instructions.
modules/dasVulkan/ARCHITECTURE.md New: records module architecture (layers, generator, ownership, portability, docs pipeline).
modules/dasSpirv/spirv/spirv_emit.das Fixes coopmatClamp LoopMerge; adds [arch] citations for emission mechanisms.
modules/dasSpirv/spirv/spirv_builtins.das Adds cm2 marker struct + [arch] citation for marker parsing mechanism.
modules/dasSpirv/REVIEW.md Tightens emitter-fixture duties; strengthens census expectations.
modules/dasSpirv/ARCHITECTURE.md Adds anchors + documents small-int surface + cm2 marker mechanism + loop unroll rule.
modules/dasLLAMA/tests/test_vulkan_tier.das Updates default-mode comment; adds extra forensics logging on mismatch.
modules/dasLLAMA/tests/test_gpu_tier.das Adds seam tests for ids-form resident prefill installation/forwarding.
modules/dasLLAMA/tests/_vkd_oracles.das Adds CPU oracles for K4/K6 f16 GEMMs.
modules/dasLLAMA/REVIEW.md Tightens wording and adds override announce semantics.
modules/dasLLAMA/REVIEW_GPU.md Updates GPU checklist to include Vulkan doc + new rules for embed-gate and decode spelling.
modules/dasLLAMA/README.md Documents new ARCHITECTURE_GPU_VULKAN.md companion.
modules/dasLLAMA/ENVIRONMENT.md Adds/updates Vulkan knobs docs (KV merge, overlap, GPU embed, SPV dump).
modules/dasLLAMA/dasllama/dasllama_vulkan_common.das Implements cm2 default resolution + SPV dump + full-subgroup pipelines + submission helpers.
modules/dasLLAMA/dasllama/dasllama_version.das Bumps DASLLAMA_VERSION to 12 with updated rationale.
modules/dasLLAMA/dasllama/dasllama_math_vulkan.das Installs ids-form resident prefill seat for Vulkan backend registration.
modules/dasLLAMA/dasllama/dasllama_gpu_tier.das Adds ids-form resident prefill seat + embed-f32 cap helpers and install API.
modules/dasLLAMA/dasllama/dasllama_gpu_resident.das Adds embed-gather accounting + gate registration + marks swap integration.
modules/dasLLAMA/dasllama/dasllama_env.das Updates VulkanEnv docs and adds new env fields (KV merge, overlap, GPU embed, SPV dump, fullsg).
modules/dasLLAMA/dasllama/dasllama_common.das Adds embed-gate plumbing to skip CPU embed when Vulkan gather is available; backfills on decline.
modules/dasLLAMA/dasllama/dasllama_blocks.das Ensures embd-entry clears embed-gpu pending state before body.
modules/dasLLAMA/benchmarks/REVIEW.md Routes Vulkan backend followups to followup_vulkan.md.
modules/dasLLAMA/ARCHITECTURE.md Notes new Vulkan resident-driver companion document.
modules/dasLLAMA/ARCHITECTURE_RUNTIME.md Adds anchors + documents f16 feed group constraint; anchors env-knobs section.
modules/dasLLAMA/ARCHITECTURE_MEASUREMENT.md Adds anchors + documents Vulkan GEMM probe’s attribution axes.
modules/dasLLAMA/ARCHITECTURE_GPU.md Adds anchors + documents embed gather design and Vulkan bake semantics.
modules/dasLLAMA/ARCHITECTURE_GPU_VULKAN.md New: Vulkan resident-driver mechanisms (window chain, cm2 decode, residency plan, etc.).
modules/dasLLAMA/.lint_config Removes enforce_arch opt-in (no longer needed).
LAWS.md Records new repo-wide rulings for mandatory [arch] enforcement and harvest duties.
history/README.md Updates archive notes to reference new dasVulkan ARCHITECTURE.md.
doc/source/reference/language/lint.rst Updates LINT026/027 documentation for always-on reverse + subtree constraint.
doc/reflections/das2rst.das Removes enforce_arch_at from lint_config docs grouping.
daslib/shader_lingua_franca.das Adds unpack8 overloads for int16/uint16 -> byte2/ubyte2 with [arch] citations.
daslib/REVIEW.md Minor wording adjustment for architecture-doc additions rule.
daslib/lint_config.das Removes enforce_arch_at; documents mandatory LINT026/027 behavior.
daslib/ARCHITECTURE_EMIT.md Adds anchor and documents width-variant “lowered-by-name” policy.
CLAUDE.md Updates document-system summary for mandatory [arch] citations and LINT027 behavior.
ARCHITECTURE_COMMON.md Makes “anchor+citation within subtree” rule explicit.
.github/workflows/REVIEW.md Clarifies that diff-derived inputs can widen folder-rule blast radius.
.github/workflows/extended_checks.yml Runs lint on changed .das and .md files to arm doc rules.
.claude/settings.json Adds PostToolUse hook for arch section injection + reset hooks.
.claude/hooks/README.md Documents arch_inject hook behavior and updated IO rules.
.claude/hooks/LAWS.md New: rulings/provenance for hook behavior.
.claude/hooks/arch_inject.py New: inject cited arch section context after edits to cited functions.
.claude/agents/harvester.md Requires FACT filings to include anchor + citer + [arch] citation.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread modules/dasLLAMA/dasllama/dasllama_gpu_resident.das
Comment thread utils/lint/main.das
Copilot AI review requested due to automatic review settings August 28, 2026 21:07
Copilot AI reviewed Aug 28, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 75 out of 75 changed files in this pull request and generated 1 comment.

Comment thread .claude/hooks/arch_inject.py
Copilot AI review requested due to automatic review settings August 28, 2026 21:13
Copilot AI reviewed Aug 28, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 75 out of 75 changed files in this pull request and generated 1 comment.

Comment thread modules/dasLLAMA/dasllama/dasllama_vulkan_common.das
Copilot AI review requested due to automatic review settings August 28, 2026 21:56
Copilot AI reviewed Aug 28, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 76 out of 76 changed files in this pull request and generated 1 comment.

Comment thread .claude/hooks/README.md Outdated
@borisbat @claude
dasLLAMA vulkan: the PR round - harvest landings, review fixes, the m…
86caaf6 
…issing tests

Harvest: the branch's comments file into the document system - two REVIEW_GPU.md
rules (the 16-bit decode spelling, the ids-form accept/probe parity), the sec.1.5
embed-gather asymmetry and a new sec.2.2j (merged k|v, the overlap ring, the K/V
host-DMA placement) in ARCHITECTURE_GPU.md, the group-feed statement in
ARCHITECTURE_RUNTIME.md, the coopmatClamp Unroll mechanism in dasSpirv's
ARCHITECTURE.md, the shaderInt16 / subgroup-pair truths in dasVulkan's CLAUDE.md;
the renames the comments were standing in for (pf_cmd_ring, g_pf_ids_stage,
EmbArgs.embed_scale, submit_nofence, full_sg_on, q8_quant1, the fused-feed helper
pf_qkv_feed_fused, PF_CHUNK_MAX); EmbArgs.nbb derived in-kernel.

Fixes the review round found: vk_rdec_set_emb / vk_rdec_upload_emb_f32 take the
dry-bake guard every sibling seam has (a .dlim bake on a tied q8 model dereferenced
a null g_rd); the untied "own q8 plane" arm was dead (t.cls_q8 implies tied) - the
gather now mirrors embed_row's ladder exactly, and the rail pre-flights its class
before arming; the raw f32 table joins the residency plan, is capped by the device
storage range, and declines with a warning; the embed gate is scoped to the active
prefill override and embed_gpu_pending self-heals on every CPU path; the Vulkan 1.3
subgroup pair chains into every int_dot creator (DASLLAMA_VK_FULLSG was invalid on
the now-default cm2 device); cm2_tile_cols' short-window rule sits ahead of the
unknown-SM return; cooperative_matrix2_supported probes shaderInt16; announce lines
name their env spelling; the profiled run pins the single submit; the SPV dump
captures the emitted words and warns on a failed write; DASLLAMA_VERSION 12;
DASLLAMA_VK_KVM renamed DASLLAMA_VK_KV_MERGE (the | broke the generated table).

Tests: the new kernel cells skip instead of feint, sentinel-prefill their
GPU-vs-GPU planes, assert residual liveness, and carry poison legs; the rope gates
grow the merged-k|v layout arm; the embed gather runs at a nonzero slab base with
repeated ids; the ids-form seat gets its contract rows; the resolved coopmat
default and the tile pick get a table cell; an unpack8 16-bit golden fixture
(VectorExtractDynamic joins the census); int8 element stores from adjacent
invocations on a live device.

Probe: every rate row marked timing-only, the header names its attribution axes,
the ported sfix arm is deleted, the reference arm loses its attribution name.

The relink round: every mechanism the harvest filed is cited from the code that
implements it - ~130 [arch(at=...)] citations across dasLLAMA, dasSpirv, dasVulkan
and daslib, the comments those sections replaced deleted. ARCHITECTURE_GPU.md splits
its Vulkan sections into ARCHITECTURE_GPU_VULKAN.md (2.2j-2.2p: the window chain,
the cm2 decode lanes and tile pick, the class-pipeline seat, the residency plan,
the GPU-slot marks, the Q8 byte store); modules/dasVulkan/CLAUDE.md splits into
CLAUDE.md / ARCHITECTURE.md / REVIEW.md. The lint side goes mandatory: the
[docs] enforce_arch key is gone, LINT026's reverse pass runs over every root as one
cited-set from one read per .das, a citation must name a document in the citer's own
folder tree, LINT027 arms on any folder holding a rule document, the walkers skip
CMake build trees, and the changed-set CI lane lints changed .md too. REVIEW_COMMON.md
carries the harvest duty (statement + anchor + citation, same diff), ARCHITECTURE_COMMON.md
the anchor form, the harvester's FACT verdict names its citer. A checklist audit round
over the arc then landed: a section false for one citer, three dropped facts, three
missing override announces, the attribution slips, twenty-five checklist-text defects,
and dasllama_vulkan_classes.das renormalized from the CRLF it was committed with. The
[arch] section-injection Edit hook (.claude/hooks/arch_inject.py) lands for its trial.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015ZQUVVRi3EUqiZePdhVug7
Copilot AI review requested due to automatic review settings August 28, 2026 22:03
Copilot AI reviewed Aug 28, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 76 out of 76 changed files in this pull request and generated 1 comment.

Suppressed comments (2)

utils/lint/main.das:210

  • scan_docs_folders skips build-tree children (!is_build_tree(full)), but still treats the root itself as source even if it contains CMakeCache.txt. If lint is invoked on a build directory directly, LINT025/027 folder discovery can traverse FetchContent trees under it and report unrelated document findings.

Include is_build_tree(path) in the early-return condition so a build-tree root is skipped consistently.
utils/lint/main.das:326

  • scan_arch_folder skips build-tree children (is_build_tree(full)), but if a build directory is passed as a root it is still walked, and its .md anchors / .das citations can incorrectly influence the repo’s LINT026 results.

Skip the root itself when it is a build tree.

Comment thread utils/lint/main.das
@borisbat
Merge remote-tracking branch 'origin/master' into bbatkin/vkclass
bb78a03 
# Conflicts:
#	.github/workflows/REVIEW.md
#	modules/dasLLAMA/REVIEW.md
#	utils/REVIEW.md
Copilot AI review requested due to automatic review settings August 28, 2026 22:45
Copilot AI reviewed Aug 28, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 75 out of 75 changed files in this pull request and generated 1 comment.

Comment thread utils/common/arch_citations.das
@borisbat
borisbat merged commit d544dc2 into master Aug 28, 2026
40 checks passed
@borisbat
borisbat deleted the bbatkin/vkclass branch August 28, 2026 23:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@borisbat