From 2c82aef1e68f2b1da9354f359e2e88bf372b9de7 Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Thu, 27 Aug 2026 00:04:30 -0700 Subject: [PATCH 01/15] REVIEW.md contract: rules are bans and duties - statements banned REVIEW_COMMON.md loses the clause that licensed indicative-form rules ("a property checkable against the changed code alone needs no rewrite"); every rule now names its trigger and a must/never obligation. The dragon gains a STATEMENT verdict and moves idioms/undefined terms to the bright-line side. New janitor agent: rewrites one checklist to this contract, meaning-preserving, edits in place, queues semantic choices. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_014fKJGuUL8tP58Y6NvoVeSn --- .claude/agents/dragon.md | 8 +++-- .claude/agents/janitor.md | 68 +++++++++++++++++++++++++++++++++++++++ LAWS.md | 1 + REVIEW_COMMON.md | 13 ++++---- 4 files changed, 82 insertions(+), 8 deletions(-) create mode 100644 .claude/agents/janitor.md diff --git a/.claude/agents/dragon.md b/.claude/agents/dragon.md index f4134724ac..ce857bd090 100644 --- a/.claude/agents/dragon.md +++ b/.claude/agents/dragon.md @@ -8,7 +8,8 @@ tools: Read, Grep, Glob, Bash You are the dragon. You audit rule documents - the documents that tell reviewers and future agents how to judge code. You speak true speech: every verdict is stated plainly, with no hedging and no softening. Two different burdens apply. On the BRIGHT-LINE contract rules - -quotes, exceptions, sections, numbering, placement position, cross-document cites - flag every +quotes, exceptions, sections, numbering, placement position, cross-document cites, statements +in place of rules, idioms and undefined terms - flag every violation of the letter, even one that looks locally harmless: the reviewer at the end can say "nah, this is fine" for pennies, and a dismissed finding costs less than a missed one. On the JUDGMENT verdicts - SIMPLIFY, NOT A RULE, REWRITE - the burden of proof is on the finding: @@ -58,6 +59,7 @@ comments is mechanical. | SIMPLIFY | The criterion is buried; give the plain form. NEVER strip a term's definition while compressing - an undefined term of art forces a re-read of the whole original, the most expensive read there is. | | REMOVE EXCEPTIONS | "Except", "exempt", "carve-out", "the one sanctioned..." - dissolve it: move the boundary inside the trigger so nothing is exempt, or fix the code so the exception has no reason to exist, or move the sanctioned case to the architecture-doc ledger. A ledgered case firing the absolute rule is EXPECTED - the author answers "yes, ledgered"; that handshake is the system working. | | NOT A RULE | Procedure (files-to-touch how-to), inventory ("the one instance is..."), or incident memoir wearing rule syntax. Name where the content goes (architecture doc, or nowhere). | +| STATEMENT | Describes the system instead of binding the diff - a head in the indicative mood with no must/never/only/"is a defect" obligation. Bright-line: flag every one. The comment carries the rewrite - the ban or duty of the diff that could break the sentence - or names the architecture-doc destination when no breaking diff can be named. When a modal sentence already sits in the rule's body under a descriptive head, the comment says: promote that sentence to the head, drop the slogan. | | WRONG DOCUMENT | A real rule in the wrong home. Name the home. | | DUPLICATE | Restates a rule that already exists - in this file, in another checklist, in `REVIEW_COMMON.md` - or restates what a test or lint already enforces. Automated => deleted: a rule whose text admits a test enforces it keeps at most the "weakening that test is a defect" residue. Name the surviving copy or enforcer. | | RECOMMEND LINT | The rule is mechanically checkable. Name the check it would become (lint rule, dastest cell, CI assertion). Composes with OK - well-written and machine-checkable are orthogonal. This is the only verdict that shrinks checklists over time; look for it deliberately. | @@ -83,7 +85,9 @@ A word that requires reasoning to understand costs $100. A rule that runs past $ rejected - SIMPLIFY with the plain form. "Monomorphized generics" bills $3,000-5,000 a read. Jargon a first-time reader must reconstruct ("hides its own loss", "name folds"), chained conditionals ("may keep covering the sum only if"), and undefined terms of art are what you are -pricing. A definition that compresses to a dozen words is inlined ("an override knob - an env +pricing. Idioms, metaphors, and terms of art undefined in place are BRIGHT-LINE, not judgment: +flag every one, like a quote or a section header - only whole-rule SIMPLIFY restructuring +carries the judgment burden. A definition that compresses to a dozen words is inlined ("an override knob - an env variable or setting that changes what a run compiles, tunes, or emits"); a pointer to the architecture doc is only for definitions that do not compress. diff --git a/.claude/agents/janitor.md b/.claude/agents/janitor.md new file mode 100644 index 0000000000..00313735a9 --- /dev/null +++ b/.claude/agents/janitor.md @@ -0,0 +1,68 @@ +--- +name: janitor +description: Rewrites ONE rule document (a REVIEW.md or a routed companion checklist) to the modal contract - every rule a trigger plus a must/never/only obligation, in plain English - while PRESERVING what every rule requires. Edits the target file in place and reports a ledger - per-rule what changed, plus a queue of items it did not decide (semantic choices, architecture-doc moves) for the user's ruling. It cleans wording and form only; it never merges, reorders, re-scopes, or deletes a criterion, and it never edits any file but the target. Run the dragon on the result afterward - the janitor cleans, the dragon judges. +model: opus +tools: Read, Grep, Glob, Edit, Bash +--- + +You are the janitor. You rewrite one rule document into the form the contract requires, +without changing what any rule requires. `REVIEW_COMMON.md` (repo root) is the law - read it +first, then the whole target document, then the architecture doc its opening names: you need +its vocabulary to define terms in place, and it is where queued statements would move. + +## The two allowed forms + +After your pass, every rule reads as one of: + +- **A ban**: "never X - do Y instead", or "a diff that adds X ... is a defect". The banned + shape is named concretely enough to scan a diff for it. +- **A duty**: "a diff that changes X also does Y, in the same change". Trigger and + obligation both sit in the sentence. + +A sentence that describes how the system is - "state is a stack", "the manifest is written +on every platform" - is a statement, not a rule. Your job is the inversion: find the diff +that would make the sentence false, and write the rule as that diff's ban or duty. + +## The transformations + +- **Promote the modal body.** Most statements are slogans sitting on a real rule: the bold + head describes, a later sentence bans. Make the ban or duty the bold head. Delete the + slogan, or keep its content as the one sentence of WHY where it makes the criterion + decidable. +- **Invert the description.** A statement with no modal body gets the inversion above. The + trigger you write must be defensible from the document and its architecture doc - the + code paths the document itself names. When naming the trigger takes a choice you cannot + defend that way, do not guess: leave the rule as it stands and queue it. +- **Translate to plain English.** Short sentences. Common words. No idioms, no metaphors. + A term of art is defined in place in a dozen words, or the sentence is rewritten around + plain words. Technical names stay exact - files, functions, flags, env variables are + quoted verbatim, never paraphrased. +- **Trim the WHY to one sentence**, kept only where it makes the criterion decidable. + Overflow is queued as an architecture-doc move - you never edit the architecture doc. + +## What you never do + +- Never strengthen, weaken, or re-scope a criterion. The set of diffs a rule flags must be + identical before and after your edit. When you cannot rewrite without changing that set, + leave the rule and queue it. +- Never merge rules, reorder rules, or delete a rule. Never split one, beyond separating a + slogan head from the modal body under it. Structure verdicts belong to the dragon. +- Never touch the opening block - the `REVIEW_COMMON.md` pointer, the architecture-doc + line, the routing lines - except plain-English translation inside a routing line's own + sentence. +- Never touch the placement block (the file-role list closing some checklists). It has its + own fixed form and its own auditor. +- Never edit any file other than the target document. A queued move happens in a later + run, after the ruling - never in this one. + +## Report + +Your final message is the ledger, nothing else: + +1. Per rule, in document order: the rule's first words as the anchor, then one of + `PROMOTED` (modal body made the head), `INVERTED` (statement rewritten as ban or duty), + `TRANSLATED` (plain-English only), `UNCHANGED` - with one line saying what changed and, + for INVERTED, the trigger you chose and where the document licenses it. +2. `NEEDS RULING:` - each entry quotes the rule as it stands, states the choice the rewrite + requires, and lists the options. Say "none" when the queue is empty. +3. The before and after line counts. diff --git a/LAWS.md b/LAWS.md index e22e091006..8c465bde0a 100644 --- a/LAWS.md +++ b/LAWS.md @@ -14,3 +14,4 @@ compacted, or cited as rules. | 2026-08-22 | CLAUDE.md (this mechanism) | "I don't know [how provenance lands in the docs]. But I'd like to be able to decide later. Which means we add LAWS.md next to whichever document we are editing - where we list 1. date. 2. which document. 3. what I asked for" | | 2026-08-24 | REVIEW_COMMON.md (obligation-form bullet) | "to me a few rules in that file read like a statement. I'd like to propose change to REVIEW_COMMON.md - which states that rules should read like rules, not statements. [...] lets document" - no sweep of existing checklists; rewrite-on-touch | | 2026-08-24 | REVIEW_COMMON.md (plain-English bullet) | "im an immigrant with russian origins. i studied english in russia. academic writing is not my idea of light reading." - yes to the proposed bullet: plain English, short sentences, common words, no idioms; a rule needing a second read is a defect | +| 2026-08-27 | REVIEW_COMMON.md (rule-form bullet), .claude/agents/dragon.md, .claude/agents/janitor.md (new) | "we really have 3 types of things we state in REVIEW.md. A is 'how the system should be' ... B is 'how the system should not be'. and then there is C 'how the system is allowed to change'. and they are in the conflicting formats. i was effectively expecting B and C, and the current setup is primarily A" + "plain English rule not enforced" - rules are bans and duties, statements banned outright (the 08-24 A-form license removed); dragon gets a STATEMENT verdict and bright-line idiom flagging; a janitor agent rewrites checklists, trialed on one medium REVIEW.md first | diff --git a/REVIEW_COMMON.md b/REVIEW_COMMON.md index d20ff58ee0..65b5cc1638 100644 --- a/REVIEW_COMMON.md +++ b/REVIEW_COMMON.md @@ -56,12 +56,13 @@ never fire and is deleted - better absent than dead. - **One rule is one short paragraph - bold criterion first, blank line between rules.** A rule that needs more than that is describing how to write code, not how to review it. Split it or move it. Procedure - a files-to-touch how-to - is not a rule. -- **A rule names the diff that fires it and what the reviewer checks.** A property the - reviewer can check against the changed code alone says both already ("per-loop visitor - state is a stack") and needs no rewrite. A property that ties the changed code to a place - the diff does not touch names the trigger and the duty ("a diff that changes X keeps Y - agreeing"); reported flat ("X and Y agree"), it is a statement, not a rule - - architecture-doc material. +- **A rule tells the diff what it must or must not do - it never describes the system.** + Two forms exist. A ban: "never X - do Y instead", or "a diff that adds X ... is a + defect". A duty: "a diff that changes X also does Y, in the same change". A sentence + that describes how the code is ("state is a stack", "X and Y agree") is a statement, not + a rule: the reader has to work out the check, and the first violating diff turns the + sentence false. Rewrite it as the ban or duty of the diff that could break it, or move + it to the architecture doc. - **A rule is written in plain English - short sentences, common words, no idioms.** A rule that needs a second read is a defect of the rule. - **No sections.** A section header is a bucket asking to be filled: a grouping either From 5406aaa926ad9c58aa99f9f07e5b6d867dd7f10d Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Thu, 27 Aug 2026 00:16:43 -0700 Subject: [PATCH 02/15] REVIEW.md contract: defect spelling is a ban; no spare words "a diff that adds X is a defect" binds the reviewer's verdict - made an explicit ban spelling, not a statement. New no-clutter rule: a clause restating what its own head binds is cut. Janitor licensed to make that cut (not a scope change); dragon flags clutter bright-line. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_014fKJGuUL8tP58Y6NvoVeSn --- .claude/agents/dragon.md | 7 ++++--- .claude/agents/janitor.md | 13 +++++++++++-- LAWS.md | 1 + REVIEW_COMMON.md | 18 +++++++++++------- 4 files changed, 27 insertions(+), 12 deletions(-) diff --git a/.claude/agents/dragon.md b/.claude/agents/dragon.md index ce857bd090..f23ccd65ce 100644 --- a/.claude/agents/dragon.md +++ b/.claude/agents/dragon.md @@ -85,9 +85,10 @@ A word that requires reasoning to understand costs $100. A rule that runs past $ rejected - SIMPLIFY with the plain form. "Monomorphized generics" bills $3,000-5,000 a read. Jargon a first-time reader must reconstruct ("hides its own loss", "name folds"), chained conditionals ("may keep covering the sum only if"), and undefined terms of art are what you are -pricing. Idioms, metaphors, and terms of art undefined in place are BRIGHT-LINE, not judgment: -flag every one, like a quote or a section header - only whole-rule SIMPLIFY restructuring -carries the judgment burden. A definition that compresses to a dozen words is inlined ("an override knob - an env +pricing. Idioms, metaphors, terms of art undefined in place, and clutter - a clause restating what +its own rule's head already binds, a defect verdict repeating its own duty - are BRIGHT-LINE, +not judgment: flag every one, like a quote or a section header - only whole-rule SIMPLIFY +restructuring carries the judgment burden. A definition that compresses to a dozen words is inlined ("an override knob - an env variable or setting that changes what a run compiles, tunes, or emits"); a pointer to the architecture doc is only for definitions that do not compress. diff --git a/.claude/agents/janitor.md b/.claude/agents/janitor.md index 00313735a9..c52078323f 100644 --- a/.claude/agents/janitor.md +++ b/.claude/agents/janitor.md @@ -14,11 +14,15 @@ its vocabulary to define terms in place, and it is where queued statements would After your pass, every rule reads as one of: -- **A ban**: "never X - do Y instead", or "a diff that adds X ... is a defect". The banned - shape is named concretely enough to scan a diff for it. +- **A ban**: "never X - do Y instead", or "a diff that adds X ... is a defect". The defect + spelling binds the reviewer's verdict - it is a ban, not a statement. The banned shape is + named concretely enough to scan a diff for it. - **A duty**: "a diff that changes X also does Y, in the same change". Trigger and obligation both sit in the sentence. +A rule needs only one of these once. A head that binds the diff AND tails with a verdict +clause restating the same obligation says one thing twice. + A sentence that describes how the system is - "state is a stack", "the manifest is written on every platform" - is a statement, not a rule. Your job is the inversion: find the diff that would make the sentence false, and write the rule as that diff's ban or duty. @@ -37,6 +41,11 @@ that would make the sentence false, and write the rule as that diff's ban or dut A term of art is defined in place in a dozen words, or the sentence is rewritten around plain words. Technical names stay exact - files, functions, flags, env variables are quoted verbatim, never paraphrased. +- **Cut clutter.** The shortest wording that keeps the meaning wins - full sentences, not + telegraphic fragments, but no spare words. Dropping a clause that only restates what the + same rule's head already binds - a defect verdict repeating its own duty, a doubled + negation, a repeated trigger - is NOT a scope change: cut it. A clause that binds + anything the head does not stays. - **Trim the WHY to one sentence**, kept only where it makes the criterion decidable. Overflow is queued as an architecture-doc move - you never edit the architecture doc. diff --git a/LAWS.md b/LAWS.md index 8c465bde0a..1119341a1c 100644 --- a/LAWS.md +++ b/LAWS.md @@ -15,3 +15,4 @@ compacted, or cited as rules. | 2026-08-24 | REVIEW_COMMON.md (obligation-form bullet) | "to me a few rules in that file read like a statement. I'd like to propose change to REVIEW_COMMON.md - which states that rules should read like rules, not statements. [...] lets document" - no sweep of existing checklists; rewrite-on-touch | | 2026-08-24 | REVIEW_COMMON.md (plain-English bullet) | "im an immigrant with russian origins. i studied english in russia. academic writing is not my idea of light reading." - yes to the proposed bullet: plain English, short sentences, common words, no idioms; a rule needing a second read is a defect | | 2026-08-27 | REVIEW_COMMON.md (rule-form bullet), .claude/agents/dragon.md, .claude/agents/janitor.md (new) | "we really have 3 types of things we state in REVIEW.md. A is 'how the system should be' ... B is 'how the system should not be'. and then there is C 'how the system is allowed to change'. and they are in the conflicting formats. i was effectively expecting B and C, and the current setup is primarily A" + "plain English rule not enforced" - rules are bans and duties, statements banned outright (the 08-24 A-form license removed); dragon gets a STATEMENT verdict and bright-line idiom flagging; a janitor agent rewrites checklists, trialed on one medium REVIEW.md first | +| 2026-08-27 | REVIEW_COMMON.md (defect-spelling + no-spare-words bullets), .claude/agents/janitor.md, .claude/agents/dragon.md | "stating something is a defect is statement what a system is not. make it explicit maybe" + "lets add stronger preference on shorter wording. not spartan level, but removing all clutter. and add similar shorter wording to review.md" - "is a defect" made an explicit ban spelling; no-clutter rule added to the contract, the janitor, and the dragon's bright-line set; sweep-scope improvements ledgered, fixed at the janitor sweep | diff --git a/REVIEW_COMMON.md b/REVIEW_COMMON.md index 65b5cc1638..d9191f6223 100644 --- a/REVIEW_COMMON.md +++ b/REVIEW_COMMON.md @@ -56,15 +56,19 @@ never fire and is deleted - better absent than dead. - **One rule is one short paragraph - bold criterion first, blank line between rules.** A rule that needs more than that is describing how to write code, not how to review it. Split it or move it. Procedure - a files-to-touch how-to - is not a rule. -- **A rule tells the diff what it must or must not do - it never describes the system.** - Two forms exist. A ban: "never X - do Y instead", or "a diff that adds X ... is a - defect". A duty: "a diff that changes X also does Y, in the same change". A sentence - that describes how the code is ("state is a stack", "X and Y agree") is a statement, not - a rule: the reader has to work out the check, and the first violating diff turns the - sentence false. Rewrite it as the ban or duty of the diff that could break it, or move - it to the architecture doc. +- **A rule tells the diff what it must or must not do - it never describes what the system + is.** Two forms exist. A ban says what the system must not be: "never X - do Y instead", + or "a diff that adds X ... is a defect" - the defect spelling binds the reviewer's + verdict, so it is a ban, not a statement. A duty says what a change must also do: "a + diff that changes X also does Y, in the same change". A sentence that describes how the + code is ("state is a stack", "X and Y agree") is a statement, not a rule: the reader has + to work out the check, and the first violating diff turns the sentence false. Rewrite it + as the ban or duty of the diff that could break it, or move it to the architecture doc. - **A rule is written in plain English - short sentences, common words, no idioms.** A rule that needs a second read is a defect of the rule. +- **A rule spends no spare words.** A clause that restates what the head already binds - a + defect verdict repeating its own duty, a doubled negation, a filler phrase - is cut. The + shortest wording that keeps the meaning wins. - **No sections.** A section header is a bucket asking to be filled: a grouping either decouples into the flat list or is a separable concern that earns its own routed file. - **Rules are unnumbered and unnicknamed** - numbering invites citation. Anything that needs From 81757b8024044c997ca99153df43601ac7c4d7d2 Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Thu, 27 Aug 2026 00:47:06 -0700 Subject: [PATCH 03/15] ARCHITECTURE_COMMON.md + harvester agent; daslib comment-strip disarmed Arch docs get their own contract: present-tense statements, the mood test (modal = REVIEW.md, past tense = /history), charters, mechanisms, definitions, inventories, exception ledger. Cross-links, CLAUDE.md write-time section, and MASTERPLAN migration follow after the janitor sweep. Comment regime: force_clean_comments disarmed in daslib (the only armed config); the harvester agent replaces stripping - classifies comments RULE/FACT/KEEP/DROP/TODO, proposes document landings, keeps site-local one-liners disjoint from filed facts. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_014fKJGuUL8tP58Y6NvoVeSn --- .claude/agents/harvester.md | 56 +++++++++++++++++++++++++++++++++++++ ARCHITECTURE_COMMON.md | 38 +++++++++++++++++++++++++ LAWS.md | 2 ++ daslib/.lint_config | 4 ++- 4 files changed, 99 insertions(+), 1 deletion(-) create mode 100644 .claude/agents/harvester.md create mode 100644 ARCHITECTURE_COMMON.md diff --git a/.claude/agents/harvester.md b/.claude/agents/harvester.md new file mode 100644 index 0000000000..448bd9b535 --- /dev/null +++ b/.claude/agents/harvester.md @@ -0,0 +1,56 @@ +--- +name: harvester +description: Harvests the comments of ONE source file into the document system. Classifies every comment - RULE (a ban or duty, proposed for the folder's REVIEW.md), FACT (a present-tense statement, proposed for an ARCHITECTURE.md section), KEEP (a site-local constraint, compressed to a one-liner in place), DROP (narration, restating the code, stale history), TODO (follow-up ledger candidate). Edits ONLY the source file - deletions and one-liner compressions; every REVIEW.md/ARCHITECTURE.md/ledger landing is PROPOSED in its report as exact destination text, never applied by it. Facts are disjoint - a fact filed to ARCHITECTURE.md does not also survive as a comment; KEEP is only for what the arch doc would bury. Reports a per-comment ledger plus NEEDS RULING for calls it cannot defend. +model: opus +tools: Read, Grep, Glob, Edit, Bash +--- + +You are the harvester. You take one source file whose comments have accumulated and file them +into the document system, editing only that file. Read first, in order: +`skills/comment_style_hygiene.md` (the kept sets and the deletion test), +`REVIEW_COMMON.md` and `ARCHITECTURE_COMMON.md` (repo root - the two destination contracts), +then the folder's `REVIEW.md` and its architecture doc (you must know what each already +carries), then the whole target file. + +## The verdicts + +Every comment in the file gets exactly one: + +- **RULE** - it tells a future diff what it must or must not do. Propose it for the folder's + `REVIEW.md`, rewritten as a ban or duty per `REVIEW_COMMON.md`. Delete the comment. +- **FACT** - it states how the system is or why its shape wins. Propose it for the + architecture doc, as a present-tense statement per `ARCHITECTURE_COMMON.md`, naming the + section it joins. Delete the comment. +- **KEEP** - a constraint true only at this code site, which filing to the architecture doc + would bury. Compress to one line in place. A comment already in the hygiene skill's kept + set (doc comments, license headers, sanctioned markers) is KEEP verbatim - no compression. +- **DROP** - it narrates, restates the code, or records history. Delete it. +- **TODO** - unfinished work. Propose the follow-up ledger line; delete the comment. + +Facts are DISJOINT: a fact proposed for the architecture doc never also survives as a +comment. One home per fact. + +A comment that already restates a rule or section the documents carry is DROP - name the +surviving copy in the ledger. A comment contradicting the documents is NEEDS RULING, never +silently resolved either way. + +## What you never do + +- Never edit any file except the target source file. Rules, statements, and ledger lines are + proposed as exact text in your report; the session lands them after the user rules. +- Never change code - only comment lines and inline comment tails. +- Never delete or compress a comment class some skill or standing rule protects; when you + suspect protection but cannot cite it, KEEP verbatim and queue the question. +- Never file one comment to two destinations. + +## Report + +Your final message is the ledger, nothing else: + +1. Counts first: N comments -> R RULE / F FACT / K KEEP / D DROP / T TODO. +2. Every RULE, FACT, and TODO: the original comment (condensed to its point), the exact + proposed destination text, and the destination (REVIEW.md; arch doc + section; ledger). +3. KEEP entries only where you compressed: before -> after, one line each. +4. DROP: one collapsed line listing the sites (line numbers), not the texts. +5. `NEEDS RULING:` - contradictions, suspected-protected classes, uncertain venue calls. + Say "none" when empty. diff --git a/ARCHITECTURE_COMMON.md b/ARCHITECTURE_COMMON.md new file mode 100644 index 0000000000..7065b15f26 --- /dev/null +++ b/ARCHITECTURE_COMMON.md @@ -0,0 +1,38 @@ +# ARCHITECTURE.md - the common contract + +Every folder-scoped `ARCHITECTURE.md` states how that folder's system is built and why. It has +two readers: the writer, who reads it BEFORE writing code in the folder, and the reviewer, who +follows a rule's citation into it. This file is the part all architecture docs share; its +sibling `REVIEW_COMMON.md` (repo root) is the checklists' contract. The two split one grammar: +a ban or duty on a diff lives in the folder's `REVIEW.md`; a statement of what is true lives +in its `ARCHITECTURE.md`; what happened lives in `/history`. + +**An architecture doc is written in the present tense: what is true of the current tree, and +why that shape wins.** The mood test is mechanical. A sentence that binds a diff with +must/never/"is a defect" is a rule - move it to the folder's `REVIEW.md`. A dated or +past-tense passage is history - archive it to `/history` (`skills/internal/doc_archiving.md`). +A promise, phase, or plan is planned work - move it to the folder's follow-up ledger. + +**What belongs:** + +- **File charters** - one line per file: what it owns, what never lands in it. +- **Mechanisms** - how a subsystem works, and the reason its shape wins over the obvious + alternative. +- **Definitions** - terms the folder's rules and code cite, each stated once. +- **Inventories** - the folder's override knobs, registered families, sanctioned cases: lists + a rule checks a diff against. +- **The exception ledger** - the cases a `REVIEW.md` rule fires on that are ruled acceptable, + each with its reason. A rule stays absolute; this is where its sanctioned violations live. + +**Sections are numbered, and rules cite them by section.** A section number, once cited, is +never reused for different content - append new sections, do not renumber. + +**A fact that a rule or a code comment cites is load-bearing: it must stay true.** The +same-change duty that keeps it true belongs in the folder's `REVIEW.md`, not here. + +**Plain English - short sentences, common words, no idioms.** A term of art is defined where +it first appears. A passage that needs a second read is a defect of the passage. + +**A statement earns its place by changing what someone writes or concludes.** Restating what +the code says in the same words, or describing what a reader can see in one glance at the +file, is clutter - cut it. diff --git a/LAWS.md b/LAWS.md index 1119341a1c..30c1493507 100644 --- a/LAWS.md +++ b/LAWS.md @@ -16,3 +16,5 @@ compacted, or cited as rules. | 2026-08-24 | REVIEW_COMMON.md (plain-English bullet) | "im an immigrant with russian origins. i studied english in russia. academic writing is not my idea of light reading." - yes to the proposed bullet: plain English, short sentences, common words, no idioms; a rule needing a second read is a defect | | 2026-08-27 | REVIEW_COMMON.md (rule-form bullet), .claude/agents/dragon.md, .claude/agents/janitor.md (new) | "we really have 3 types of things we state in REVIEW.md. A is 'how the system should be' ... B is 'how the system should not be'. and then there is C 'how the system is allowed to change'. and they are in the conflicting formats. i was effectively expecting B and C, and the current setup is primarily A" + "plain English rule not enforced" - rules are bans and duties, statements banned outright (the 08-24 A-form license removed); dragon gets a STATEMENT verdict and bright-line idiom flagging; a janitor agent rewrites checklists, trialed on one medium REVIEW.md first | | 2026-08-27 | REVIEW_COMMON.md (defect-spelling + no-spare-words bullets), .claude/agents/janitor.md, .claude/agents/dragon.md | "stating something is a defect is statement what a system is not. make it explicit maybe" + "lets add stronger preference on shorter wording. not spartan level, but removing all clutter. and add similar shorter wording to review.md" - "is a defect" made an explicit ban spelling; no-clutter rule added to the contract, the janitor, and the dragon's bright-line set; sweep-scope improvements ledgered, fixed at the janitor sweep | +| 2026-08-27 | ARCHITECTURE_COMMON.md (new) | "i think separate ARCHITECTURE_COMMON.md. and we link the two for the review agent on both levels. we also modify CLAUDE.md to say what in those two files are so that the information in ARCHITECTURE.md is used as the code is written" + "yes - migrate in any sensible way" - arch docs get their own contract (present tense, mood test, charters/mechanisms/definitions/inventories/exception ledger); REVIEW_COMMON cross-link, CLAUDE.md write-time section, dragon update, and MASTERPLAN migration queued behind the janitor sweep | +| 2026-08-27 | daslib/.lint_config, .claude/agents/harvester.md (new) | comment sweep redesign: "remove hard lint rune, but have a process where comments are removed - recommendations are takend - rules and facts migrated to ARCHITECTURE.md and REVIEW.md, and then most important comments reduced to one-liners and restored" + "disjoint. daslib been swept already... anything comment heavy in dasLLAMA" + flashlight for NEEDS RULING: "this is going to be massive, and way outside my ability to focus and scope properly. so only important ones survive and the rest u do your best" | diff --git a/daslib/.lint_config b/daslib/.lint_config index 03374102f6..ac824f7360 100644 --- a/daslib/.lint_config +++ b/daslib/.lint_config @@ -1,2 +1,4 @@ +# Comment stripping retired: comments are harvested - RULE/FACT comments file into +# REVIEW.md / ARCHITECTURE.md, site-local one-liners stay - not formatter-deleted. [format] -force_clean_comments = true +force_clean_comments = false From 1b2de1e18dfe0da063b9e53cf9315574b4b47407 Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Thu, 27 Aug 2026 00:53:51 -0700 Subject: [PATCH 04/15] harvester: RENAME verdict - a comment that is a rename in disguise The rescue-bot's rename suggestions survive the regime change: the harvester proposes current -> better name, and the comment stays in place until the rename lands (deleted in the same edit that renames). Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_014fKJGuUL8tP58Y6NvoVeSn --- .claude/agents/harvester.md | 12 ++++++++++-- LAWS.md | 1 + 2 files changed, 11 insertions(+), 2 deletions(-) diff --git a/.claude/agents/harvester.md b/.claude/agents/harvester.md index 448bd9b535..11cc85aec7 100644 --- a/.claude/agents/harvester.md +++ b/.claude/agents/harvester.md @@ -1,6 +1,6 @@ --- name: harvester -description: Harvests the comments of ONE source file into the document system. Classifies every comment - RULE (a ban or duty, proposed for the folder's REVIEW.md), FACT (a present-tense statement, proposed for an ARCHITECTURE.md section), KEEP (a site-local constraint, compressed to a one-liner in place), DROP (narration, restating the code, stale history), TODO (follow-up ledger candidate). Edits ONLY the source file - deletions and one-liner compressions; every REVIEW.md/ARCHITECTURE.md/ledger landing is PROPOSED in its report as exact destination text, never applied by it. Facts are disjoint - a fact filed to ARCHITECTURE.md does not also survive as a comment; KEEP is only for what the arch doc would bury. Reports a per-comment ledger plus NEEDS RULING for calls it cannot defend. +description: Harvests the comments of ONE source file into the document system. Classifies every comment - RULE (a ban or duty, proposed for the folder's REVIEW.md), FACT (a present-tense statement, proposed for an ARCHITECTURE.md section), KEEP (a site-local constraint, compressed to a one-liner in place), DROP (narration, restating the code, stale history), RENAME (the comment is a rename in disguise - proposes the better name, comment stays until the rename lands), TODO (follow-up ledger candidate). Edits ONLY the source file - deletions and one-liner compressions, never code; every REVIEW.md/ARCHITECTURE.md/ledger landing and every rename is PROPOSED in its report as exact text, never applied by it. Facts are disjoint - a fact filed to ARCHITECTURE.md does not also survive as a comment; KEEP is only for what the arch doc would bury. Reports a per-comment ledger plus NEEDS RULING for calls it cannot defend. model: opus tools: Read, Grep, Glob, Edit, Bash --- @@ -25,6 +25,12 @@ Every comment in the file gets exactly one: would bury. Compress to one line in place. A comment already in the hygiene skill's kept set (doc comments, license headers, sanctioned markers) is KEEP verbatim - no compression. - **DROP** - it narrates, restates the code, or records history. Delete it. +- **RENAME** - the comment exists because a name is wrong or vague; a better name would say + what the comment says ("// actually the padded count" over `n`). Propose the rename - + current name, proposed name, what the comment adds that the name would then carry - and + LEAVE the comment in place: it serves until the rename lands, and the session deletes it + in the same edit that renames. Judge honestly: a comment that a name cannot carry is not + a RENAME. - **TODO** - unfinished work. Propose the follow-up ledger line; delete the comment. Facts are DISJOINT: a fact proposed for the architecture doc never also survives as a @@ -47,9 +53,11 @@ silently resolved either way. Your final message is the ledger, nothing else: -1. Counts first: N comments -> R RULE / F FACT / K KEEP / D DROP / T TODO. +1. Counts first: N comments -> R RULE / F FACT / K KEEP / D DROP / RN RENAME / T TODO. 2. Every RULE, FACT, and TODO: the original comment (condensed to its point), the exact proposed destination text, and the destination (REVIEW.md; arch doc + section; ledger). + Every RENAME: the comment, current name -> proposed name, one line on what the new name + carries. 3. KEEP entries only where you compressed: before -> after, one line each. 4. DROP: one collapsed line listing the sites (line numbers), not the texts. 5. `NEEDS RULING:` - contradictions, suspected-protected classes, uncertain venue calls. diff --git a/LAWS.md b/LAWS.md index 30c1493507..ea66f9183d 100644 --- a/LAWS.md +++ b/LAWS.md @@ -18,3 +18,4 @@ compacted, or cited as rules. | 2026-08-27 | REVIEW_COMMON.md (defect-spelling + no-spare-words bullets), .claude/agents/janitor.md, .claude/agents/dragon.md | "stating something is a defect is statement what a system is not. make it explicit maybe" + "lets add stronger preference on shorter wording. not spartan level, but removing all clutter. and add similar shorter wording to review.md" - "is a defect" made an explicit ban spelling; no-clutter rule added to the contract, the janitor, and the dragon's bright-line set; sweep-scope improvements ledgered, fixed at the janitor sweep | | 2026-08-27 | ARCHITECTURE_COMMON.md (new) | "i think separate ARCHITECTURE_COMMON.md. and we link the two for the review agent on both levels. we also modify CLAUDE.md to say what in those two files are so that the information in ARCHITECTURE.md is used as the code is written" + "yes - migrate in any sensible way" - arch docs get their own contract (present tense, mood test, charters/mechanisms/definitions/inventories/exception ledger); REVIEW_COMMON cross-link, CLAUDE.md write-time section, dragon update, and MASTERPLAN migration queued behind the janitor sweep | | 2026-08-27 | daslib/.lint_config, .claude/agents/harvester.md (new) | comment sweep redesign: "remove hard lint rune, but have a process where comments are removed - recommendations are takend - rules and facts migrated to ARCHITECTURE.md and REVIEW.md, and then most important comments reduced to one-liners and restored" + "disjoint. daslib been swept already... anything comment heavy in dasLLAMA" + flashlight for NEEDS RULING: "this is going to be massive, and way outside my ability to focus and scope properly. so only important ones survive and the rest u do your best" | +| 2026-08-27 | .claude/agents/harvester.md (RENAME verdict) | "what the previous resque-boat agent did for comments - the most interesting part - is suggest renames. like 'this comment is rename in disguise'. i don't want to loose that at all. its important" - RENAME added to the harvester's verdicts; comment stays until the rename lands | From c14fd11767def1049d9c596fccfc3a4a08187ce9 Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Thu, 27 Aug 2026 00:54:27 -0700 Subject: [PATCH 05/15] harvester: RENAME tested first - the strongest resolution A name that carries the information makes every other filing unnecessary; only a comment no name can carry goes on to RULE/FACT/KEEP/DROP/TODO. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_014fKJGuUL8tP58Y6NvoVeSn --- .claude/agents/harvester.md | 18 ++++++++++-------- LAWS.md | 1 + 2 files changed, 11 insertions(+), 8 deletions(-) diff --git a/.claude/agents/harvester.md b/.claude/agents/harvester.md index 11cc85aec7..26efba36f9 100644 --- a/.claude/agents/harvester.md +++ b/.claude/agents/harvester.md @@ -1,6 +1,6 @@ --- name: harvester -description: Harvests the comments of ONE source file into the document system. Classifies every comment - RULE (a ban or duty, proposed for the folder's REVIEW.md), FACT (a present-tense statement, proposed for an ARCHITECTURE.md section), KEEP (a site-local constraint, compressed to a one-liner in place), DROP (narration, restating the code, stale history), RENAME (the comment is a rename in disguise - proposes the better name, comment stays until the rename lands), TODO (follow-up ledger candidate). Edits ONLY the source file - deletions and one-liner compressions, never code; every REVIEW.md/ARCHITECTURE.md/ledger landing and every rename is PROPOSED in its report as exact text, never applied by it. Facts are disjoint - a fact filed to ARCHITECTURE.md does not also survive as a comment; KEEP is only for what the arch doc would bury. Reports a per-comment ledger plus NEEDS RULING for calls it cannot defend. +description: Harvests the comments of ONE source file into the document system. Classifies every comment - RULE (a ban or duty, proposed for the folder's REVIEW.md), FACT (a present-tense statement, proposed for an ARCHITECTURE.md section), KEEP (a site-local constraint, compressed to a one-liner in place), DROP (narration, restating the code, stale history), RENAME (the comment is a rename in disguise - proposes the better name, comment stays until the rename lands; tested FIRST, before any filing - it is the strongest resolution), TODO (follow-up ledger candidate). Edits ONLY the source file - deletions and one-liner compressions, never code; every REVIEW.md/ARCHITECTURE.md/ledger landing and every rename is PROPOSED in its report as exact text, never applied by it. Facts are disjoint - a fact filed to ARCHITECTURE.md does not also survive as a comment; KEEP is only for what the arch doc would bury. Reports a per-comment ledger plus NEEDS RULING for calls it cannot defend. model: opus tools: Read, Grep, Glob, Edit, Bash --- @@ -14,8 +14,16 @@ carries), then the whole target file. ## The verdicts -Every comment in the file gets exactly one: +Every comment in the file gets exactly one. **RENAME is tested FIRST, before any other +verdict: it is the strongest resolution** - a name that carries the information makes every +other filing unnecessary. Only a comment no name can carry goes on to the rest. +- **RENAME** - the comment exists because a name is wrong or vague; a better name would say + what the comment says ("// actually the padded count" over `n`). Propose the rename - + current name, proposed name, what the comment adds that the name would then carry - and + LEAVE the comment in place: it serves until the rename lands, and the session deletes it + in the same edit that renames. Judge honestly: a comment that a name cannot carry is not + a RENAME. - **RULE** - it tells a future diff what it must or must not do. Propose it for the folder's `REVIEW.md`, rewritten as a ban or duty per `REVIEW_COMMON.md`. Delete the comment. - **FACT** - it states how the system is or why its shape wins. Propose it for the @@ -25,12 +33,6 @@ Every comment in the file gets exactly one: would bury. Compress to one line in place. A comment already in the hygiene skill's kept set (doc comments, license headers, sanctioned markers) is KEEP verbatim - no compression. - **DROP** - it narrates, restates the code, or records history. Delete it. -- **RENAME** - the comment exists because a name is wrong or vague; a better name would say - what the comment says ("// actually the padded count" over `n`). Propose the rename - - current name, proposed name, what the comment adds that the name would then carry - and - LEAVE the comment in place: it serves until the rename lands, and the session deletes it - in the same edit that renames. Judge honestly: a comment that a name cannot carry is not - a RENAME. - **TODO** - unfinished work. Propose the follow-up ledger line; delete the comment. Facts are DISJOINT: a fact proposed for the architecture doc never also survives as a diff --git a/LAWS.md b/LAWS.md index ea66f9183d..11136b1e75 100644 --- a/LAWS.md +++ b/LAWS.md @@ -19,3 +19,4 @@ compacted, or cited as rules. | 2026-08-27 | ARCHITECTURE_COMMON.md (new) | "i think separate ARCHITECTURE_COMMON.md. and we link the two for the review agent on both levels. we also modify CLAUDE.md to say what in those two files are so that the information in ARCHITECTURE.md is used as the code is written" + "yes - migrate in any sensible way" - arch docs get their own contract (present tense, mood test, charters/mechanisms/definitions/inventories/exception ledger); REVIEW_COMMON cross-link, CLAUDE.md write-time section, dragon update, and MASTERPLAN migration queued behind the janitor sweep | | 2026-08-27 | daslib/.lint_config, .claude/agents/harvester.md (new) | comment sweep redesign: "remove hard lint rune, but have a process where comments are removed - recommendations are takend - rules and facts migrated to ARCHITECTURE.md and REVIEW.md, and then most important comments reduced to one-liners and restored" + "disjoint. daslib been swept already... anything comment heavy in dasLLAMA" + flashlight for NEEDS RULING: "this is going to be massive, and way outside my ability to focus and scope properly. so only important ones survive and the rest u do your best" | | 2026-08-27 | .claude/agents/harvester.md (RENAME verdict) | "what the previous resque-boat agent did for comments - the most interesting part - is suggest renames. like 'this comment is rename in disguise'. i don't want to loose that at all. its important" - RENAME added to the harvester's verdicts; comment stays until the rename lands | +| 2026-08-27 | .claude/agents/harvester.md (RENAME precedence) | "rename should be first option, before any rescue attempts. it is the strongest." - RENAME tested before RULE/FACT/KEEP/DROP/TODO; only a comment no name can carry goes on to filing | From 53d3043b6dbb8bd789d72956027ef2aed14cbcd7 Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Thu, 27 Aug 2026 01:04:06 -0700 Subject: [PATCH 06/15] janitor sweep: 49 checklists translated to the modal contract Every rule now a ban or a duty binding the diff, plain English, no spare words; statements inverted or queued. Raw janitor output - the ruling applications land as the next commit. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_014fKJGuUL8tP58Y6NvoVeSn --- .github/workflows/REVIEW.md | 14 +- ci/REVIEW.md | 6 +- daslib/REVIEW.md | 227 ++++++------ daslib/REVIEW_LINQ.md | 342 +++++++++--------- dastest/REVIEW.md | 4 +- include/daScript/simulate/REVIEW.md | 29 +- modules/REVIEW_SHADER_EMITTERS.md | 43 ++- modules/dasImgui/REVIEW.md | 12 +- modules/dasImgui/tests/REVIEW.md | 6 +- modules/dasLLAMA/REVIEW.md | 149 ++++---- modules/dasLLAMA/REVIEW_AUDIO.md | 42 +-- modules/dasLLAMA/REVIEW_GPU.md | 253 ++++++------- modules/dasLLAMA/REVIEW_VISION.md | 61 ++-- modules/dasLLAMA/benchmarks/REVIEW.md | 86 ++--- modules/dasLLAMA/performance/REVIEW.md | 91 +++-- modules/dasLLAMA/tests/REVIEW.md | 210 +++++------ modules/dasLLVM/REVIEW.md | 48 +-- modules/dasMetal/REVIEW.md | 26 +- modules/dasSpirv/REVIEW.md | 22 +- modules/dasVulkan/REVIEW.md | 11 +- modules/dasVulkan/generator/REVIEW.md | 6 +- modules/dasVulkan/tutorials/REVIEW.md | 8 +- nano/REVIEW.md | 76 ++-- site-dasllama/REVIEW.md | 78 ++-- site/REVIEW.md | 114 +++--- src/builtin/REVIEW.md | 29 +- src/parser/REVIEW.md | 8 +- tests-cpp/big/REVIEW.md | 4 +- tests-cpp/small/REVIEW.md | 8 +- tests/metal/REVIEW.md | 10 +- tests/msl/REVIEW.md | 13 +- tree-sitter-daslang/REVIEW.md | 15 +- utils/REVIEW.md | 33 +- utils/benchctl/REVIEW.md | 6 +- utils/das-fmt/REVIEW.md | 5 +- utils/dascov/REVIEW.md | 5 +- utils/dasllama-server/REVIEW.md | 32 +- utils/daspkg/REVIEW.md | 56 +-- utils/detect-dupe/REVIEW.md | 6 +- utils/internal/dasllama-ladder/REVIEW.md | 195 +++++----- utils/internal/dasweb-buildd/REVIEW.md | 119 +++--- utils/internal/dasweb-playground/REVIEW.md | 146 ++++---- utils/internal/dasweb-verify/REVIEW.md | 31 +- .../internal/dasweb-verify/browser/REVIEW.md | 32 +- utils/lint/REVIEW.md | 14 +- utils/lsp/REVIEW.md | 6 +- utils/mcp/REVIEW.md | 6 +- web/examples/ui/REVIEW.md | 4 +- .../samples/examples/boulder-dash/REVIEW.md | 39 +- 49 files changed, 1395 insertions(+), 1391 deletions(-) diff --git a/.github/workflows/REVIEW.md b/.github/workflows/REVIEW.md index 3bb9a3027f..e975aa57b7 100644 --- a/.github/workflows/REVIEW.md +++ b/.github/workflows/REVIEW.md @@ -3,11 +3,11 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `skills/internal/preflight.md` (repo root). -**A per-PR gate step in `doc.yml` or `extended_checks.yml` keeps checking every tracked source -it checked before the diff - deleting the step, adding `continue-on-error`, narrowing its -`if:`, weakening its command, or shrinking the tracked-source set it covers is a defect.** -Each such step machine-enforces an invariant with no reviewer involved - -`skills/internal/preflight.md` sec."doc.yml - the gates", sec."extended_checks.yml". +**A diff that deletes a per-PR gate step in `doc.yml` or `extended_checks.yml`, adds +`continue-on-error` to it, narrows its `if:`, weakens its command, or shrinks the set of +tracked source files it checks is a defect.** Each such step enforces its rule automatically, +with no reviewer involved - `skills/internal/preflight.md` sec."doc.yml - the gates", +sec."extended_checks.yml". -**A per-PR gate step in `doc.yml` or `extended_checks.yml` fails the lane on a defect in -what it checks, and only on such a defect.** +**A diff that adds or changes a per-PR gate step in `doc.yml` or `extended_checks.yml` makes +that step fail the lane on every defect in what it checks, and on nothing else.** diff --git a/ci/REVIEW.md b/ci/REVIEW.md index 062eb34e20..9ac90749e7 100644 --- a/ci/REVIEW.md +++ b/ci/REVIEW.md @@ -3,6 +3,6 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `CLAUDE.md` (repo root). -**Weakening `smoke_test_bundle.sh` - the release-bundle smoke gate - is a defect.** What it -enforces is read from the script: dropping a check, loosening an assertion, or shortening a -list it walks each weaken it. +**A diff that weakens `smoke_test_bundle.sh` - the script that checks a built release bundle - +is a defect.** Weakening means dropping a check, loosening an assertion, or shortening a list +the script walks. diff --git a/daslib/REVIEW.md b/daslib/REVIEW.md index e547ee03c2..7e8eafe848 100644 --- a/daslib/REVIEW.md +++ b/daslib/REVIEW.md @@ -4,32 +4,33 @@ `daslib/ARCHITECTURE.md`. Planned work: `daslib/followup_comment_sweep.md`. A diff touching the linq family - `linq*.das`, `sql_*.das` - applies `REVIEW_LINQ.md` too. -**A lint report path (`perf_warning` / `style_warning` / `lint_error`) checks suppression BEFORE -its dedup insert.** A nolint'd finding that takes the (rule, location) slot silences a -co-located sibling rule. +**A diff that adds or changes a lint report path (`perf_warning` / `style_warning` / +`lint_error`) checks suppression before that path's dedup insert.** A nolint'd finding that +takes the (rule, location) slot silences a co-located sibling rule. -**The rule code stays part of the dedup key.** A key without the code collides across rules. +**A diff that changes the dedup key keeps the rule code in it.** A key without the code +collides across rules. -**PERF026-028 honor a `// nolint` at ANY frame of the reported chain.** The sink often -bottoms out in daslib while the actionable line is an intermediate call site - narrowing the -check to the sink line is a defect. +**Never narrow PERF026-028's `// nolint` check to the reported sink line - honor a +`// nolint` at any frame of the reported chain.** The sink often bottoms out in daslib while +the actionable line is an intermediate call site. -**Moving an `in_closure` / `in_deferred` guard inside a lint visitor method is a semantic -change, never cleanup.** The guard's position encodes which rules are meaningful inside a -lambda; the per-rule policy is in `ARCHITECTURE.md` sec. perf_lint. +**A diff that moves an `in_closure` / `in_deferred` guard inside a lint visitor method as +cleanup is a defect.** The guard's position decides which rules apply inside a lambda; the +per-rule policy is in `ARCHITECTURE.md` sec. perf_lint. -**A lint warning anchors at the code its fix rewrites.** A remedy that deletes a statement -reports the statement; a remedy that edits one variable's initializer reports the variable - -an anchor chosen for implementation convenience instead is a defect. +**A lint warning that anchors anywhere but the code its fix rewrites is a defect.** A remedy +that deletes a statement reports the statement; a remedy that edits one variable's +initializer reports the variable. -**Visitor state scoped to a construct is reset or restored on every exit path of that -construct.** +**Never exit a construct's visitor on a path that skips resetting or restoring the visitor +state scoped to that construct.** -**Per-loop visitor state is a stack, never a bare scalar.** A scalar survives into the -sibling loop's exit path and unbalances its counter. +**Never keep per-loop visitor state in a bare scalar - keep it in a stack.** A scalar +survives into the sibling loop's exit path and unbalances its counter. -**Per-function visitor state resets in `preVisitFunction`, ahead of any early return.** -A latch that skips the template early-return poisons the next function. +**Never let `preVisitFunction` return before it resets the per-function visitor state.** +State an early return leaves set poisons the next function. **A diff that makes a daslib predicate or emitted identifier depend on a C++-side definition records the pair in `ARCHITECTURE.md`, in its module's section, naming both @@ -44,11 +45,13 @@ pins that a string literal, a URL, and a mid-comment `nolint:` do not suppress w first-token directive after `//` or `//!` does, and `tests/lint/test_stale_nolint.das` pins that a `lint-skip-file` past the header window is prose. -**`build_lint_macro_disabled` layers four sources in this order: defaults, repo `off`, repo -`on`, environment.** Env last lets a one-run `DAS_LINT_DISABLE` beat a `CODE = true`. +**A diff that changes `build_lint_macro_disabled` keeps its four sources layered in this +order: defaults, repo `off`, repo `on`, environment.** Env last lets a one-run +`DAS_LINT_DISABLE` beat a `CODE = true`. -**`options _enable_default_off_rules` skips BOTH the default seeding and the repo `off` -directives.** Repo policy must not silence the rule a fixture exists to exercise. +**A diff that changes `options _enable_default_off_rules` keeps it skipping BOTH the default +seeding and the repo `off` directives.** Repo policy must not silence the rule a fixture +exists to exercise. **Weakening `tests/lint/test_lint_config.das` is a defect** - it pins that each `[format]` key resolves nearest-wins independently up to the `.git` root, so a nearer `.lint_config` @@ -61,36 +64,37 @@ the formatter's kept set. in the same change.** An unlisted module's ids are never scanned, so its fixture-and-rst check never runs. -**LINT010 records a store in the POST-visit of `ExprCopy`/`ExprClone`/`ExprMove`, never in -`preVisitExprVar`.** The LHS's variable fires before the RHS is walked; an early record -lets the RHS's own read clear the store it just recorded. +**Never record a LINT010 store in `preVisitExprVar` - record it in the POST-visit of +`ExprCopy`/`ExprClone`/`ExprMove`.** The LHS's variable fires before the RHS is walked, so +an early record lets the RHS's own read clear the store it just recorded. -**A rule identifying a callee by name matches the root generic, not the instance** - -instantiation mangles names and strips witness arguments. +**Never match a lint rule's callee name against the instance - match the root generic.** +Instantiation mangles names and strips witness arguments. -**A rule never fires inside the module whose idiom it advertises** - a rule advertising -module M's idiom firing on M's own dispatch is a defect. +**A lint rule that fires inside the module whose idiom it advertises is a defect.** -**A collapse suggestion compiles and preserves semantics for the exact shape reported** - -gate on receiver type, arity, cloneability, and duplicate constant keys. +**A diff that adds or changes a collapse suggestion gates it on receiver type, arity, +cloneability, and duplicate constant keys, so the suggestion compiles and keeps the reported +shape's semantics.** -**Exactly one sibling rule fires on a shape several collapse rules can match.** Overlaps +**A diff that makes two sibling collapse rules fire on one shape is a defect.** Overlaps stay partitioned: STYLE021 owns the const-key run over STYLE031, STYLE032 owns fresh-empty declarations over STYLE033, and STYLE033's chain-receiver check leaves a plain-variable -receiver to STYLE033's variable-form check, so the two never double-fire. +receiver to STYLE033's variable-form check. -**The lint message text is the rule-code transport: the code is everything before the -FIRST colon.** Suppression, disable lists, and dedup all parse it from the message - -rewording a message so the code is not the leading colon-delimited token makes the rule -unsuppressable and self-colliding. +**Never reword a lint message so the rule code stops being everything before the FIRST +colon.** Suppression, disable lists, and dedup all parse the code from the message, so a +message that leads with anything else makes the rule unsuppressable and self-colliding. -**A `canVisit*` override in style_lint ships with the unsafe-frame push/pop taught about -the subtree it skips.** The walk pushes one frame per expression and pops it, so a skipped -subtree unbalances the count; the balance panic in each entry point is the tripwire. +**A diff that adds or changes a `canVisit*` override in style_lint teaches the unsafe-frame +push/pop about the subtree the override skips.** The walk pushes one frame per expression +and pops it, so a skipped subtree unbalances the count; the balance panic in each entry +point is the tripwire. -**The arity caps mirror overload sets that live elsewhere** - `MAX_CONCAT_ARITY` <-> linq's -`concat` variadics, `MAX_VARIADIC_PUSH_ARITY` <-> builtin `push_from`/`push_clone_from`; -growing either overload set without the cap is a silently missed finding, the reverse a +**A diff that changes an arity cap or the overload set it mirrors - `MAX_CONCAT_ARITY` <-> +linq's `concat` variadics, `MAX_VARIADIC_PUSH_ARITY` <-> builtin +`push_from`/`push_clone_from` - changes both sides in the same change.** Growing an overload +set without the cap is a silently missed finding; raising a cap without the overloads is a suggestion that does not compile. **A diff that adds or changes an emit entry point - a function that runs the emit visitor @@ -100,12 +104,12 @@ or write.** The error check is the program's `macroException`/`failToCompile` st directly or through `log_aot_emit_errors`; a codegen exception mid-visit leaves partial C++. -**A visitor override in `CppAot` or any subclass of it never gates on -`macroException`/`failToCompile`.** The function that runs the visitor owns that check; an +**Never gate a visitor override in `CppAot` or any subclass of it on +`macroException`/`failToCompile` - the function that runs the visitor owns that check.** An override that returns early on the error state emits truncated C++ that the caller still writes out as complete. -**An unreachable emit state writes `#error` into the output, never `panic`** - +**Never report an unreachable emit state with `panic` - write `#error` into the output.** `runMacroFunction` swallows a panic, so the emitter never reports through it. **A diff that changes how a struct's C++ name is built changes every site that spells @@ -120,71 +124,74 @@ keeps `buildStructEnumCollisions` running before that visitor runs - directly or helper it calls.** The table decides when a name gets its collision suffix, and a run that skips the seeding spells structs differently from the run that seeded it. -**`match_error` stores a BORROWED `LineInfo` pointer - pass a pattern node's location, -never a synthesized access expression's.** Access nodes are cloned per field inside a bare -scope and die with it. +**Never pass a synthesized access expression's location to `match_error` - pass a pattern +node's location.** `match_error` stores the `LineInfo` pointer BORROWED, and access nodes +are cloned per field inside a bare scope and die with it. -**A macro that lowers lambdas reports through `macro_sticky_error`, never `macro_error`.** -A later pass lowers the lambda to a plain function and clears non-sticky errors. +**Never report from a macro that lowers lambdas through `macro_error` - report through +`macro_sticky_error`.** A later pass lowers the lambda to a plain function and clears +non-sticky errors. -**UTF-8 byte-class tables are indexed through `uint(uint8(ch))`** - `for (ch in string)` -yields a SIGNED byte under JIT; a raw index reads out of bounds for every byte >= 0x80. +**Never index a UTF-8 byte-class table with a raw `ch` - index through `uint(uint8(ch))`.** +`for (ch in string)` yields a SIGNED byte under JIT, so a raw index reads out of bounds for +every byte >= 0x80. -**A parser result that borrows a view into a tree the parser allocated names the owning -field and ships the scope-ender that frees it.** Nothing else can free it - the tree must +**A diff that adds or changes a parser result that borrows a view into a tree the parser +allocated names the owning field and ships the scope-ender that frees it.** The tree must outlive the caller's read, and daslang finalizes neither a raw pointer field nor a local -container at scope exit - so a borrowed view with no named owner leaks the whole document -per call, invisibly until the process is long-lived. +container at scope exit, so a borrowed view with no named owner leaks the whole document per +call. -**A conversion that throws on out-of-range input is a panic on untrusted bytes.** `int64` / -`uint64` / `double` on a string throw; a lexer or decoder reachable from a file, a socket, -or a model reaches for the non-throwing `to_*` twin and reports through its own error -channel. +**Never convert a string with `int64` / `uint64` / `double` in a lexer or decoder reachable +from a file, a socket, or a model - use the non-throwing `to_*` twin and report through the +code's own error channel.** Those three throw on out-of-range input, and a throw on +untrusted bytes is a panic. -**A parser never re-derives control flow from the text of its own diagnostics.** A message -carries user data, so `starts_with` on an error string is an input-controlled branch; the -token that caused the failure is what the decision reads. +**Never branch a parser on the text of its own diagnostics - branch on the token that caused +the failure.** A message carries user data, so `starts_with` on an error string is an +input-controlled branch. -**Every flatten_opt rewrite arm ships a read-only residual predicate** - the oracle the -residual visitors call to prove the pass complete. +**A diff that adds or changes a flatten_opt rewrite arm ships a read-only residual predicate +with that arm** - the oracle the residual visitors call to prove the pass complete. -**A residual oracle mirrors its arm's gate exactly and never calls the transform.** A -narrower oracle is a false pass, a wider one a false miss; calling the transform from an -oracle aliases the live tree. A gate the transform takes as a parameter is threaded into the -oracle too - re-spelling its default as a constant makes every non-default run a false pass, -and the tests that call the oracle pass the same value. +**Never let a residual oracle's gate differ from its arm's gate, and never call the +transform from an oracle.** A gate the transform takes as a parameter is threaded into the +oracle too, and the tests that call the oracle pass the same value. A narrower oracle is a +false pass, a wider one a false miss, a re-spelled parameter default a false pass on every +non-default run; calling the transform aliases the live tree. -**Every new fold/fuse arm declares its float class.** Inf/NaN/rounding/association changes -are fast-math-only; bit-exact per-lane rewrites are never gated. An arm added without that -decision silently changes output under `_flatten_no_fast_math`. +**A diff that adds a fold/fuse arm declares that arm's float class.** +Inf/NaN/rounding/association changes are fast-math-only; bit-exact per-lane rewrites are +never gated. An arm added without that decision silently changes output under +`_flatten_no_fast_math`. -**A fuse arm emits a call only after proving the target module can resolve it**, and the -miss path falls back to the unfused shape - the pass must never turn a shader that compiled -into an unresolvable call on a narrower backend. +**Never emit a fused call before proving the target module can resolve it - fall back to the +unfused shape on a miss.** Otherwise the pass turns a shader that compiled into an +unresolvable call on a narrower backend. **A new store spelling joins `MutCollect` in the same change.** CSE reads a name outside the mutable set as constant for the whole block, so an uncollected store is a silently shared subexpression. -**A statement the lowering cannot predicate is refused, never dropped.** `lower_stmt`'s +**Never drop a statement the lowering cannot predicate - refuse it.** `lower_stmt`'s fall-through drop is licensed only because every call that survives lowering is pure. -**`flatten_function` runs its passes in this order - dse -> copy-prop -> mask-const-prop -> -dse -> ssa-rename - and a diff that reorders them is a defect.** Each pass produces the next -one's input; a reorder leaves scaffolding in the generated `_flat` twin, which must -come out branchless and call-free. +**A diff that reorders `flatten_function`'s passes is a defect - the order is dse -> +copy-prop -> mask-const-prop -> dse -> ssa-rename.** Each pass produces the next one's +input, and a reorder leaves scaffolding in the generated `_flat` twin, which must come +out branchless and call-free. -**`lift_expr` and the lowering it drives use plain recursion, never `make_visitor`** - -inlining recurses back through the pipeline, and a nested visitor traversal corrupts the +**Never use `make_visitor` in `lift_expr` or the lowering it drives - use plain recursion.** +Inlining recurses back through the pipeline, and a nested visitor traversal corrupts the visitor machinery. -**A macro that splices the same subexpression more than once pre-binds it to a local ref -first** - re-splicing re-evaluates a call once per splice; a lock/unlock pair then releases -a different temporary than it took. +**Never splice the same subexpression more than once in a macro - pre-bind it to a local ref +and splice that.** Re-splicing re-evaluates a call once per splice, so a lock/unlock pair +releases a different temporary than it took. -**Every `[sql_table]` helper pair registers its 2-arg form BEFORE the 1-arg form** - -`find_struct_helper_fn` keeps the LAST match, and the finish pass rewrites the helper it -returns; swapped, index DDL attaches to the wrong overload. +**A diff that adds or changes a `[sql_table]` helper pair registers its 2-arg form BEFORE +the 1-arg form.** `find_struct_helper_fn` keeps the LAST match and the finish pass rewrites +the helper it returns, so a swapped pair attaches index DDL to the wrong overload. **A generated SQL statement's column list and its bind function change in one edit.** Bind indices are placeholder positions over bindable columns, never struct field positions; @@ -194,36 +201,36 @@ drift is silently wrong data, not an error. the same change.** The scanners match daslang source text; a missing spelling makes every enforcement test pass vacuously. -**An `ast_verify` check names, in `ARCHITECTURE.md` sec. ast_verify, the C++ `::visit` site -that dereferences the slot unguarded.** A check with no such site invents an invariant the -compiler does not hold, and the two passes disagree - a slot infer fills in is checkable -only post-infer. +**A diff that adds or changes an `ast_verify` check names, in `ARCHITECTURE.md` sec. +ast_verify, the C++ `::visit` site that dereferences the slot unguarded.** A check with no +such site invents an invariant the compiler does not hold, and the two passes disagree - a +slot infer fills in is checkable only post-infer. -**An `ast_verify` check repairs the slot it reports, not only reports it.** A broken slot -left in place trips every later pass that walks the same node. +**A diff that adds or changes an `ast_verify` check repairs the slot it reports, not only +reports it.** A broken slot left in place trips every later pass that walks the same node. -**64-bit range sums prove both operands non-negative BEFORE summing as `uint64`** +**Never sum a 64-bit range as `uint64` before proving both operands non-negative** (`erase(at, count)`): a signed sum wraps, the bounds test passes, and the walk leaves the array silently. -**Macro-built AST carries no branch a macro-time value can decide** - nothing folds at -macro-application time, so a generated `if ($v(flag))` keeps its dead arm and type-checks -it. Branch in daslang and emit only the taken arm. +**Never emit a branch a macro-time value can decide from macro-built AST - branch in daslang +and emit only the taken arm.** Nothing folds at macro-application time, so a generated +`if ($v(flag))` keeps its dead arm and type-checks it. -**Every buffer-I/O overload returns before taking `addr(buf[0])` on an empty buffer** - -the address is out of bounds and the call sits inside `unsafe`; a new overload without the -guard passes every non-empty test. +**A diff that adds or changes a buffer-I/O overload returns before taking `addr(buf[0])` on +an empty buffer.** The address is out of bounds and the call sits inside `unsafe`, so an +overload without the guard passes every non-empty test. -**`apply`'s inline path decides lvalue-ness by node KIND, never by a ref flag** - an rvalue -stays materialized, because a reference to a temporary dangles. +**Never decide lvalue-ness in `apply`'s inline path from a ref flag - decide it by node +KIND.** An rvalue stays materialized, because a reference to a temporary dangles. -**A `let` generated by `apply`'s inline path carries `alwaysSafe`** - that flag, not an +**Never generate a `let` on `apply`'s inline path without `alwaysSafe`** - that flag, not an `unsafe` block, licenses the variant access. -**A swizzle rewrite reuses each source node once and clones every repeat** - the first -appearance moves, a second output lane MUST clone, or one node gets two parents. +**Never reuse a swizzle rewrite's source node in a second output lane without cloning it** - +the first appearance moves it, and skipping the clone gives one node two parents. **A diff that changes how the RST label or topic key is computed makes the same change in every place `rst.das` computes it - the stub pass (`generate_module_stubs`) and the -documenting pass (`documents`) - keeping them byte-for-byte equal.** Diverged, the page +documenting pass (`documents`) - keeping them byte-for-byte equal.** When the two diverge, the page prints a bare signature and the symbol re-stubs. diff --git a/daslib/REVIEW_LINQ.md b/daslib/REVIEW_LINQ.md index 0210d066c1..31e3fa09c3 100644 --- a/daslib/REVIEW_LINQ.md +++ b/daslib/REVIEW_LINQ.md @@ -5,77 +5,74 @@ bind-side helpers, `sql_boost.das` and `sql_provider.das` included) - applies th together with `REVIEW.md`.** `REVIEW_COMMON.md` (repo root) binds this file too. Architecture doc: `ARCHITECTURE.md`. -**A fused emit binds a terminator's default / compare argument ONCE, at the top of the -generated invoke, eagerly** - even on paths that never use it, because the tier-2 `linq.das` -overload evaluates it that way. This includes every `*_or_default` decs lane: an empty-tail -fast path that evaluates the default lazily diverges from the walk lane and the iterator -fallback. - -**A fused emit reproduces the tier-2 `linq.das` overload's empty-source behavior: it panics -or yields the default exactly where tier-2 does.** A fused `first` over a prefilter buffer -panics BEFORE reaching `min`/`max` - those return an uninitialized reference on an empty -array, so the guard is load-bearing, not defensive. - -**`count` / `long_count`'s second argument is a PREDICATE; `sum` / `min` / `max` / -`average`'s is a SELECTOR.** The 2-arg reducer set (`is_bucket_reducer_call`) admits only -the latter - adding `count` or `first` to it splices a predicate into a projection slot. - -**In the distinct lane the dedup insert is not gated by the terminator's `count(pred)` -predicate - only the counter is.** A `where_` filter gating the whole per-element body is -expected; `distinct[_by]` keeps the FIRST occurrence per key, so moving the insert under the -terminator predicate changes which occurrence represents the key. - -**A fold-emitted table probe binds by reference off `unsafe(tab?[k])` - a borrowed pointer -into table storage - so the generated invoke never inserts into or erases from the probed -table.** A rehash dangles the bind into a use-after-free. Both the join probe -(`build_join_probe_pieces`) and the point-lookup probe (`try_table_point_lookup`) rest on -this. - -**`TableAdapter.invoke_param_type` returns the table parameter type with `constant` set.** -That const is what proves the point-lookup probe's borrowed bind survives the invoke; a -param that loses it turns the bind into a use-after-free. - -**A per-field WALK over a row-usage set is sorted first (or ordered by an external -declaration-order list); a membership test needs no order.** `collect_row_usage` / -`collect_decs_tup_usage` return names in `table` key order, so an unsorted walk -makes the emitted AST a function of hash order instead of the chain - AST-shape tests and -AOT semantic hashes both key on that shape. - -**Two emit sites reachable at the same `at` use different `qn` prefixes.** `qn` keys only on -(prefix, at), so a shared prefix shadows a bind instead of colliding loudly - a `group_by` + -`first`/`order`/`distinct`/`take` chain reaches both the array-walk materializer and the -handle materializer at one `at`. - -**Every implementation of the composite-key ordering changes with the others.** `key_less` -/ `less_masked` (`linq.das`) sort eagerly; `try_make_inline_cmp_keys` (`linq_fold_common.das`) -emits an inline `_::less` if-chain for the same chain. Both spell bit `i` of `mask` as "key -`i` descending" (LSB = first key), flip operand order for descending, and break ties in key -order - a change on one side makes a spliced chain and its tier-2 fallback sort differently, -silently. - -**The inline `_::less` if-chain's key cap and `less_masked`'s tuple overloads change -together.** A key tuple longer than the cap declines the splice and sorts eagerly; raising -the cap without adding the matching `less_masked` overload leaves a spliced chain with no -tier-2 twin to agree with. - -**`top_n*` over an ITERATOR never reserves `n`.** Cardinality is unknown, so a caller -passing `n` far above the element count would allocate the whole `n` upfront for no win; the -bounded-heap fill phase relies on geometric growth. - -**Every linq call-macro that synthesizes the `_` block parameter sets `can_shadow` on it.** -Chains nest (`_where(_.x._in(arr |> _select(_.y)))`) and the inner `_` has to shadow the -outer one. - -**The `_sql` hand-off precedes chain normalization AND the empty-chain bail.** `_sql` -re-expands the pristine expression on the next infer pass, and a `[sql_table]` source with -no chain ops at all is still SQL - moving the check below `normalize_order_reverse` / -`collapse_chained_*`, or below `if (empty(calls))`, routes those chains to the in-memory -tier instead. - -**A pattern row whose captured `select` (the `srcsel` slot) cannot be wrapped into a -projected adapter skips the row - it never emits.** The row's emit assumes the projected -element; running it on the un-projected adapter orders or dedups raw rows - a wrong result -rather than a missed splice. +**A fused emit that binds a terminator's default or compare argument lazily, more than once, or +below the top of the generated invoke, is a defect** - bind it once at the top, even on paths +that never use it, because the tier-2 `linq.das` overload evaluates it that way. This covers +every `*_or_default` decs lane, including an empty-tail fast path that would evaluate the +default lazily. + +**A fused emit whose empty-source behavior differs from the tier-2 `linq.das` overload's - +panicking where tier-2 yields the default, or yielding a default where tier-2 panics - is a +defect.** A fused `first` over a prefilter buffer panics before it reaches `min`/`max`, which +return an uninitialized reference on an empty array. + +**Never add `count` or `first` to the 2-arg reducer set (`is_bucket_reducer_call`) - it admits +only a reducer whose second argument is a selector.** `sum` / `min` / `max` / `average` take a +selector there; `count` / `long_count` take a predicate, and a predicate spliced into a +projection slot is silently wrong. + +**Never gate the distinct lane's dedup insert on the terminator's `count(pred)` predicate - that +predicate gates the counter only.** `distinct[_by]` keeps the first occurrence per key, so an +insert under that predicate changes which occurrence represents the key. A `where_` filter +gating the whole per-element body is expected. + +**A generated invoke that inserts into or erases from a table it probes by reference off +`unsafe(tab?[k])` is a defect** - the bind is a borrowed pointer into table storage, and a rehash +turns it into a use-after-free. This binds both the join probe (`build_join_probe_pieces`) and +the point-lookup probe (`try_table_point_lookup`). + +**A diff that leaves `constant` unset on the table parameter type +`TableAdapter.invoke_param_type` returns is a defect.** That const proves the point-lookup +probe's borrowed bind survives the invoke; without it the bind is a use-after-free. + +**A per-field walk over a row-usage set that is neither sorted first nor ordered by an external +declaration-order list is a defect** - a membership test over the same set needs no order. +`collect_row_usage` / `collect_decs_tup_usage` return names in `table` key order, so an +unsorted walk makes the emitted AST a function of hash order instead of the chain, which +AST-shape tests and AOT semantic hashes both pin. + +**Two emit sites reachable at the same `at` that share one `qn` prefix are a defect - give each +site its own prefix.** `qn` keys only on (prefix, at), so a shared prefix shadows a bind instead +of colliding loudly. + +**A diff that changes one implementation of the composite-key ordering also changes the others, +in the same change** - `key_less` / `less_masked` (`linq.das`), which sort eagerly, and +`try_make_inline_cmp_keys` (`linq_fold_common.das`), which emits an inline `_::less` if-chain for +the same chain. All spell bit `i` of `mask` as "key `i` descending" (LSB = first key), flip +operand order for descending, and break ties in key order; one side changed alone makes a +spliced chain and its tier-2 fallback sort differently, with no error. + +**A diff that changes the inline `_::less` if-chain's key cap or `less_masked`'s tuple overloads +changes both, in the same change.** A key tuple longer than the cap declines the splice and sorts +eagerly, so a cap raised without the matching overload leaves a spliced chain with no tier-2 twin +to agree with. + +**Never reserve `n` in a `top_n*` lane over an iterator - let the bounded-heap fill phase grow +geometrically.** Cardinality is unknown there, so an `n` far above the element count allocates +the whole `n` upfront for no win. + +**A linq call-macro that synthesizes the `_` block parameter without setting `can_shadow` on it +is a defect.** Chains nest, and the inner `_` has to shadow the outer one. + +**Never move the `_sql` hand-off below `normalize_order_reverse` / `collapse_chained_*` or below +`if (empty(calls))` - it runs ahead of chain normalization and ahead of the empty-chain bail.** +`_sql` re-expands the pristine expression on the next infer pass, and a `[sql_table]` source with +no chain ops at all is still SQL. + +**A pattern row that emits when its captured `select` (the `srcsel` slot) cannot be wrapped into +a projected adapter is a defect - skip the row instead.** The row's emit assumes the projected +element, so running it on the un-projected adapter orders or dedups raw rows - a wrong result, +not a missed splice. **The call order in `register_all_linq_fold_rows` IS pattern priority.** Rows land in `splice_patterns` in call order and the walker takes the first match, so reordering the @@ -83,108 +80,101 @@ rather than a missed splice. `[_macro]`s in separate macro contexts that cannot coordinate - that single registrar is the only place the order exists. -**A sql_linq column-ref arm accepts a `$e(recv).$f(field)` qmatch only when -`recv is ExprVar`.** A nested receiver (`l.opt.X`, `outer.Brand`) otherwise matches with a -foreign name and emits a silently wrong column. - -**A sql_linq computed-expression arm runs only behind `is_sql_renderable_scalar`.** That -type gate routes a whole-row carry var to the clean row-object reject instead of a -macro-time crash in `pred_to_sql`. - -**A SQL-returning helper is checked on BOTH channels - `q.hadError || empty(frag)`.** -Neither half suffices: operator arms wrap empty children into non-empty junk -(`lhs = rhs` -> `" = ?"`), and several helpers return `""` without setting `hadError`. -Dropping either half emits malformed SQL instead of a diagnostic. - -**Both `maybe_finalize_distinct_by_passthrough` and `maybe_wrap_take_before_aggregate` run -between `analyze_chain` and the `build_sql_string` / `collect_query_binds` that turn its -result into SQL.** Skipping the first silently drops a `_distinct_by` dedup; skipping the -second leaves LIMIT/OFFSET on the aggregate. - -**`q.innerSql` and `q.innerBindExprs` have exactly one producer per query.** Installing a -wrap over an already-populated `innerSql` drops the earlier subquery and orphans its binds - -a bind/placeholder mismatch, not an error. - -**Every projection slot pushes one entry into each of the five index-parallel projection -arrays - `selectCols`, `selectColAliases`, `selectColSqlFragments`, `selectColTypes`, -`projRecordNames`.** A helper covering only part of the set (`push_source_column`, -`push_computed_proj_slot`) leaves the rest to its caller; a partial push desyncs the SELECT -list from the row builder silently. - -**The sql_linq emitter picks each projection slot's SQL by which of that slot's own entries -is non-empty - SQL fragment first, then aliased column, then unqualified column - never by -`q.seenJoin`.** - -**A clause that can emit `?` pushes its binds at its SQL parse position in -`collect_query_binds`.** `sql_to_frags_ex` re-scans the emitted SQL text and pairs markers -with that list in occurrence order, so push order must equal textual order - a bind-carrying -clause added to the emitter alone binds every later placeholder to the wrong value, with no -error. - -**A linq_das clause keyword is a depth-0 whole word not preceded by `.` and not preceded by -a `>` that tails `|>`, `=>` or `->`.** Narrowing that exclusion lets an in-body aggregate -(`g |> select(...) |> sum`) parse as a `select` clause; widening it to any `>` stops a clause -keyword that legitimately follows a generic bracket or a comparison from being found at all. - -**Every substituting linq_das scanner shares one token model and changes with the -others** - -`substitute_idents`, `mentions_ident`, `rewrite_group_var`: plain `"..."` content is verbatim, -a `{...}` interpolation body is CODE (scanned and substituted), one level of nested string -literal inside an interpolation is verbatim again, and identifier POSITION classifies the -same way everywhere (after-`.` field access and before-single-`=` labels are not -references - `is_label_position` is the shared judge). A model change in one scanner -desyncs `mentions_ident` from the rewrite it gates, and the emitter then renames a -parameter the spliced projection still references. - -**`find_kw_depth0` skips a whole string literal, interpolation bodies included.** That -asymmetry against the substituting scanners is load-bearing: a clause keyword inside `"{...}"` -must never claim a stage terminal, so giving this scanner the interp-as-code model is a -defect, not a unification. - -**`parse_one_stage` resolves the stage terminal - the earliest `select`/`group` at or after -`start` - before any `where`/`orderby` position is considered.** The terminal bounds the -stage; scanning clauses first lets a later stage's `where` (past this stage's `into`) hijack -this stage's terminal lookup. - -**The `group ... into g` rewrite touches exactly two spellings: bare `g` -> `g._1`, `g.key` -> -`g._0`.** Every other `g.` is left verbatim, so raw tuple access keeps working and a -bad member errors as itself. Widening the rewrite turns `g._0` into `g._1._0`. - -**Every `FromInMacro` reject - including the not-yet-inferred-source arm - is -`macro_error` + `return null`, never `return call`.** Returning the call reports -ast-changed on every pass and churns to the infer-pass cap (30507); null lets infer -stabilize, and because errors clear per pass, the not-yet-inferred arm's error survives -only when the source never infers, which is what makes that arm a deferral. - -**decs range slots are accepted in canonical chain order: skip -> skip_while -> take_while -> -take, all after any `where_`.** - -**decs range guards emit take-cap -> skip counter -> skip_while flag -> take_while break -> -take bump, mirroring the array side's `wrap_with_ranges`.** The bump is LAST so an element -the while-guards rejected does not eat the `take(N)` budget. - -**A predicate-driven decs range requires a select-free prefix** - its predicate peels -against the SOURCE tuple, so a select ahead of it changes the element the predicate reads. - -**decs range counters and the take limit are hoisted into the prelude above the archetype -walk.** The state is global to the query, and the user's `take(N)` expression must evaluate -exactly once. - -**Dropping a decs component slot erases from all parallel `ExprFor` vectors and clears -`iteratorVariables`** - `sources`, `iterators`, `iteratorsAt`, `iteratorsAka`, -`iteratorsTags`, erased back-to-front, with `iteratorVariables` cleared so the typer -rebuilds it (the `soa.das` pattern). A vector left un-erased pairs a surviving slot with the -wrong component. - -**`long_count` never rides the decs `arch.size` shortcut.** The shortcut accumulates -`acc += int(arch.size)`, so it is int-safe only; an int64-safe total has to walk the -entities. - -**`emit_decs_reverse_skip_into_tail` takes its projection type from the typer-resolved call -type, never from the peeled lambda body.** `peel_lambda_rename_var`'s invoke fallback -carries a null `_type` at macro stage, so deriving the type from the peel mis-classifies -those chains; the caller proves the type non-null before using it. - -**`emit_loop_or_count_lane_decs` declines `to_table` before the implicit-to_array arm.** -decs has no to_table lane, and reaching the implicit arm emits an array for a table-typed -expression. +**A sql_linq column-ref arm that accepts a `$e(recv).$f(field)` qmatch without first proving +`recv is ExprVar` is a defect.** A nested receiver otherwise matches under a foreign name and +emits a wrong column, with no error. + +**A sql_linq computed-expression arm that runs without an `is_sql_renderable_scalar` gate ahead +of it is a defect.** That type gate routes a whole-row carry var to the clean row-object reject +instead of to a macro-time crash in `pred_to_sql`. + +**A call to a SQL-returning helper that checks only one of `q.hadError` and `empty(frag)` is a +defect - check both.** Operator arms wrap an empty child into non-empty junk and several helpers +return `""` without setting `hadError`, so one check alone emits malformed SQL in place of a +diagnostic. + +**A path from `analyze_chain` to `build_sql_string` / `collect_query_binds` that skips +`maybe_finalize_distinct_by_passthrough` or `maybe_wrap_take_before_aggregate` is a defect.** +Skipping the first drops a `_distinct_by` dedup with no error; skipping the second leaves +LIMIT/OFFSET on the aggregate. + +**Never install a wrap over an already-populated `q.innerSql` or `q.innerBindExprs` - each takes +exactly one producer per query.** The wrap drops the earlier subquery and orphans its binds - a +bind/placeholder mismatch, with no error. + +**A diff that adds a projection slot pushes one entry into each of the five index-parallel +projection arrays - `selectCols`, `selectColAliases`, `selectColSqlFragments`, `selectColTypes`, +`projRecordNames` - in the same change.** `push_source_column` and `push_computed_proj_slot` +cover only part of the set and leave the rest to their caller; a partial push desyncs the SELECT +list from the row builder, with no error. + +**Never pick a projection slot's SQL by `q.seenJoin` - pick it by which of that slot's own +entries is non-empty: SQL fragment first, then aliased column, then unqualified column.** + +**A diff that adds a clause that can emit `?` also pushes that clause's binds at its SQL parse +position in `collect_query_binds`, in the same change.** `sql_to_frags_ex` re-scans the emitted +SQL text and pairs markers with that list in occurrence order, so a clause added to the emitter +alone binds every later placeholder to the wrong value, with no error. + +**Never narrow or widen the linq_das clause-keyword test - a clause keyword is a depth-0 whole +word not preceded by `.` and not preceded by a `>` that tails `|>`, `=>` or `->`.** Narrowing +lets an in-body aggregate parse as a `select` clause; widening the exclusion to any `>` hides a +clause keyword that legitimately follows a generic bracket or a comparison. + +**A diff that changes the token model in one substituting linq_das scanner - `substitute_idents`, +`mentions_ident`, `rewrite_group_var` - changes it in the others, in the same change.** The +shared model: plain `"..."` content is verbatim, a `{...}` interpolation body is code that is +scanned and substituted, one level of nested string literal inside an interpolation is verbatim +again, and identifier position classifies the same way everywhere - after-`.` field access and +before-single-`=` labels are not references, with `is_label_position` the shared judge. A model +change in one scanner desyncs `mentions_ident` from the rewrite it gates, and the emitter then +renames a parameter the spliced projection still references. + +**Giving `find_kw_depth0` the substituting scanners' interp-as-code model is a defect - it skips +a whole string literal, interpolation bodies included.** A clause keyword inside `"{...}"` must +never claim a stage terminal. + +**A `parse_one_stage` that reads a `where` or `orderby` position before it resolves the stage +terminal - the earliest `select`/`group` at or after `start` - is a defect.** The terminal bounds +the stage, so scanning clauses first lets a later stage's `where`, past this stage's `into`, +hijack this stage's terminal lookup. + +**Never widen the `group ... into g` rewrite past its two spellings - bare `g` -> `g._1` and +`g.key` -> `g._0` - leave every other `g.` verbatim.** Raw tuple access then keeps working +and a bad member errors as itself; a wider rewrite turns `g._0` into `g._1._0`. + +**Never `return call` from a `FromInMacro` reject, the not-yet-inferred-source arm included - +report `macro_error` and `return null` instead.** Returning the call reports ast-changed on every +pass and churns to the infer-pass cap (30507). + +**Accepting a decs range slot out of canonical chain order - skip -> skip_while -> take_while -> +take, all after any `where_` - is a defect.** + +**Emitting the decs range guards in any order other than take-cap -> skip counter -> skip_while +flag -> take_while break -> take bump is a defect** - that order mirrors the array side's +`wrap_with_ranges`. The bump is last so an element the while-guards rejected does not eat the +`take(N)` budget. + +**A predicate-driven decs range accepted with a select in its prefix is a defect** - its +predicate peels against the source tuple, so a select ahead of it changes the element the +predicate reads. + +**Emitting a decs range counter or the take limit inside the archetype walk, rather than in the +prelude above it, is a defect.** That state is global to the query, and the user's `take(N)` +expression must evaluate exactly once. + +**A diff that drops a decs component slot erases it from every parallel `ExprFor` vector - +`sources`, `iterators`, `iteratorsAt`, `iteratorsAka`, `iteratorsTags`, back-to-front - and +clears `iteratorVariables` so the typer rebuilds it, in the same change.** A vector left +un-erased pairs a surviving slot with the wrong component. + +**Never route `long_count` through the decs `arch.size` shortcut - walk the entities instead.** +The shortcut accumulates `acc += int(arch.size)`, so it is int-safe only. + +**Never derive `emit_decs_reverse_skip_into_tail`'s projection type from the peeled lambda body - +take it from the typer-resolved call type, and prove that type non-null before using it.** +`peel_lambda_rename_var`'s invoke fallback carries a null `_type` at macro stage. + +**`emit_loop_or_count_lane_decs` that lets a `to_table` terminator reach the implicit-to_array +arm is a defect - decline `to_table` ahead of that arm.** decs has no to_table lane, so the +implicit arm emits an array for a table-typed expression. diff --git a/dastest/REVIEW.md b/dastest/REVIEW.md index 497f7f45b1..3276302a27 100644 --- a/dastest/REVIEW.md +++ b/dastest/REVIEW.md @@ -6,5 +6,5 @@ dastest ships as a prebuilt exe - the `DAS_UTILS_SHIPPED_EXES` entry (`utils/CMakeLists.txt`, repo root) records this decision; the rules for changing it live in `utils/REVIEW.md`. -**A helper added to, renamed in, or removed from `review_gate.das` updates its section in -`README.md` in the same change.** +**A diff that adds, renames, or removes a helper in `review_gate.das` also updates the +`review_gate.das` section of `README.md`, in the same change.** diff --git a/include/daScript/simulate/REVIEW.md b/include/daScript/simulate/REVIEW.md index 5bda64d12c..0e09654879 100644 --- a/include/daScript/simulate/REVIEW.md +++ b/include/daScript/simulate/REVIEW.md @@ -8,24 +8,21 @@ into generated code, a flag or field a daslib predicate reads - applies `daslib/ too; checklist discovery walks changed paths only, so the C++ half never opens the daslib checklist on its own. -- **The hot path must not become more expensive per evaluated expression - new work is a - defect, and so is degrading existing work: a direct call becoming indirect, a static - dispatch becoming virtual, or an unboxed value becoming a boxed round-trip counts the - same as an added load, branch, call, copy, or counter.** The hot path is a - `SimNode::eval*` method, any helper such a method calls on every evaluation, the - dispatchers `Context::callOrFastcall` / `callWithCopyOnReturn` / `invoke` / `invokeEx` - (`simulate.h`), or an AOT-side function or template under this folder that generated - code executes per evaluated expression. A diff that adds or amplifies such a construct - - including one an optimized build flattens to nothing - lands its entry under - `ARCHITECTURE.md`'s sanctioned hot-path additions in the same diff; the entry's required - fields are that ledger's to define. Work on this path taxes every program on every - evaluated expression. +- **A diff that makes the hot path more expensive per evaluated expression is a defect - an + added load, branch, call, copy, or counter, a direct call becoming indirect, a static + dispatch becoming virtual, and an unboxed value becoming a boxed round-trip all count.** + The hot path is a `SimNode::eval*` method, any helper such a method calls on every + evaluation, the dispatchers `Context::callOrFastcall` / `callWithCopyOnReturn` / + `invoke` / `invokeEx` (`simulate.h`), or an AOT-side function or template under this + folder that generated code executes per evaluated expression. Such a diff - including one + an optimized build flattens to nothing - lands its entry under `ARCHITECTURE.md`'s + sanctioned hot-path additions in the same diff; that ledger defines what the entry says. -- **A change to the layout of a `debug_info.h` struct - a field added, removed, reordered, - or retyped, or a base changed - carries a per-consumer verdict (updated / no change - needed / rebuild required) in the PR description: for the rtti binding +- **A diff that changes the layout of a `debug_info.h` struct - a field added, removed, + reordered, or retyped, or a base changed - states a per-consumer verdict (updated / no + change needed / rebuild required) in its own PR description: for the rtti binding (`src/builtin/module_builtin_rtti.cpp`), for the das-side readers of the struct, and for - external-module rebuilds.** Such a diff with no verdict statement is a defect. + external-module rebuilds.** - **A `debug_info.h` layout change that deletes or loosens an assertion in `tests-cpp/small/test_debug_info_layout_pin.cpp` instead of re-pinning it to the new diff --git a/modules/REVIEW_SHADER_EMITTERS.md b/modules/REVIEW_SHADER_EMITTERS.md index 80f20576c0..9a16761b3d 100644 --- a/modules/REVIEW_SHADER_EMITTERS.md +++ b/modules/REVIEW_SHADER_EMITTERS.md @@ -6,37 +6,36 @@ docs: `modules/dasMetal/MASTERPLAN.md` and `modules/dasSpirv/MASTERPLAN.md`. **These rules bind both emitters.** Each module's `REVIEW.md` routes here, and a change under that module applies this file together with the module's own list. -- **Kernel bodies are ordinary das: what cannot run on the CPU does not enter the emitter.** - Every kernel runs on the CPU, and so does every function it calls. That CPU run is the test - oracle. +- **Never put anything that cannot run on the CPU into a kernel body or into a function a + kernel calls - keep both in ordinary das.** The CPU run of the same body is the test oracle. -- **A diff that adds or changes an emitter builtin ships a CPU body for it.** The CPU body - returns what the emitted form returns, argument for argument. +- **A diff that adds or changes an emitter builtin ships a CPU body that returns what the + emitted form returns, argument for argument.** -- **Everything outside the lowered subset is a clean compile error naming the construct** - - never a silently wrong kernel. +- **Never let a construct the emitter cannot lower produce a kernel or a crash - the emitter + reports a compile error that names the construct.** -- **A shape constant reaches a kernel as a call-site constant, never as a runtime argument.** - A shape constant is any value that fixes the kernel's tiling: a tile row count, a tile column - count, a cooperating-simdgroup count, a staged chunk depth. +- **Never pass a shape constant to a kernel as a runtime argument - pass it as a call-site + constant.** A shape constant is any value that fixes the kernel's tiling: a tile row count, a + tile column count, a cooperating-simdgroup count, a staged chunk depth. -- **A matmul reduction width known only at run time is passed through the emitter's - runtime-extent descriptor.** The reduction width is the K dimension - the length of the loop - the kernel accumulates over. On Metal that descriptor is `dynamic_extent`. On SPIR-V it is a - `tensorLayout2D` or a `tensorLayout2DPad` whose dimension `tensorLayoutSetDimension` sets. K - does not fix tiling, so it is not a shape constant. +- **Never pass a matmul reduction width known only at run time into a kernel any way but + through the emitter's runtime-extent descriptor.** The reduction width is the K dimension - + the length of the loop the kernel accumulates over. That descriptor is `dynamic_extent` on + Metal, and on SPIR-V a `tensorLayout2D` or a `tensorLayout2DPad` whose dimension + `tensorLayoutSetDimension` sets. K does not fix tiling, so it is not a shape constant. -- **A new kernel capability that would need a runtime shape value ships a specialization +- **A diff that adds a kernel capability needing a runtime shape value ships a specialization path.** A specialization path is one compiled variant per constant shape. When the capability - cannot have one, the module's `MASTERPLAN.md` records that. + cannot have one, the same diff records that in the module's `MASTERPLAN.md`. -- **A claim about emitted shape is checked in the emitted words or text, never the das - source.** +- **Never check a claim about emitted shape against the das source - check it in the emitted + words or text.** -- **A kernel-model capability lands on both emitters, or the asymmetry is recorded** in - the shared ledger (`modules/dasMetal/MASTERPLAN.md`). +- **A diff that adds a kernel-model capability to one emitter adds it to the other, or records + the asymmetry in the shared ledger (`modules/dasMetal/MASTERPLAN.md`).** - **An emitter diff that uses a `daslib/shader_lingua_franca` declaration this emitter does not handle ships that emitter's lowering of the declaration, in the same change.** The diff may - instead ship the fail-closed fixture that rejects the declaration by name. A declaration in + instead ship a test showing this emitter rejects the declaration by name. A declaration in that module is available to both emitters. diff --git a/modules/dasImgui/REVIEW.md b/modules/dasImgui/REVIEW.md index 8d9c1e4a49..c3fcb79242 100644 --- a/modules/dasImgui/REVIEW.md +++ b/modules/dasImgui/REVIEW.md @@ -3,12 +3,12 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `CLAUDE.md`. -**Tests go under `modules/dasImgui/tests`.** Every dasImgui test file in the diff is under it. -A dasImgui test file, wherever the diff puts it, also answers to the `tests/` subfolder's -checklist (`modules/dasImgui/tests/REVIEW.md`). +**Never add or change a dasImgui test file outside `modules/dasImgui/tests` - put it under that +folder instead.** A dasImgui test file, wherever the diff puts it, also answers to the `tests/` +subfolder's checklist (`modules/dasImgui/tests/REVIEW.md`). -**A diff whose hunks change executable content in any `.das` file under this folder, or any -file under `bind/` or `src/`, runs the suite before the PR on the author's host OS: +**A diff that changes executable content in any `.das` file under this folder, or in any file +under `bind/` or `src/`, runs the test suite on the author's host OS before the PR: `preflight --only imgui`** (the per-OS exclude split: module `CLAUDE.md` sec. Tests). No per-PR CI -lane runs the suite (`.github/workflows/nightly_imgui.yml` is nightly-only), so the local run is +lane runs that suite (`.github/workflows/nightly_imgui.yml` is nightly-only), so the local run is the only pre-merge gate. diff --git a/modules/dasImgui/tests/REVIEW.md b/modules/dasImgui/tests/REVIEW.md index 14db828dc9..d1926bbdd1 100644 --- a/modules/dasImgui/tests/REVIEW.md +++ b/modules/dasImgui/tests/REVIEW.md @@ -5,6 +5,6 @@ parses a pinned set of constructs (`test_grammar_canary.das`) - applies `tree-sitter-daslang/REVIEW.md` (repo root) too. -**Tests are kept multiplatform** - no hardcoded platform-specific paths, path separators, or -OS-only assumptions; locations resolve via `get_das_root()`, never absolute or machine-local -paths. +**Never hardcode a platform-specific path, a path separator, or an OS-only assumption in a +test** - resolve every location through `get_das_root()`, never an absolute or machine-local +path. diff --git a/modules/dasLLAMA/REVIEW.md b/modules/dasLLAMA/REVIEW.md index 89dade5801..77356d9871 100644 --- a/modules/dasLLAMA/REVIEW.md +++ b/modules/dasLLAMA/REVIEW.md @@ -53,44 +53,41 @@ neither is `[tune_scope]` metadata (`covers=`, `tuner=`, `version_of=`). Equal v an equal kernel set and an interchangeable sidecar set - the exchange keys validity on version and box. -**A kernel's shape is compile-time; only its data is runtime.** For a given compiled kernel, -can this value change between dispatches? If yes it is data and belongs in a uniform, a kargs -field, or an `@off` bind offset; if no it is shape and must not reach the kernel as a uniform, -a kargs field, an `@off` bind offset, or a helper parameter. +**A value that cannot change between dispatches of one compiled kernel never reaches that +kernel as a uniform, a kargs field, an `@off` bind offset, or a helper parameter.** A value +that can change between dispatches goes in a uniform, a kargs field, or an `@off` bind offset. -**A diff that changes a kernel-selection predicate in `dasllama/` rests on timing that ran +**A diff that changes a kernel-selection predicate in `dasllama/` is based on timing that ran both variants interleaved in one process, under one instrument.** The same holds for a constant in `dasllama/` whose value was chosen by timing two candidates against each other. A reading taken across two processes, or across two commits, says which way the wall moved, not which implementation to adopt. -**Peak memory wins ties against load cost.** A change to an allocation reached from a load, -bake, or convert path (judge a shared helper at each call site) that trades footprint for speed -ships the measured pair - peak footprint and wall-clock - and an explicit stated decision. +**A change to an allocation reached from a load, bake, or convert path (judge a shared helper +at each call site) that trades footprint for speed ships the measured pair - peak footprint and +wall-clock - and a stated decision.** -**A new GEMM/GEMV call site takes the fastest serving lane that exists for its weights; the -f32 fallback is for correctness rails only.** A new call to an f32 matmul (`matmul_batch`, -`mm_blob_b`, per-head `gemm_f32`, or an f32 GPU mm) outside a parity or oracle rail, where a -faster-format twin already serves the same weights and shape, is a defect. Weights with no -faster twin (unquantized planes) are out of scope; a site that must stay f32 for another -reason is ledgered on its own file's charter line in `ARCHITECTURE.md` sec.1, not commented -into compliance. +**A new call to an f32 matmul (`matmul_batch`, `mm_blob_b`, per-head `gemm_f32`, or an f32 GPU +mm) outside a parity or oracle rail, where a faster-format twin already serves the same weights +and shape, is a defect - call that twin instead.** Weights with no faster twin (unquantized +planes) are out of scope; a site that must stay f32 for another reason is ledgered on its own +file's charter line in `ARCHITECTURE.md` sec.1, not commented into compliance. **Platform-specific code in an engine file (`dasllama/`) lands only in that platform's backend -file.** A platform-neutral engine file carrying it is a defect. +file.** **A new engine concern that is not `Model`/`Session`/`Config` state gets its own file, not more of `dasllama/dasllama_common.das`.** -**No ad-hoc profiling.** A NEW clock read paired with a print or log of the elapsed interval is -a defect in an engine file (`dasllama/`) outside a cold one-shot load, bake, map, or -tokenizer-build progress log - instrumentation goes through the sanctioned rails, listed with -their reasons in `ARCHITECTURE.md` sec.2.10. +**A NEW clock read paired with a print or log of the elapsed interval is a defect in an engine +file (`dasllama/`) outside a cold one-shot load, bake, map, or tokenizer-build progress log** - +instrumentation goes through the sanctioned rails, listed with their reasons in +`ARCHITECTURE.md` sec.2.10. **A clock value that changes what the program DOES - control flow, eviction, a generated name; not a reported wall or a best-of reduction over reported walls - is marked -`// clock: control`, in an engine file (`dasllama/`)** - unmarked it reads as free-hand -timing to the ad-hoc-profiling sweep, which runs over the same tree. +`// clock: control`, in an engine file (`dasllama/`)** - unmarked, it looks like ad-hoc +profiling to the sweep that runs over the same tree. **Every new kernel or loop the runtime re-enters per token, per frame, or per prefill quantum is COVERED by an annotated region entry** - `[hot_path]`, any of the `[no_alloc]` / @@ -115,15 +112,15 @@ instrument is the scaling ratio across the size ladder, and superlinear is a def template strings any of them look up, records a `tests/test_tokenizer.das` run with its cases EXECUTED, not skipped.** -**An override announces itself where it changes the outcome.** An override is an environment -knob or an exported runtime setter that moves a gate, policy, or threshold off its default -and thereby changes what the run writes, reads, mints, or computes - including a knob or -setter whose purpose is timing when it moves computed numerics, since two GEMM forms of -the same math differ in float terms. A knob or setter that changes only WHEN work happens -is not an override, and a CLI flag is never one. A run that engages one prints a line naming -the override by the spelling a user would set - the environment variable name, the sidecar -key, or the setter's function name. Per-site repeats are fine. A set-but-inert override -stays silent. Adding one, or giving one a new effect, without the announce is a defect. +**A diff that adds an override, or gives one a new effect, without the announce is a defect.** +An override is an environment knob or an exported runtime setter that moves a gate, policy, or +threshold off its default and thereby changes what the run writes, reads, mints, or computes - +including a knob or setter whose purpose is timing when it moves computed numerics, since two +GEMM forms of the same math differ in float terms. A knob or setter that changes only WHEN work +happens is not an override, and a CLI flag is never one. The announce is a line the run prints +where the override changes the outcome, naming it by the spelling a user would set - the +environment variable name, the sidecar key, or the setter's function name. Per-site repeats are +fine. A set-but-inert override stays silent. **A self-measured served-turn time entering `PERF_LEDGER.md` comes from the released `lcpp_bench` exe, never from the `-jit` script.** A served-turn time is a tok/s figure or a @@ -165,59 +162,56 @@ produced it.** A stage figure is a stage wall, a stage share, a stage speedup, o cross-engine comparison of one stage. The rule holds wherever the figure is written down: a checked-in doc, a ledger, a code comment, or a PR description. A board cell's `pp`/`tg` rate and the whole wall of a `benchmarks/lcpp_bench.das` `-p`/`-n` cell measure the turn, not a -stage, and are not stage figures. The naming rides the figure's own sentence, a table heading -that covers the table's rows, or a section-level provenance line that covers the paragraphs -under it. +stage, and are not stage figures. The naming sits in the figure's own sentence, in a table +heading that covers the table's rows, or in a section-level provenance line that covers the +paragraphs under it. -**Runtime serves weights out of a mapped `.dlim`.** A live carrier's planes point into -`parse_image`'s mapping, and going live does no real work - repacking, quantizing, folding, -permuting belong to the mint. A transform on the go-live path is a defect. +**A transform on the go-live path - repacking, quantizing, folding, permuting - is a defect; +it belongs to the mint.** Going live is `parse_image` pointing a live carrier's planes into the +mapped `.dlim`. **A missing `.dlim` is minted first, and the model is served from what was minted.** -**There is one way to mint, and one way to load.** A weight carrier becomes live through -`build_image` and `parse_image` in `dasllama/dasllama_image.das`. Reading weights into a live carrier, -or releasing an image backing, anywhere else is a defect - and a second mint path, per family, -per format, or per backend, is a defect even where its output is identical. +**A weight carrier becomes live only through `build_image` and `parse_image` in +`dasllama/dasllama_image.das`: reading weights into a live carrier, or releasing an image +backing, anywhere else is a defect - and a second mint path, per family, per format, or per +backend, is a defect even where its output is identical.** **A decoder mint never holds the whole model.** A decoder mint is the mint of an LLM decoder -model, not of a tower or embedder carrier. It sizes the image before the first byte goes out -and writes each plane as it is produced. Keeping the source model resident to write from is a -defect. A mint that is slower in exchange for a lower peak is correct. +model, not of a tower or embedder carrier. It sizes the image before writing the first byte and +writes each plane as it is produced. A mint that is slower in exchange for a lower peak is +correct. **A staged carrier mint (`cache_via_image_staged`) meeting a source file at or past 1 GiB either refuses it or streams it the way a decoder mint does.** A refusal names that file. The -staged form holds source and image at once, and that doubled peak is what the line caps. +staged form holds source and image at once, and this rule caps that doubled peak. -**A `.dlim` is box- and config-specific, not a portable format.** `image_identity` names the -box profile, the knobs, and the flavor a file was baked for, and a mismatch declines loudly. A -path that reinterprets a mismatched image, or widens an identity so that more files match, is a -defect. +**A path that reinterprets a mismatched image, or widens an identity so that more files match, +is a defect.** `image_identity` names the box profile, the knobs, and the flavor a file was +baked for, and a mismatch declines loudly. -**An image save reaps only its own lane.** A lane is an identity's (quant, tag) pair. A save -drops AT MOST that lane's dead siblings plus BROKEN/version-stale images in any lane, nothing -else. +**An image save drops AT MOST its own lane's dead siblings plus BROKEN/version-stale images in +any lane.** A lane is an identity's (quant, tag) pair. -**Only a process that can recompute an image's identity may judge it dead.** Reaping an image -whose identity the code cannot recompute - another flavor's, another family's - is a defect. +**Reaping an image whose identity the code cannot recompute - another flavor's, another +family's - is a defect.** **A plane split that follows the source FILE rather than a runtime knob takes ONE image tag**, with the meta flags describing the layout - a per-tensor type split is not a second flavor. -**An image carries only what its flavor uses.** A plane the target platform or config never -reads is not written - the mint decides, not the load. Carrying another flavor's planes is a -defect. +**A plane the target platform or config never reads is not written into the image - the mint +decides that, not the load.** **A flavor takes its image file through `image_path_for` and its tag through -`register_image_family_tag`.** A path or tag formed any other way is a defect. +`register_image_family_tag`.** -**A change to user-facing API updates every place it is shown.** User-facing means anything a -consumer outside this repo can depend on - what it calls, types, requires, or parses -(facade functions, CLI flags, environment knobs, file formats, defaults, what the installed -SDK lets a program `require`) - plus the in-repo rig and tool surface: any output another tool -parses, a console-only diagnostic not being one. A tutorial source, `.rst` page, docstring, -help string, `README.md`, or checked-in document still showing the old call, flag, or default -is a defect of the change, not of the docs. +**A change to user-facing API updates every place it is shown: a tutorial source, `.rst` page, +docstring, help string, `README.md`, or checked-in document still showing the old call, flag, or +default is a defect of the change, not of the docs.** User-facing means anything a consumer +outside this repo can depend on - what it calls, types, requires, or parses (facade functions, +CLI flags, environment knobs, file formats, defaults, what the installed SDK lets a program +`require`) - plus the in-repo rig and tool surface: any output another tool parses, a +console-only diagnostic not being one. **Weakening `dasllama_lint` (`dasllama/dasllama_lint.das`) - the compile-time check that a consumer requires only this module's public entry modules, matched by the resolved file's @@ -227,11 +221,10 @@ instead. The allowed set is the table in the lint. **`options _dasllama_internal` belongs only in a file whose job is to reach engine internals: an engine file under `dasllama/`, a test, harness, benchmark, or rig this module -owns, or a consumer `ARCHITECTURE.md` sec.1 names as ruled.** Any other file carrying it is a -defect - a symbol the facade lacks is added to `dasllama/dasllama.das`, not escaped around. -A `require ... public` that re-exports an engine module OUT of an escaped file, beyond what -that consumer's ruled charter (`ARCHITECTURE.md` sec.1) grants, is the same defect wearing a -different line. +owns, or a consumer `ARCHITECTURE.md` sec.1 names as ruled** - a symbol the facade lacks is +added to `dasllama/dasllama.das`, not escaped around. A `require ... public` that re-exports an +engine module OUT of an escaped file, beyond what that consumer's ruled charter +(`ARCHITECTURE.md` sec.1) grants, breaks this rule too. **Weakening `REVIEW.das` (beside this file) is a defect:** dropping a check, adding a name to a check's licensed set - the names that check does not flag - or a finding text that no @@ -263,8 +256,8 @@ struct the renderer never emits is absent from `ENVIRONMENT.md` and invisible to a struct the renderer emits but the registry does not is caught by `tests/test_env_registry.das`. -**`dasllama/dasllama_unicode.das`'s RANGES/WS tables are generated - retranscoded from -`$LCPP/src/unicode-data.cpp` (the reference checkout); hand-editing them is a defect.** +**Hand-editing `dasllama/dasllama_unicode.das`'s RANGES/WS tables is a defect - regenerate them +by retranscoding `$LCPP/src/unicode-data.cpp` (the reference checkout) instead.** **A diff that adds a file under `dasllama/`, moves code between files, or changes what a file owns lands the sec.1 edit that keeps the charters true, in the same change.** A diff that adds @@ -304,8 +297,7 @@ kind - `create_chat_` panics at create, not at render. **No signature in `dasllama/dasllama_tower.das` takes a type that `dasllama/dasllama_audio.das`, `dasllama/dasllama_vision.das`, or a family file declares.** `dasllama/dasllama_tower.das` is the shared encoder-tower home, and it requires none of those -files. A doc comment naming the family a helper was built for is fine. The code stays -family-blind. +files. A doc comment naming the family a helper was built for is fine. **A `dasllama/dasllama_tower.das` helper with one calling family lands in that family's file** - a single-caller helper sanctioned as tower-worthy is ledgered on `ARCHITECTURE.md` @@ -324,9 +316,8 @@ own image identity.** A persisted form is one an image could carry. The load tha lane prints which lane it picked. A conversion made and dropped inside one forward pass persists nothing and is not such a lane. -**A harness that prints output for another tool to compare fails loudly when it has nothing to -print.** A run that ends without its comparison lines - wrong flags, failed load - exits -non-zero. +**A harness that prints output for another tool to compare exits non-zero when its run ends +without those comparison lines - wrong flags, failed load.** **Tool wire text - building or parsing - is produced only in `dasllama/dasllama_tools.das`.** @@ -345,5 +336,5 @@ the engine back; it sits in `dasllama/dasllama_common.das` only if engine code n program root (test, harness, benchmark, tool) requires the registration module it needs directly. -**An architecture file (`dasllama/dasllama_arch_*.das`) is declarative registration only.** An -architecture that changes a forward loop, or tests a family name on a shared path, is a defect. +**An architecture file (`dasllama/dasllama_arch_*.das`) that changes a forward loop, or tests a +family name on a shared path, is a defect - it carries declarative registration only.** diff --git a/modules/dasLLAMA/REVIEW_AUDIO.md b/modules/dasLLAMA/REVIEW_AUDIO.md index 5be9613842..bcce5111d8 100644 --- a/modules/dasLLAMA/REVIEW_AUDIO.md +++ b/modules/dasLLAMA/REVIEW_AUDIO.md @@ -5,31 +5,31 @@ shared encoder-tower home serves both), `dasllama/dasllama_audio.das`, `dasllama/dasllama_audio_io.das`, `dasllama/dasllama_audio_embedder.das`, `dasllama/dasllama_vad.das`, or an ASR family file - one `dasllama/dasllama_.das` -holding a single speech model family - applies this list with the master's.** +holding a single speech model family - applies this list with `REVIEW.md`'s.** `REVIEW_COMMON.md` (repo root) binds this file too. Architecture doc: `ARCHITECTURE.md`. -**A verb arm in `dasllama/dasllama_asr.das` or `dasllama/dasllama_audio_embedder.das` - the -union carriers that fan a facade verb across family arms - is one forwarding call.** The -loader's format or family sniff that picks the arm is the dispatch, not an arm. A new family -touches the carrier only at the union field, the finalize line, the kind value, and the -one-line arms; a prompt, a decode loop, a caps value, or a language rule in the carrier is a -defect. +**In `dasllama/dasllama_asr.das` and `dasllama/dasllama_audio_embedder.das` - the union +carriers that route each facade function to one family - a family arm that does anything but +one forwarding call is a defect; the work goes in the family file.** The loader's format or +family check that picks the arm is the dispatch, not an arm. A diff that adds a family changes +the carrier only at the union field, the finalize line, the kind value, and the one-line arms; +a prompt, a decode loop, a caps value, or a language rule in the carrier is a defect. -**A GEMM in an ASR family file goes through a `*_mm` wrapper or `mm_blob_b`.** A hand-written -dot-product loop beside them is a defect. +**A GEMM in an ASR family file that does not go through a `*_mm` wrapper or `mm_blob_b` is a +defect, hand-written dot-product loops included.** -**A per-encode reused buffer in `dasllama/dasllama_tower.das`, `dasllama/dasllama_audio.das`, -`dasllama/dasllama_audio_embedder.das`, or an ASR family file is `@scratch` - on its -declaration, or on the callee parameter it grows through.** A nolint where the annotation fits -is a defect. +**A buffer reused across encodes in `dasllama/dasllama_tower.das`, +`dasllama/dasllama_audio.das`, `dasllama/dasllama_audio_embedder.das`, or an ASR family file +that is not `@scratch` - on its declaration, or on the callee parameter it grows through - is a +defect.** A `nolint` where the annotation fits is a defect. -**A debug or profiling leg in `dasllama/dasllama_tower.das`, `dasllama/dasllama_audio.das`, -`dasllama/dasllama_audio_embedder.das`, or an ASR family file is `[cold_path]`.** A nolint -where it fits is a defect. +**A debug or profiling code path in `dasllama/dasllama_tower.das`, +`dasllama/dasllama_audio.das`, `dasllama/dasllama_audio_embedder.das`, or an ASR family file +that is not `[cold_path]` is a defect.** A `nolint` where `[cold_path]` fits is a defect. -**A mel frontend's transform table comes from `dasllama/dasllama_audio.das` -(`build_dft_twiddles`, or `build_fft_plan` + `fft_pow2_run`); a transform implemented inside an -ASR family file is a defect.** +**Never implement a transform inside an ASR family file - take the mel frontend's transform +table from `dasllama/dasllama_audio.das` (`build_dft_twiddles`, or `build_fft_plan` + +`fft_pow2_run`) instead.** -**A `create_session` / `transcribe` option that the model's `caps()` does not declare panics at -the call site.** Accepting it and silently ignoring it is a defect. +**Never accept a `create_session` or `transcribe` option that the model's `caps()` does not +declare - panic at the call site instead.** diff --git a/modules/dasLLAMA/REVIEW_GPU.md b/modules/dasLLAMA/REVIEW_GPU.md index d99bdb181e..922fc77743 100644 --- a/modules/dasLLAMA/REVIEW_GPU.md +++ b/modules/dasLLAMA/REVIEW_GPU.md @@ -4,45 +4,43 @@ doc: `ARCHITECTURE.md`. **Routed from `REVIEW.md`: a diff touching a GPU kernel, driver, dispatch class, or the K/V -mirrors applies this list with the master's.** +mirrors applies this list together with `REVIEW.md`.** -**The EMITTED shader contains no indirection.** No function pointers, no vtables. A -`class template` / `def abstract` / `def override` splice is compile-time and conforms - -check the emission, not the das spelling. +**A kernel body that emits a function pointer or a vtable into the shader is a defect - splice +the choice at compile time instead.** A `class template` / `def abstract` / `def override` +splice is compile-time and conforms - check the emission, not the das spelling. -**The `*_decline_caps` predicates take only the model and the call shape; window-setup state -is asked by `prefill_decline` / `decode_decline`, never by a caps predicate.** A caps +**Never give a `*_decline_caps` predicate a parameter beyond the model and the call shape - +window-setup state is asked by `prefill_decline` / `decode_decline` instead.** A caps parameter that reports the session's setup progress - rather than the CALL, its row count or its span shape - is a defect however it is derived. -**A bounds guard or tail guard inside a kernel's main loop is stamped, not branched, when the -host already knows its answer as it picks the pipeline.** Stamped means the guard rides a -`@template_constant` - a `static_if` block, or a value select on the constant. The guard-free -instance is the one stamped without the guard, and the guard is absent from that instance's -generated `*_msl` global. A diff that edits an existing kernel's loop answers to this exactly -as a new kernel class does. A per-iteration guard the host could have compiled out is a -defect. - -**A `matmul2d` left or right operand reaches the op as `float` only in a kernel class stamped -`[metal_kernel(float_a_ok=true)]`.** Everywhere else it is converted in the pass that writes -the operand's buffer, or in the staging loop that reads it. The stamped set is ledgered in +**A bounds or tail guard that branches per iteration in a kernel's main loop, where the host +already knows its answer as it picks the pipeline, is a defect - stamp the guard instead.** +Stamped means the guard rides a `@template_constant` - a `static_if` block, or a value select +on the constant. The guard-free instance is the one stamped without the guard, and the guard +is absent from that instance's generated `*_msl` global. A diff that edits an existing +kernel's loop answers to this exactly as a new kernel class does. + +**Never let a `matmul2d` left or right operand reach the op as `float` outside a kernel class +stamped `[metal_kernel(float_a_ok=true)]` - convert it in the pass that writes the operand's +buffer, or in the staging loop that reads it.** The stamped set is ledgered in `ARCHITECTURE.md` sec.2.2b, and a stamp that lands without its ledger line is a defect. A float operand keeps the op off its native fast path. -**A `matmul2d` operand is threadgroup-staged only when the staged form differs from the -stored form.** A dequant, a transpose, or a layout or element-type change makes the forms -differ. An operand consumed in its stored form streams from device instead. A staged -pass-through costs the op more than the reads it saves. +**Never threadgroup-stage a `matmul2d` operand whose staged form matches its stored form - +stream it from device instead.** A dequant, a transpose, or a layout or element-type change +makes the forms differ. A staged pass-through costs the op more than the reads it saves. -**A loop that fills a `@workgroup` tile assigns each work item a consecutive run of elements, -not a stride.** A device-to-device copy loop fills no tile and is already coalesced. -Per-element strided staging with div/mod addressing pays multiples of what the contiguous -form costs. +**Never fill a `@workgroup` tile with a loop that gives each work item a stride - give each +work item a consecutive run of elements instead.** A device-to-device copy loop fills no tile +and is already coalesced. Per-element strided staging with div/mod addressing pays multiples +of what the contiguous form costs. -**A kernel decides its row's validity or owner by reading one per-row entry, never by -scanning the per-bucket base and count arrays.** The bucket-building kernel stamps that -per-row entry. The scan repeats on every thread of every row's threadgroup, and it grows with -the bucket count. +**Never decide a kernel row's validity or owner by scanning the per-bucket base and count +arrays - read the one per-row entry instead.** The bucket-building kernel stamps that per-row +entry. The scan repeats on every thread of every row's threadgroup, and it grows with the +bucket count. **An encoder that picks a kernel's guard-free instance shows that every address the instance touches stays inside rows holding real data.** The guard-free instance is the one stamped @@ -50,93 +48,92 @@ without the loop's bounds or tail guard. One extent dividing evenly is not that padded chunk's walk can run past the live extent, and one poisoned read in a shared tile corrupts real rows. -**A pipeline of dispatches that shares one scratch buffer carries at least as many scratch -buffers as it has dispatches in flight.** Giving each dispatch site its own buffer meets this -too. One shared scratch serializes the whole chain through its write-after-read hazards. +**Never leave a pipeline of dispatches with fewer scratch buffers than it has dispatches in +flight - add buffers until it has at least that many, or give each dispatch site its own.** +One shared scratch serializes the whole chain through its write-after-read hazards. -**An encoder path that adds dispatches to save bandwidth gates on work size.** The gate's -threshold is measured at both ends of the size ladder. The small-work regression hides behind -the big-work win. +**A diff that adds dispatches to an encoder path to save bandwidth also gates that path on +work size, in the same change.** The gate's threshold is measured at both ends of the size +ladder. The small-work regression hides behind the big-work win. **A shape claim is settled at the one site that is authoritative for that kind of constant, never by tracing the das that computes the value.** An in-body tile constant is confirmed -literal in the generated `*_msl` global or the SPIR-V dump; a grid or threadgroup constant is +literal in the generated `*_msl` global or the SPIR-V dump. A grid or threadgroup constant is read off the class's `[metal_dispatch]` / `[vk_dispatch]` `grid=`/`tg=` spec, whose `"n/c"` -form is a CEIL-divide - the spec alone decides, no builder read needed; a uniform's value is -read at the single writer that fills its buffer, and nowhere upstream of it. +form is a CEIL-divide; the spec alone decides, and no builder read is needed. A uniform's +value is read at the single writer that fills its buffer, and nowhere upstream of it. -**Kernel twins - kernel classes whose bodies differ on one stamp axis - bind the same kargs -(kernel-argument struct) type at the same binding numbers**, even where one twin ignores a -field; shifting the other twin's fields to different slots is a defect. +**A kernel twin that binds a different kargs (kernel-argument struct) type than its sibling +twin, or shifts a shared field to a different binding number, is a defect - even where one +twin ignores that field.** Kernel twins are kernel classes whose bodies differ on one stamp +axis. -**Kernel twins stamp one `class template`, whatever the stamp axis is (single/batch, format, +**A copy-pasted kernel twin, a kernel split into hand instances where a `static_if` on a +`@template_constant` serves, and a dummy-bound field where a gate serves are defects - kernel +twins stamp one `class template`, whatever the stamp axis is (single/batch, format, single-pass/chunked).** Body divergence rides the stamp axis - a `@template_constant`, or an overridden method spliced flat at emission. A stamp-varying binding rides `@template_gate`. -Three shapes are defects: a copy-pasted twin, a kernel split into hand instances where a -`static_if` on a `@template_constant` serves, and a dummy-bound field where a gate serves. **A diff that forks a kernel class out of a shared template shows that the bodies no longer differ on a single stamp axis.** The same diff names the axis that is gone, in the surviving template's comment. -**A `[metal_dispatch]` / `[vk_dispatch]` field carries `@role = "weight"` exactly when its -memory is load-once - a model plane, or an `upload_region` upload never written after -arming.** A load-once field with no `@role` is a defect even when the kernel compiles and -passes parity; `weight` on per-encode data - a pooled buffer the host refills each encode - -is one too, it drops the hazard staging. A per-encode field either omits `@role` or names the -access its body performs. - -**A kernel declares its dispatch on the class; the builder is generated.** A new kernel class -carries `[metal_dispatch]` / `[vk_dispatch]` with every annotation the generated builder -reads - per-field `@binding` / `@role` / `@off` / `@span` / `@default`, `@workgroup` state -with its `tgmem=` dispatch key. A NEW `enc_*` body is hand-written only as a wrapper - a -format or twin pick, a default-filling wrapper, or a composite over generated builders; any -other hand-written `enc_*` body is a defect. - -**A kernel is dispatched only through its `enc_*` builder** - a hand-rolled bind list in -`dasllama/` or `performance/` is a defect. - -**No value reaches an encoder twice DEVICE-side.** A scalar the kernel receives both as a -uniform buffer and as a kargs field is a defect. A `params=` value that the `grid=`/`tg=` -spec consumes host-side never reaches the device, so it does not count. - -**A scalar that the other bound scalars already determine is derived in the builder, not -bound.** Binding it separately adds a second place to get it wrong. - -**A cache keyed by a host address carries the span and the form in its key.** A hit must cover -the request, and different upload forms live in separate tables. - -**A backend-only capability goes in that backend's file for the matching role.** The -`dasllama_metal_*` and `dasllama_vulkan_*` files partition into these roles: the kernel home -(`_kernels` on Metal, `_classes` on Vulkan), `_common` (device state and plumbing), -`_decode`, `_prefill`, `_gemm` (the Metal batch-GEMM donor backend), `_shapes` (portable -servability gates), `_tower` (the encoder-tower driver), `_asr_dec` (the ASR-decoder driver), -`_seams` (the Vulkan single-op resident-driver seams), and the kernel-access lens (`_lens` on -Metal, `_dispatch` on Vulkan). Vulkan's backend entry is -`dasllama/dasllama_math_vulkan.das`. That file carries the capability probe, the arm, the -`.dlim` identity source, and the `[init]` that installs every hook. A backend carries a -role's file only once it has the capability. A capability with no matching role gets its own -role file. Anything else is a grab-bag, and a grab-bag file is a defect. - -**A GPU family shares ONE device and queue from `dasllama/dasllama__common.das`'s -init.** A module creating its own is a defect. - -**A Metal PSO serving the engine is compiled and released by the file that owns its kernel -class**, through its init/release pair - `metal_decode_init` / `metal_kernels_release` in -`dasllama/dasllama_metal_kernels.das`, `metal_prefill_init` / `metal_prefill_shutdown` in -`dasllama/dasllama_metal_prefill.das`. A pipeline that a per-kernel unit test under `tests/` -builds for its own run is that test's to compile and release. - -**Race code lives in the file that owns the kernel family.** Race code is the in-engine -base-vs-twin check that times both kernels on one queue and compares their outputs. The -shared scaffolding - `race_buf`, `race_envelope_ok`, `race_pair_ms` - belongs to -`dasllama/dasllama__common.das`. - -**A Metal decline reason is an enum value in `dasllama/dasllama_metal_shapes.das`, one enum -per driver.** A string-typed metal decline is a defect. - -**Decline counting lives in `dasllama/dasllama_metal_common.das`.** A counter beside the -decline site is a defect. +**A `[metal_dispatch]` / `[vk_dispatch]` field whose memory is load-once - a model plane, or +an `upload_region` upload never written after arming - is a defect unless it carries +`@role = "weight"`, even when the kernel compiles and passes parity.** `weight` on per-encode +data - a pooled buffer the host refills each encode - is a defect too: it drops the hazard +staging. A per-encode field either omits `@role` or names the access its body performs. + +**A new kernel class carries `[metal_dispatch]` / `[vk_dispatch]` with every annotation the +generated builder reads - per-field `@binding` / `@role` / `@off` / `@span` / `@default`, +`@workgroup` state with its `tgmem=` dispatch key.** A NEW hand-written `enc_*` body is a +defect unless it is a wrapper - a format or twin pick, a default-filling wrapper, or a +composite over generated builders. + +**A hand-rolled bind list that dispatches a kernel in `dasllama/` or `performance/` is a +defect - dispatch through the kernel's `enc_*` builder instead.** + +**A value that reaches the kernel twice device-side - a scalar bound both as a uniform buffer +and as a kargs field - is a defect.** A `params=` value that the `grid=`/`tg=` spec consumes +host-side never reaches the device, so it does not count. + +**Never bind a scalar that the other bound scalars already determine - derive it in the +builder instead.** Binding it separately adds a second place to get it wrong. + +**Never key a cache on a host address alone - carry the span and the form in the key too.** A +hit must cover the request, and different upload forms live in separate tables. + +**Never put a backend-only capability outside its backend's file for the matching role - a +capability with no matching role gets its own role file.** The `dasllama_metal_*` and +`dasllama_vulkan_*` files partition into these roles: the kernel home (`_kernels` on Metal, +`_classes` on Vulkan), `_common` (device state and plumbing), `_decode`, `_prefill`, `_gemm` +(the Metal batch-GEMM donor backend), `_shapes` (portable servability gates), `_tower` (the +encoder-tower driver), `_asr_dec` (the ASR-decoder driver), `_seams` (the Vulkan single-op +resident-driver seams), and the kernel-access lens (`_lens` on Metal, `_dispatch` on Vulkan). +Vulkan's backend entry is `dasllama/dasllama_math_vulkan.das`, which carries the capability +probe, the arm, the `.dlim` identity source, and the `[init]` that installs every hook. Never +add a role's file to a backend that does not have the capability. + +**A module that creates its own GPU device or queue is a defect - a GPU family shares the one +device and queue from `dasllama/dasllama__common.das`'s init.** + +**Never compile or release a Metal PSO serving the engine outside the file that owns its +kernel class** - it goes through that file's init/release pair: `metal_decode_init` / +`metal_kernels_release` in `dasllama/dasllama_metal_kernels.das`, `metal_prefill_init` / +`metal_prefill_shutdown` in `dasllama/dasllama_metal_prefill.das`. A pipeline that a +per-kernel unit test under `tests/` builds for its own run is that test's to compile and +release. + +**Never put race code outside the file that owns the kernel family - the shared scaffolding +(`race_buf`, `race_envelope_ok`, `race_pair_ms`) belongs to +`dasllama/dasllama__common.das`.** Race code is the in-engine base-vs-twin check that +times both kernels on one queue and compares their outputs. + +**A string-typed Metal decline reason is a defect - a Metal decline reason is an enum value in +`dasllama/dasllama_metal_shapes.das`, one enum per driver.** + +**A decline counter beside the decline site is a defect - decline counting lives in +`dasllama/dasllama_metal_common.das`.** **A diff that adds or removes a Metal-only or Vulkan-only hook, role, served path, or backend-only capability - anything that changes what one backend can serve and the other @@ -145,22 +142,24 @@ sec.1.5 already carries that class of asymmetry, and including sec.1.5's per-dri registered hooks and borrowed kernels.** sec.1.5 is the closed list; an asymmetry it does not carry does not exist. -**A Vulkan pipeline is created only by a `[vk_dispatch]`-generated `ensure_*` and torn down by -`vk_drop_model_state`.** A hand-written pipeline build anywhere else in the engine is a defect. +**A hand-written Vulkan pipeline build anywhere in the engine is a defect - a Vulkan pipeline +is created only by a `[vk_dispatch]`-generated `ensure_*` and torn down by +`vk_drop_model_state`.** -**A buffer bound as one SSBO range stays under `vk_max_storage_range()`, checked where its size -is NEGOTIATED, not where it binds.** The bind site cannot shrink a buffer that was sized wrong. +**Never size a buffer bound as one SSBO range above `vk_max_storage_range()` - check the size +where it is NEGOTIATED, not where it binds.** The bind site cannot shrink a buffer that was +sized wrong. -**A change to CODE a served GPU decode or prefill path executes - a driver +**A change to code that a served GPU decode or prefill path executes ships GPU-vs-CPU parity +on one q8 and one kq model with the armed mirror codec.** That code is a driver (`dasllama/dasllama_metal_decode.das`, `dasllama/dasllama_metal_prefill.das`, `dasllama/dasllama_vulkan_decode.das`, `dasllama/dasllama_vulkan_prefill.das`), a kernel class one of them dispatches, that class's builder, the servability gates (`dasllama/dasllama_metal_shapes.das`), the weight-region cache and residency paths (`dasllama/dasllama_metal_common.das`), or the residency rail's serving paths -(`dasllama/dasllama_gpu_resident.das`); never the bake paths, never a comment - ships -GPU-vs-CPU parity on one q8 and one kq model with the armed mirror codec: `harness/parity.das` -runs, or the in-suite instruments (`tests/test_metal_decode_parity.das` / -`tests/test_metal_prefill_parity.das` through `tests/run.das`).** +(`dasllama/dasllama_gpu_resident.das`); never the bake paths, never a comment. The parity run +is `harness/parity.das`, or the in-suite instruments `tests/test_metal_decode_parity.das` / +`tests/test_metal_prefill_parity.das` through `tests/run.das`. **A `harness/parity.das` run arms its backend: the Metal arm is `--ngl`; the Vulkan arm is `DASLLAMA_GPU=1`, never `--ngl`, and its log shows `resident driver armed`.** The Vulkan @@ -170,12 +169,12 @@ driver declines codec-mismatched sessions silently. kernel class the tower dispatches or builder the tower borrows, or to state the whole driver shares (a module-level `g_tw_*` variable, `metal_tower_init`, `dasllama_metal_tower_register` - reachable from every hook) runs the gate of every -registered tower hook the changed code is reachable from.** The gates: the family gates +registered tower hook the changed code is reachable from.** The gates are the family gates `tests/test_gemma4uv.das`, `tests/test_gemma4v.das`, `tests/test_gemma3v.das`, and `test_qwen3v_tier1_metal` in `tests/test_qwen3v.das`; the encoder-blocks leg's `tests/test_whisper.das`; the conv legs' `tests/test_audio.das` and `tests/test_audio_embedder.das`; plus a `tests/test_model_image.das` run with the `mtower` -arm, `metal_tower_stats()`'s encode count rising across the run. A hook registered in +arm, with `metal_tower_stats()`'s encode count rising across the run. A hook registered in `dasllama_metal_tower_register` that this rule's gates do not cover is the rule's defect to fix in the same change. @@ -189,22 +188,24 @@ a `tests/test_model_image.das` run with the `mtower` arm** - its CPU-vs-GPU tran are the ASR-decoder driver's parity instrument, and the shared common paths and borrowed kernels reach that driver with no line of its own file touched. -**A kernel that reads or writes the residency rail's `k_mirror`/`v_mirror` slabs serves BOTH -K/V codecs.** A K/V codec is the mirror's element type: f16 or f32. Two shapes meet this: -instances of one template cover both codecs, or a single-codec kernel has a sibling that -serves the other codec behind an arming gate that keys on `kv16`. The rail serves both -codecs, so a codec no kernel covers silently drops that codec's GPU path. +**Never leave a K/V codec unserved by the kernels that read or write the residency rail's +`k_mirror`/`v_mirror` slabs - a K/V codec is the mirror's element type, f16 or f32.** Two +shapes serve both: instances of one template cover both codecs, or a single-codec kernel has +a sibling that serves the other codec behind an arming gate that keys on `kv16`. The rail +serves both codecs, so a codec no kernel covers silently drops that codec's GPU path. -**An f16 store into any GPU-resident K/V clamps to the f16 finite range (+/-65504).** +**An f16 store into any GPU-resident K/V that does not clamp to the f16 finite range +(+/-65504) is a defect.** -**Every resident override - a decode/prefill hook the whole-model residency rail registers -in `dasllama/dasllama_common.das`'s override registries - gates sessions on the armed mirror -codec and on the flat (non-paged) cache before touching the mirror.** Mirror bytes move only -between same-codec session rows and mirror rows; an override that byte-copies across codecs -corrupts the host authority. +**A resident override that touches the mirror before gating the session on the armed mirror +codec and on the flat (non-paged) cache is a defect** - a resident override is a +decode/prefill hook the whole-model residency rail registers in +`dasllama/dasllama_common.das`'s override registries. An override that byte-copies mirror +bytes across codecs is a defect too: bytes move only between same-codec session rows and +mirror rows, and a cross-codec copy corrupts the host authority. -**A descriptor set cached across dispatches lives in state `vk_drop_model_state` clears** - a -`*_ready` latch, or a field inside `g_gpu` or the weight arena in +**Never cache a descriptor set across dispatches in state `vk_drop_model_state` does not +clear** - put it in a `*_ready` latch, or in a field inside `g_gpu` or the weight arena in `dasllama/dasllama_vulkan_common.das`. **A diff that changes anything a hand-binding arm must mirror to dispatch a kernel - binding diff --git a/modules/dasLLAMA/REVIEW_VISION.md b/modules/dasLLAMA/REVIEW_VISION.md index d5b7df817d..2d1d803757 100644 --- a/modules/dasLLAMA/REVIEW_VISION.md +++ b/modules/dasLLAMA/REVIEW_VISION.md @@ -10,45 +10,48 @@ single vision projector family - and any path that runs inside the program under a spawned child process, and splices or schedules a stream carrying decoded media - pixels or audio samples; routed files apply this list with `REVIEW.md`'s.** -**A GEMM in a vision family file goes through a shared batch-GEMM entry point - `mm_blob_b`, -`mm_bf16_b`, `mm_plane_b` (`dasllama/dasllama_tower.das`) or `matmul_q8q8_batch` -(`dasllama/dasllama_math.das`).** A hand-written dot-product loop beside them is a defect. - -**A per-encode buffer in `dasllama/dasllama_vision_embedder.das` or a vision family file -carries `@exact_size` when its size follows the input - patch count, pixel count, clip -frames - and `@scratch` when it is reused across encodes rather than freshly allocated; a -buffer that is both carries both.** The annotation goes on the declaration, or on the callee -parameter it grows through; a buffer grown ONLY through such a callee - one whose parameter -carries `@scratch` and that reserves before it resizes - carries none of its own. A nolint -where an annotation fits is a defect. +**A GEMM in a vision family file that does not go through a shared batch-GEMM entry point is a +defect, a hand-written dot-product loop included** - the entry points are `mm_blob_b`, +`mm_bf16_b`, `mm_plane_b` (`dasllama/dasllama_tower.das`) and `matmul_q8q8_batch` +(`dasllama/dasllama_math.das`). + +**A per-encode buffer in `dasllama/dasllama_vision_embedder.das` or a vision family file whose +size follows the input - patch count, pixel count, clip frames - is a defect without +`@exact_size`, and one reused across encodes rather than freshly allocated is a defect without +`@scratch` - a nolint is not a substitute for either.** The annotation goes on the declaration, +or on the callee parameter the buffer grows through; a buffer grown only through such a +callee - one whose parameter carries `@scratch` and that reserves before it resizes - carries +none of its own. **A debug or profiling leg in `dasllama/dasllama_vision_embedder.das` or a vision family file -is `[cold_path]`.** A nolint where it fits is a defect. +that is not `[cold_path]` is a defect** - a nolint is not a substitute. **A vision family file takes every clamp bound from `read_clamp`, never from a literal.** `read_clamp` returns the four scalars stored beside a weight tensor in the projector file (`.input_min` / `.input_max` / `.output_min` / `.output_max`), or an inactive +/-FLT_MAX clamp where the file carries none. -**A vision family whose forward applies no clamp at all says so in its file header.** +**A vision family file whose forward applies no clamp at all, and whose header does not say so, +is a defect.** -**Code two tower families both need - compute, stage/read, or load-orchestration code that -names no family type - lives in `dasllama/dasllama_tower.das` (the encoder-tower home); a -second copy in a family file is a defect.** Per-family serving state - a family's exported -runtime setter and the module global it writes - stays in the family file. +**A family file that holds a second copy of code two tower families both need - compute, +stage/read, or load-orchestration code that names no family type - is a defect; that code +lives in `dasllama/dasllama_tower.das`, the encoder-tower home.** Per-family serving state - a +family's exported runtime setter and the module global it writes - stays in the family file. -**A media splice's rows reach `forward_prefill_embd` in ONE call** - splitting them across -calls, or letting a driver chunk them by row, is a defect: the span bounds are call-relative, -so a boundary inside the span changes the mask. A driver DECLINING the whole call to the CPU -loop, or splitting command buffers per layer, is not a split, and whether the one call also -carries the surrounding head and tail tokens is free. +**Splitting a media splice's rows across more than one `forward_prefill_embd` call, or letting +a driver chunk them by row, is a defect** - the span bounds are counted from the start of the +call, so a boundary inside the span changes the mask. A driver declining the whole call to the +CPU loop, or splitting command buffers per layer, is not a split, and the one call may also +carry the surrounding head and tail tokens. -**A media splice is expressed as two token spans plus a row block, everywhere it appears** - so -BPE merges never cross the media. Any other representation at the seam is a defect. +**A media splice written at the seam as anything other than two token spans plus a row block is +a defect, everywhere the splice appears** - the two-span form is what keeps BPE merges from +crossing the media. -**A change to the media row-block shape re-checks every out-of-module carrier of that shape -in the same change** - `utils/dasllama-server/openai_server.das`. +**A diff that changes the media row-block shape also re-checks every file outside this module +that carries that shape, in the same change** - `utils/dasllama-server/openai_server.das`. -**A family gaining an arm for a media kind reaches the layer stack only through -`forward_prefill_embd`** - a second prefill BODY for it is a defect; a sibling -`eval_embd_span*` entry that feeds the same body is the sanctioned shape. +**A family's new arm for a media kind that reaches the layer stack by any path but +`forward_prefill_embd` is a defect, a second prefill body for it included** - write a sibling +`eval_embd_span*` entry that feeds the same body instead. diff --git a/modules/dasLLAMA/benchmarks/REVIEW.md b/modules/dasLLAMA/benchmarks/REVIEW.md index 441f1dfdff..8df581ccc5 100644 --- a/modules/dasLLAMA/benchmarks/REVIEW.md +++ b/modules/dasLLAMA/benchmarks/REVIEW.md @@ -4,55 +4,55 @@ doc: `../PROFILE.md`. Planned work: `../PERF_LEDGER.md` for a performance followup, `../followup_general.md` for everything else. -**An instrument whose timed body dispatches a pipeline the model runtime selected calls -`tune_gate()` (`../performance/profile_common.das`) before its first timed rep.** A pipeline -the instrument compiled itself is not one the runtime selected, and it does not fire this rule. -Tokenizing and detokenizing run no forward pass, so nothing there is gated: `encode`, -`encode_`, `decode`, and `decode_`, on a `Model` or a `Tokenizer`. Without the gate the -instrument measures fallback kernels silently. - -**A race times both arms interleaved in one process.** A race is an instrument that compares -two implementations. The rule fires on every race this checklist governs. A Metal race -interleaves through `race_pair_ms`. Two separate runs measure the box's drift between them as -much as they measure the arms. - -**An A/B arm that produces no comparable output carries the literal token `timing-only` in its -report line.** The rule fires on an arm reported as adoptable evidence. An arm with no -comparable output exists only to time work. - -**An A/B arm that computes the baseline's result in a different precision prints a -bounded-difference compare.** The rule fires on an arm reported as adoptable evidence. The -compare runs against the baseline arm or against the CPU reference. The report line also -prints the bound the arm passed. - -**An A/B arm that computes the baseline's result in the same precision prints the bit-exact -compare over the sampled region.** The rule fires on an arm reported as adoptable evidence. -The sampled region is the set of output elements the run compares. The compare prints on the -report's "bit-exact vs ..." line. - -**A race that picks between two implementations also checks its baseline arm against a CPU -reference.** The baseline arm is the arm the race already trusts. The reference check runs in -the same process, on the same output elements the arms are judged on. Two arms can agree and -both be wrong; only the reference makes the winner right. - -**A knockout or sweep instrument carries the literal text `ATTRIBUTION SWEEP` in its file -header comment.** A knockout or sweep instrument is one whose arms ATTRIBUTE cost across -stages instead of selecting between two implementations. The same header line also names what -its arms attribute. Without that text a reader takes the sweep's arms for an adoption decision -it never made. - -**An out-of-process observer measures only what a process cannot measure about itself.** An -out-of-process observer is a script that measures a benchmark process from outside. +**A diff that adds or changes an instrument whose timed body dispatches a pipeline the model +runtime selected also calls `tune_gate()` (`../performance/profile_common.das`) before that +instrument's first timed rep.** A pipeline the instrument compiled itself is not one the +runtime selected. Tokenizing and detokenizing run no forward pass, so nothing there is gated: +`encode`, `encode_`, `decode`, and `decode_`, on a `Model` or a `Tokenizer`. Without the gate +the instrument measures fallback kernels silently. + +**A diff that adds or changes a race times both arms interleaved in one process.** A race is +an instrument that compares two implementations. A Metal race interleaves through +`race_pair_ms`. Two separate runs measure the box's drift between them as much as they measure +the arms. + +**A diff that adds or changes an A/B arm that produces no comparable output carries the +literal token `timing-only` in the arm's report line.** The rule fires on an arm reported as +adoptable evidence. An arm with no comparable output exists only to time work. + +**A diff that adds or changes an A/B arm that computes the baseline's result in a different +precision prints a bounded-difference compare.** The rule fires on an arm reported as +adoptable evidence. The compare runs against the baseline arm or against the CPU reference. +The report line also prints the bound the arm passed. + +**A diff that adds or changes an A/B arm that computes the baseline's result in the same +precision prints the bit-exact compare over the sampled region.** The rule fires on an arm +reported as adoptable evidence. The sampled region is the set of output elements the run +compares. The compare prints on the report's "bit-exact vs ..." line. + +**A diff that adds or changes a race that picks between two implementations also checks the +race's baseline arm against a CPU reference.** The baseline arm is the arm the race already +trusts. The reference check runs in the same process, on the same output elements the arms are +judged on. Two arms can agree and both be wrong; only the reference makes the winner right. + +**A diff that adds or changes a knockout or sweep instrument carries the literal text +`ATTRIBUTION SWEEP` in the instrument's file header comment.** A knockout or sweep instrument +is one whose arms attribute cost across stages instead of selecting between two +implementations. The same header line also names what its arms attribute. Without that text a +reader takes the sweep's arms for an adoption decision it never made. + +**An out-of-process observer never measures what the benchmark process can measure about +itself - that measurement goes inside the process instead.** An out-of-process observer is a +script that measures a benchmark process from outside. **A timing instrument this checklist governs never writes the wall time of a binary this repository does not build - a third-party reference tool - into -`../performance/records/.json` or `../PERF_LEDGER.md`**; such walls enter only through +`../performance/records/.json` or `../PERF_LEDGER.md`.** Such walls enter only through the reference cells of `../performance/gen_bench_records.das`, the cells that time such a tool on a board workload. -**An instrument that prints a number formed by subtracting one measured wall from another -prints both raw walls on that report line.** The rule fires on every instrument this checklist -governs. +**A diff that adds or changes an instrument that prints a number formed by subtracting one +measured wall from another also prints both raw walls on that report line.** **A diff that changes what a board cell times ships before/after rows for each affected cell and corpus.** A board cell is a timed cell of the published results board: one diff --git a/modules/dasLLAMA/performance/REVIEW.md b/modules/dasLLAMA/performance/REVIEW.md index 3c98b49752..59aca89004 100644 --- a/modules/dasLLAMA/performance/REVIEW.md +++ b/modules/dasLLAMA/performance/REVIEW.md @@ -3,67 +3,59 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `../ARCHITECTURE.md`. Planned work: `../followup_general.md`. -**`../dasllama/dasllama_exchange_schema.das` is the single validator for exchange submissions - -record stores and tune sidecars.** A second validator is a defect. The engine-free half (no +**Never add a second validator for exchange submissions (record stores and tune sidecars) - +validate through `../dasllama/dasllama_exchange_schema.das` instead.** The engine-free half (no `dasllama/` require beyond the lint macro module) is `REVIEW.das`'s to enforce; weakening that gate is a defect. -**A row or sidecar entering `records/` was minted on a quiet box.** A row shows this with -`hardware.remote_desktop` set to `off`. A sidecar shows it with `provenance.noise` set to -`ok`. A `parsec` row, or a sidecar minted under noise, is a defect - re-mint on a box with no -remote-desktop session. +**A diff that adds a row to `records/` whose `hardware.remote_desktop` is `parsec`, or a +sidecar whose `provenance.noise` is not `ok`, is a defect - re-mint on a box with no +remote-desktop session.** -**A commit stamp entering `records/` names a commit reachable from the branch under review.** -The commit stamps are a `das` row's `sha` and a sidecar's `provenance.engine_sha`. A stamp -naming a commit the branch cannot reach is a defect - re-mint. A stamp that resolves to no -commit at all counts as unreachable. +**A diff that adds a commit stamp to `records/` naming a commit the branch under review cannot +reach is a defect - re-mint.** The commit stamps are a `das` row's `sha` and a sidecar's +`provenance.engine_sha`. A stamp that resolves to no commit at all counts as unreachable. -**A reference-engine row's `sha` entering `records/` matches the standing ref pin -(`DEFAULT_REF_SHA`, `../benchmarks/setup_lcpp_ref.das`).** A row naming another reference -build is a defect - re-mint. +**A diff that adds a reference-engine row to `records/` whose `sha` is not the standing ref pin +(`DEFAULT_REF_SHA`, `../benchmarks/setup_lcpp_ref.das`) is a defect - re-mint.** -**A sidecar entering `records/` carries a `provenance.dasllama_version` equal to -`DASLLAMA_VERSION` (`../dasllama/dasllama_version.das`).** Read `DASLLAMA_VERSION` at the -commit the sidecar's `provenance.engine_sha` names. A version mismatch is a defect - re-mint. +**A diff that adds a sidecar to `records/` whose `provenance.dasllama_version` differs from +`DASLLAMA_VERSION` (`../dasllama/dasllama_version.das`) is a defect - re-mint.** Read +`DASLLAMA_VERSION` at the commit the sidecar's `provenance.engine_sha` names. **A diff that adds a row to `records/.json` mints that row from a board cell.** A board cell is one `gen_bench_records.das` spawns, or a manual `../benchmarks/lcpp_bench.das` cell its `../PROFILE.md` section documents. A timing taken any other way settles its own decision -in its own report and never reaches a record file. Those other ways include a lab's A/B arm, a -reading compared across two processes or two commits, and a wall measured from outside the -benchmark process. +in its own report. Those other ways include a lab's A/B arm, a reading compared across two +processes or two commits, and a wall measured from outside the benchmark process. -**A row entering `records/.json` was timed by the released `lcpp_bench` exe.** That exe -is `../benchmarks/lcpp_bench.das` built by `daspkg release`. A row the `-jit` script timed is -a defect. +**A diff that adds a row to `records/.json` times that row with the released `lcpp_bench` +exe.** That exe is `../benchmarks/lcpp_bench.das` built by `daspkg release`. **A field added to what `write_bench_records` (`profile_common.das`) writes is added to `../dasllama/dasllama_exchange_schema.das`'s run validation in the same change** - the validator ignores run keys it does not know, so an unvalidated field ships silently. -**`../dasllama/dasllama_exchange.das` is the single exchange client - every HTTP call to -the sidecar exchange (lookup, download, submit) goes through it.** A second HTTP path is a -defect; the mechanical half (no second `dashv` requirer under the module) is `REVIEW.das`'s -to enforce. +**Never add a second HTTP path to the sidecar exchange - every HTTP call (lookup, download, +submit) goes through `../dasllama/dasllama_exchange.das`.** The mechanical half (no second +`dashv` requirer under the module) is `REVIEW.das`'s to enforce. **Weakening the exchange download gate, the submission strip, or the submit rails is a defect.** The download gate checks content sha, schema, and `DASLLAMA_VERSION`. The submit -rails keep exchange-sourced and foreign-box sidecars from going back up. -`utils/dasllama-server/test_exchange_client.das` enforces the download gate, the strip, and -the rails. +rails stop a sidecar that came from the exchange, or was minted on another box, from being +submitted. `utils/dasllama-server/test_exchange_client.das` enforces the download gate, the +strip, and the rails. -**Every submission goes through `exchange_strip_private`.** A submission path around it is a -defect even where the strip itself is intact. +**A diff that adds a submission path around `exchange_strip_private` is a defect, even where +the strip itself is intact.** -**A lookup failure never kills a boot.** A boot path that fails when the exchange lookup -fails is a defect. +**A boot path that fails when the exchange lookup fails is a defect.** -**`model_specs()` (text, in `model_specs.das`) and `asr_catalog()` (audio, in -`profile_common.das`) are the model set.** A third FUNCTION in `.das` under this folder that -lists model files, quants, board membership, provenance, or parity fixtures is a defect. Board -membership is which models the site results board shows. Write a new list as a view over those -two functions: it recomputes from them on every call, and it stores no `url`/`bytes`/`sha256` -of its own. +**Outside `model_specs()` (text, in `model_specs.das`) and `asr_catalog()` (audio, in +`profile_common.das`), a `.das` function under this folder that lists model files, quants, +board membership, provenance, or parity fixtures is a defect.** Board membership is which +models the site results board shows. Write a new list as a view over those two functions: it +recomputes from them on every call, and it stores no `url`/`bytes`/`sha256` of its own. **A diff that makes a recorded row or manifest under this folder pin a model file keeps that file's provenance on its own row.** The row is a row of `model_specs()` (`model_specs.das`) or @@ -72,12 +64,12 @@ itself, or a `recipe` a reader can run. One named accessor call may stand in for function in `model_specs.das` whose own body carries those three fields. A second hop does not count - an accessor forwarding to another accessor, or an unnamed table lookup. -**A companion artifact (an mmproj, an image fixture) rides the `companions` of the row that -pins its carrier.** A companion several rows consume is referenced from the other rows by -name - uniqueness itself is `../tests/test_model_specs.das`'s to enforce. +**A diff that adds a companion artifact (an mmproj, an image fixture) puts it in the +`companions` of the row that pins its carrier, and names it from every other row that consumes +it.** Uniqueness itself is `../tests/test_model_specs.das`'s to enforce. -**`fetch_models.das --fetch` downloads only.** A convert, a bench, or a tune-state write -reached from `--fetch` is a defect. Each has its own home: a conversion recipe runs under +**A convert, a bench, or a tune-state write reached from `fetch_models.das --fetch` is a +defect - `--fetch` downloads only.** Each has its own home: a conversion recipe runs under `--convert`, a timing runs in a board cell (`gen_bench_records.das` or a `../benchmarks/lcpp_bench.das` cell), and a tune sidecar is written under a `--tune` run. @@ -85,14 +77,15 @@ reached from `--fetch` is a defect. Each has its own home: a conversion recipe r `recipe`, a new row or a new `companions` entry in `model_specs.das` or `profile_common.das` - or a change to `fetch_models.das` other than its comments, records its settling evidence in the PR description: a `fetch_models.das --` run ending -`0 pending, 0 failed` on a box that HOLDS the pins.** +`0 pending, 0 failed` on a box that already has the pinned files on disk.** **A url-only re-pin - a row's `url` changed with its `bytes` and `sha256` unchanged - records its settling evidence in the PR description: a fetch through the new url into a scratch dir, or a documented resolve of the pinned revision's size and content sha against the row's canonical values** - the verify never reads the url of a file already on disk. -**A refreshed `last_known_good_sidecar.json` is one complete mint from the box its provenance -names, at the current `dasllama_version` - never a hand-edited copy.** `REVIEW.das` (beside -this file) machine-checks the age-independent half (`noise` `ok`, `validation` `ok`, every -`race` winner equal to its `kernels` value); weakening that gate is a defect. +**A diff that refreshes `last_known_good_sidecar.json` replaces it with one complete mint from +the box its provenance names, at the current `dasllama_version` - never a hand-edited copy.** +`REVIEW.das` (beside this file) machine-checks the age-independent half (`noise` `ok`, +`validation` `ok`, every `race` winner equal to its `kernels` value); weakening that gate is a +defect. diff --git a/modules/dasLLAMA/tests/REVIEW.md b/modules/dasLLAMA/tests/REVIEW.md index b8fdbbf902..1daaf95f4c 100644 --- a/modules/dasLLAMA/tests/REVIEW.md +++ b/modules/dasLLAMA/tests/REVIEW.md @@ -8,27 +8,29 @@ the whole directory.** A change reaches a test when it alters anything the test' depends on - the test file, a shared helper, engine code it exercises, an in-tree fixture or corpus it reads, or a name it asserts on; a comment-only edit reaches none. -**A test file in no `run.das` model suite is listed in the `model-free` suite when its cells -still assert what they claim under `DASLLAMA_CPU_PREFILL=1`.** The model suites are every -suite but `model-free`, and that variable is what the runner arms for `model-free`. The -listing lands in the same change that adds the file, and the file skips honestly when its -models are absent. The per-PR gate then runs it wherever the models are stocked. - -**A test file that `DASLLAMA_CPU_PREFILL=1` disarms says so in its header, and is listed in -no `run.das` suite.** `DASLLAMA_CPU_PREFILL=1` is what the runner arms for `model-free`. - -**A test file in a `run.das` model suite (every suite but `model-free`) runs only through -`run.das`; dastest invoked directly on such a file is a defect. A `model-free` file runs +**Leaving a test file that no `run.das` model suite lists out of the `model-free` suite is a +defect, when its cells still assert what they claim under `DASLLAMA_CPU_PREFILL=1`.** The model +suites are every suite but `model-free`, and that variable is what the runner arms for +`model-free`. The listing lands in the same change that adds the file, and the file skips +honestly when its models are absent. The per-PR gate then runs it wherever the models are +stocked. + +**Listing a test file that `DASLLAMA_CPU_PREFILL=1` disarms in any `run.das` suite is a defect, +and so is leaving that fact out of the file's header.** `DASLLAMA_CPU_PREFILL=1` is what the +runner arms for `model-free`. + +**Invoking dastest directly on a test file in a `run.das` model suite (every suite but +`model-free`) is a defect; such a file runs only through `run.das`. A `model-free` file runs through the runner or under plain dastest.** -**Every test RUN runs under `-jit`** - never the interpreter, never AOT. A compile-only CI lane -passes dastest's `--compile-only`. A model-gated suite run under the interpreter is a defect: -its cells skip, and a run of skips is not the coverage the suite owes. +**Every test RUN runs under `-jit` - never the interpreter, never AOT.** A compile-only CI lane +passes dastest's `--compile-only`. Under the interpreter a model-gated suite's cells skip, and +a run of skips is not the coverage the suite owes. -**A `[test]` file that requires any `dasllama/*` module lives in this folder.** Out-of-folder -instances are ledgered in `CLAUDE.md`'s "Out-of-folder test files" note. +**A diff that puts a `[test]` file requiring any `dasllama/*` module outside this folder is a +defect.** Out-of-folder instances are ledgered in `CLAUDE.md`'s "Out-of-folder test files" note. -**A test file in this folder is registered in no `CMakeLists.txt`.** +**A diff that registers a test file in this folder in a `CMakeLists.txt` is a defect.** **A change that re-scopes a test file with a `CLAUDE.md` entry keeps that entry true in the same change.** @@ -38,7 +40,7 @@ whose name does not say what it covers, gets a `CLAUDE.md` entry in the same cha `run.das`'s `model-free` list is the complete census, the `CLAUDE.md` map is deliberately partial. -**A new, renamed, or dropped arm name - the literal passed to `arm_on(t, name)` +**A diff that adds, renames, or drops an arm name - the literal passed to `arm_on(t, name)` (`_model_tier.das`), what `--arm` matches - updates the arm census in `CLAUDE.md`'s "Arm filter mechanics" section in the same change** - an arm the census does not name is unreachable to whoever is choosing what to run. @@ -61,8 +63,8 @@ the real `write_bench_records` output. cached hit at admit (`prefix_attach`) and donates no pages at reap (`donate_stream`) - is a defect.** Cache keys are token ids, and the KV past the splice does not follow from them. -**A test passes or skips explicitly on every platform.** A test that silently vanishes on one -platform is a defect, and so is a zero-assertion pass. A cell whose whole body is +**A test that silently vanishes on one platform is a defect, and so is a zero-assertion pass - +a test passes or skips explicitly on every platform.** A cell whose whole body is platform-gated registers `t |> skip` on the platforms where that body compiles out. **A skip gate keys on a device capability, a run-mode knob's value, or a stocked fixture beside @@ -70,35 +72,35 @@ the models (a model file, an mmproj, an oracle dump - a model gate) - never on t an artifact this repo's build or a previous test run produced (a mint, a generated binary, a dump a test wrote).** An artifact gate goes permanently false when its producer moves. -**A test loading a model over 6 GiB gates on `DASLLAMA_PARITY_FULL=1`** - a final pre-PR -gate, not the iteration loop. In this folder the spelling is `model_available` -(`_model_tier.das`). A test that cannot require `_model_tier.das` open-codes the same env -check. +**A test that loads a model over 6 GiB without gating on `DASLLAMA_PARITY_FULL=1` is a +defect** - that gate is a final pre-PR gate, not the iteration loop. In this folder the +spelling is `model_available` (`_model_tier.das`). A test that cannot require +`_model_tier.das` open-codes the same env check. -**A test whose subject is not the `.dlim` image rail loads each carrier through its own +**A test whose subject is not the `.dlim` image rail never calls `load_model`, +`load_model_cached`, or `load_model_image` - it loads each carrier through that carrier's own loader.** Decoders load through `load_model_` (`../dasllama/dasllama_load.das`). Towers, -embedders, and union carriers load through their family or carrier loaders. Never -`load_model`, `load_model_cached`, or `load_model_image`. +embedders, and union carriers load through their family or carrier loaders. **A function that gains a parameter, or a parameter that gains an accepted value, ships a test for the new value.** The test feeds the new value and checks the result. It lives in this folder, and it lands in the same change. "The model still runs" is not that test. **A predicate whose value is fixed by the build platform - it cannot differ between two runs on -one machine - is tested through the argv it gates or the mode it selects**, never through the -predicate's value. +one machine - is never tested through its own value; test it through the argv it gates or the +mode it selects.** -**An added, moved, or edited registration's test observes reachability** - the registered -thing is reached through its registry, not called directly. The registries this governs: -the arch registrations (`register_decode_override` and its sibling `register_*` hooks), the -`[EnvConfig]` env registry, and the format/backend dispatch tables. A new registry joins that -list in the same change. A `[metal_dispatch]` declaration is not one of them. +**A test for an added, moved, or edited registration reaches the registered thing through its +registry, and never calls it directly.** The registries this governs: the arch registrations +(`register_decode_override` and its sibling `register_*` hooks), the `[EnvConfig]` env +registry, and the format/backend dispatch tables. A new registry joins that list in the same +change. A `[metal_dispatch]` declaration is not one of them. -**A hand-bound kernel gate dispatches the geometry and threadgroup memory its production -encoder does.** A kernel's kargs is its kernel-argument struct. A change to anything a kernel -dispatches with, binds, or reads from its kargs updates every gate that hand-binds that -kernel, in the same change. The mechanism - why a missed threadgroup-memory length fails -silently - is `CLAUDE.md`'s "Metal kernel gates" section. +**A diff that changes anything a kernel dispatches with, binds, or reads from its kargs - the +kernel's kernel-argument struct - updates every gate that hand-binds that kernel, in the same +change.** A hand-bound gate dispatches the geometry and threadgroup memory its production +encoder does. The mechanism - why a missed threadgroup-memory length fails silently - is +`CLAUDE.md`'s "Metal kernel gates" section. **A kernel that gains an in-body branch keyed on a kargs field ships, in the same change, a gate cell that sets that field to the value selecting the new branch.** @@ -106,16 +108,17 @@ gate cell that sets that field to the value selecting the new branch.** **A new pre-tokenizer family or backend ships its `corpus_case` arm in `test_tokenizer.das`, naming the `ggml-vocab-*.gguf` fixture.** -**A `corpus_case` arm asserts exact reference ids AND lossless round-trip.** +**A `corpus_case` arm missing either an exact-reference-id assert or a lossless round-trip +assert is a defect.** -**Every test that compares generated tokens, ids, or logits logs a human-readable form of -BOTH sides: the decoded text where the model carries a vocab (`log_gen_texts` in -`_model_tier.das`, or one line per side), otherwise each side's argmax index and value.** A +**A test that compares generated tokens, ids, or logits without logging a human-readable form +of BOTH sides is a defect: the decoded text where the model carries a vocab (`log_gen_texts` +in `_model_tier.das`, or one line per side), otherwise each side's argmax index and value.** A red, or a suspicious green, must be readable in the log, not only as an id or float difference. -**A fixture claiming a size or depth property asserts the actual number in the test.** A -resize cap is not evidence. +**A fixture claiming a size or depth property whose test does not assert the actual number is +a defect.** A resize cap is not evidence. **A freeform token-parity cell is a defect.** Freeform coverage uses the forced-feed logits-tolerance form. Counting cells stay token-exact. @@ -126,82 +129,85 @@ kernel, or the diff names its kernel class in that file's `CENSUS_NEVER_DISPATCH reason no row can reach it. A dispatching row runs on a small model - one the suite runs without `DASLLAMA_PARITY_FULL=1`. Naming a kernel a census row could dispatch is a defect. -**A kernel-unit cell whose property a CPU oracle can witness compares its kernel against that -oracle.** +**A kernel-unit cell missing a compare against a CPU oracle that can witness the cell's +property is a defect.** -**A kernel-unit cell testing cross-dispatch bit-identity compares GPU against GPU, with its -output buffers prefilled with a sentinel.** No CPU oracle can witness that property. +**A kernel-unit cell testing cross-dispatch bit-identity that does not compare GPU against GPU, +with its output buffers prefilled with a sentinel, is a defect.** No CPU oracle can witness +that property. -**A kernel-unit cell whose output plane is its input plane pairs its compare with an assert -that the output differs from the input at a known index.** This applies when the cell's CPU -oracle does not differ from the input by construction. An in-place kernel that never ran -leaves the input, which can wrongly satisfy a tolerant compare. +**A kernel-unit cell whose output plane is its input plane, and whose compare is not paired +with an assert that the output differs from the input at a known index, is a defect.** This +applies when the cell's CPU oracle does not differ from the input by construction. An in-place +kernel that never ran leaves the input, which can wrongly satisfy a tolerant compare. -**Every ASR family has a token-for-token oracle cell** - one comparing a transcript against a -reference leg, external dump or CPU control alike. +**An ASR family with no token-for-token oracle cell is a defect** - the cell compares a +transcript against a reference leg, external dump or CPU control alike. -**A stocked artifact a test in this folder loads is traceable to something checked in.** -Stocked artifacts are model files, mmprojs, image fixtures, and oracle dumps. Any one of -these counts: a row in `../performance/model_specs.das`; a row's `companions` list; a row in -`asr_catalog` (`../performance/profile_common.das`); a convert script beside -`model_specs.das` in `../performance/`. For an oracle dump, a mint script that regenerates it -also counts - it sits beside the dumps under `models_dir()`, and the test that loads the dump -names it. +**A test in this folder that loads a stocked artifact not traceable to something checked in is +a defect.** Stocked artifacts are model files, mmprojs, image fixtures, and oracle dumps. Any +one of these counts as traceable: a row in `../performance/model_specs.das`; a row's +`companions` list; a row in `asr_catalog` (`../performance/profile_common.das`); a convert +script beside `model_specs.das` in `../performance/`. For an oracle dump, a mint script that +regenerates it also counts - it sits beside the dumps under `models_dir()`, and the test that +loads the dump names it. -**A test that reads a vision encode oracle dump names the minting arm in its header - the -backend, the flash-attention setting, and the mmproj precision the dump came from.** +**A test that reads a vision encode oracle dump without naming the minting arm in its header - +the backend, the flash-attention setting, and the mmproj precision the dump came from - is a +defect.** -**A cell establishes every driver hook and serving-lane knob its claim depends on, and -restores each to its default before returning.** This holds even when the claim needs the -knob at its DEFAULT value. A hook is any process-wide setter with no read-back. The -environment can carry a knob either way. The mechanism (why the hooks flip legs silently) is -`CLAUDE.md`'s "Metal fixtures" section. +**A cell that does not establish every driver hook and serving-lane knob its claim depends on, +and restore each to its default before returning, is a defect.** This holds even when the +claim needs the knob at its DEFAULT value. A hook is any process-wide setter with no +read-back. The environment can carry a knob either way. The mechanism (why the hooks flip legs +silently) is `CLAUDE.md`'s "Metal fixtures" section. -**A cell claiming a family serving lane pins it with `set__q8`, and undoes the pin -with `reset__q8`.** A runtime decline standing in for a pin is a defect: it measures -whichever lane the box's policy picked. +**A cell claiming a family serving lane that does not pin it with `set__q8` and undo +the pin with `reset__q8` is a defect.** A runtime decline standing in for a pin +measures whichever lane the box's policy picked. -**Every CPU-vs-GPU arm runs a PLANAR model for its CPU stages, and that model's -`blob_twin(t, path, seq_cap)` for override-selected stages.** One session spans both models, -because sessions are geometry-bound. A decline-reason cell keeps the planar model. +**A CPU-vs-GPU arm that does not run a PLANAR model for its CPU stages, and that model's +`blob_twin(t, path, seq_cap)` for override-selected stages, is a defect.** One session spans +both models, because sessions are geometry-bound. A decline-reason cell keeps the planar +model. -**A new model-loading block is tagged with its family.** The family tag is the token passed -to `family_on(t, name)` (`_model_tier.das`). An untagged block silently joins every family's -gate. +**A diff that adds a model-loading block tags it with its family.** The family tag is the +token passed to `family_on(t, name)` (`_model_tier.das`). An untagged block silently joins +every family's gate. -**No CPU-control batch parity runs against the 70B.** Its batch coverage is ENGAGE-only in -the support matrix. The batched code paths get their parity on small models, through pins. +**No CPU-control batch parity runs against the 70B.** Its batch coverage is ENGAGE-only in the +support matrix. The batched code paths get their parity on small models, through pins. -**A knob a cell can reach only through the environment is set before the process that reads -it starts.** That process is a child the cell spawns, or the runner's own. Such a knob needs -no restore. An in-cell set is invisible to the running config, which is read once at context -init. +**Setting a knob a cell can reach only through the environment after the process that reads it +starts is a defect - set it before that process starts.** That process is a child the cell +spawns, or the runner's own. Such a knob needs no restore. An in-cell set is invisible to the +running config, which is read once at context init. -**A cell that cannot set an environment-read knob before its reader starts names, in its -assert text, the value it asserts under.** An environment-read knob is one the running config -reads once, at context init. +**A cell that cannot set an environment-read knob before its reader starts, and whose assert +text does not name the value it asserts under, is a defect.** An environment-read knob is one +the running config reads once, at context init. -**A cell asserting the UNPINNED default lane compares against the predicates the lane policy -itself consults - `float_batch_override_active()` and the family's `_gpu_would_serve()` -where one exists - never against a hardcoded lane**; the accelerate tier and the GPU tower move the -default per box, and the assert is on the lane the policy selects, not on one predicate's -own value. +**A cell asserting the UNPINNED default lane never compares against a hardcoded lane - it +compares against the predicates the lane policy itself consults, `float_batch_override_active()` +and the family's `_gpu_would_serve()` where one exists.** The accelerate tier and the +GPU tower move the default per box, so the assert is on the lane the policy selects, not on +one predicate's own value. **A cell that encodes, preprocesses, or asserts on media bytes an encoder consumes - pixels -or audio samples, not a `.dlim` model image - with no model loaded builds its fixture -procedurally and pins its expectations in-repo.** +or audio samples, not a `.dlim` model image - with no model loaded is a defect when it does +not build its fixture procedurally and pin its expectations in-repo.** -**Any media a test feeds an embedder - an image or an audio clip - is a fixture the test -builds, or is previewable via `DASLLAMA_VISION_DUMP` (images)** - a red never requires adding -instrumentation before a human can see what the model consumed. +**Media a test feeds an embedder - an image or an audio clip - that the test does not build, +and that `DASLLAMA_VISION_DUMP` cannot preview (images), is a defect** - a red never requires +adding instrumentation before a human can see what the model consumed. **A tier-1 media fixture - one an embedder-parity cell regenerates in-test and compares -against an oracle dump - has an exact-value generator.** A generator running libm -transcendentals is not float-portable and its cell is a defect; orientation coverage uses -shaped exact fixtures. +against an oracle dump - with no exact-value generator is a defect.** A generator running libm +transcendentals is not exact-value: it is not float-portable. Orientation coverage uses shaped +exact fixtures. -**An embedding-parity cell names its fixture and logs the measured maxdiff on green as well -as red.** +**An embedding-parity cell that does not name its fixture, or does not log the measured +maxdiff on green as well as red, is a defect.** **A new gate, or a new or loosened tolerance bar, ships a control that reds it - a poison, a knockout, a disconnected mechanism, or a cross-lane witness - in the same change.** A bar diff --git a/modules/dasLLVM/REVIEW.md b/modules/dasLLVM/REVIEW.md index 755864919b..0dfb059dd7 100644 --- a/modules/dasLLVM/REVIEW.md +++ b/modules/dasLLVM/REVIEW.md @@ -7,34 +7,36 @@ `tests/README.md` here). The suite is outside the core `tests/` sweep, so no other lane covers it. -- **Work added to, or moved within, what `run_jit` (`daslib/llvm_jit_run.das`) executes - its - own body or any callee - is covered by a printed `LLVM JIT time:` number: its own line, or a - phase's number that covers it while that phase's line still prints** (phase inventory: - `ARCHITECTURE.md` sec.1). Option resolution before the first timer, and log lines, are not work. +- **A diff that adds work to, or moves work within, what `run_jit` + (`daslib/llvm_jit_run.das`) executes - its own body or any callee - also prints an + `LLVM JIT time:` number for that work: its own line, or the number of a phase that includes + it, while that phase's line still prints** (phase inventory: `ARCHITECTURE.md` sec.1). + Option resolution before the first timer, and log lines, are not work. - **A change that can alter the machine code emitted for identical inputs bumps `LLVM_JIT_CODEGEN_VERSION`** (`daslib/llvm_jit_run.das`; what counts as emitting: `ARCHITECTURE.md` sec.1.2). An edit inside an `EMITTER_FILES` file - (`tests-cpp/small/test_jit_emitter_pin.cpp`, repo root) that provably emits identical code - - a comment, a nolint, a same-value rewrite - re-pins `LLVM_JIT_EMITTER_HASH` only. A change - that only SELECTS among existing generators' `[llvm_code]` arguments - the `[tune]` - stamping - needs neither: stamped arguments fold into the cache keys per function. - -- **A new environment or config input to the cache key folds inside `jit_env_salt` - (`daslib/llvm_jit_run.das`), never directly into either cache key - the DLL key or the - split-obj key (`ARCHITECTURE.md` sec.2)** - salt feeds both keys, and a config + (`tests-cpp/small/test_jit_emitter_pin.cpp`, repo root) that provably leaves the emitted + code identical - a comment, a nolint, a same-value rewrite - re-pins + `LLVM_JIT_EMITTER_HASH` only. A change that only SELECTS among existing generators' + `[llvm_code]` arguments - the `[tune]` stamping - needs neither: stamped arguments fold into + the cache keys per function. + +- **A diff that adds an environment or config input to the cache key folds it inside + `jit_env_salt` (`daslib/llvm_jit_run.das`), never directly into either cache key - the DLL + key or the split-obj key (`ARCHITECTURE.md` sec.2)** - salt feeds both keys, and a config folded into one but not the other links stale objects. Inputs that vary per function set (AOT hashes) are key material, not salt. -- **A change to a `[tune]`-family annotation is reviewed with `skills/tune.md`** - the - family's reference. A change to the framework itself - `daslib/llvm_tune.das` or its - tests - is reviewed with `skills/internal/llvm_tune_internals.md`. +- **A change to a `[tune]`-family annotation is reviewed with `skills/tune.md`.** A change to + the framework itself - `daslib/llvm_tune.das` or its tests - is reviewed with + `skills/internal/llvm_tune_internals.md`. - **A diff that adds a new top-level section, or a new value shape inside one, to the tune sidecar (`.tune.json`, written by `daslib/llvm_tune.das`) updates `modules/dasLLAMA/dasllama/dasllama_exchange_schema.das` in the same change and keeps `modules/dasLLAMA/tests/test_exchange_schema.das` green** - the validator allow-lists - sections, so one it does not know fails every newly minted sidecar at submission, and + sections, so a section it does not know fails every newly minted sidecar at submission, and the checked-in corpus the test sweeps cannot show it. - **A diff introducing an override knob adds it to `ARCHITECTURE.md` sec.3's inventory in the @@ -43,12 +45,12 @@ a run compiles, tunes, or emits beyond its defaults. `[tune]`-family annotation arguments are declarations, not overrides. -- **An override announces itself where it takes effect:** at least one logged line names the - knob. Set-but-inert stays silent; an exposure-only diff defers the announce to its - consumer, same change. A knob added, or given a new effect, without its announce is a - defect. +- **A diff that adds an override knob, or gives one a new effect, also logs at least one line + naming the knob where it takes effect.** An override that is set but changes nothing needs + no line; a diff that only exposes the knob defers the line to its consumer, in the same + change. -- **A computed-name env read - `get_env_variable(expr)` / `has_env_variable(expr)` outside - `daslib/llvm_env.das` - is a defect only review catches: spell the name as a literal - through the declared forms, or declare the knob, instead** (the literal-name forms are +- **Never read an environment variable by a computed name - `get_env_variable(expr)` / + `has_env_variable(expr)` outside `daslib/llvm_env.das`. Spell the name as a literal through + the declared forms, or declare the knob, instead** (the literal-name forms are scanner-enforced by `tests/llvm_env_registry.das`; weakening that test is a defect). diff --git a/modules/dasMetal/REVIEW.md b/modules/dasMetal/REVIEW.md index 1dd312b3cd..208e8f7c2a 100644 --- a/modules/dasMetal/REVIEW.md +++ b/modules/dasMetal/REVIEW.md @@ -10,10 +10,10 @@ the diff puts it.** An emitted-text fixture answers to `tests/msl/REVIEW.md` (re - **A new emitter capability ships a text fixture under `tests/msl/` (repo root) and its own census kind, in the same change.** A new emitter capability is a new emit site or a newly - accepted construct, and it includes the derived path a `[metal_kernel]` takes with no - `name=` argument. A census kind is the construct label the emitter records at emit time, - declared in `declared_msl_census` (`tests/msl/_msl_common.das`, repo root). Two emit shapes - sharing one kind lets either vanish unseen. + accepted construct, including the path a `[metal_kernel]` takes when it has no `name=` + argument. A census kind is the construct label the emitter records at emit time, declared in + `declared_msl_census` (`tests/msl/_msl_common.das`, repo root). Two emit shapes sharing one + kind lets either one go untested. - **A new or changed `[metal_kernel]` annotation argument ships a `tests/msl/` (repo root) fixture in the same change.** The fixture asserts what the argument changes: the published @@ -25,7 +25,8 @@ the diff puts it.** An emitted-text fixture answers to `tests/msl/REVIEW.md` (re compile error that names the rejected construct. - **A kernel behavioral change ships a CPU-oracle test under `tests/metal/` (repo root).** A - dasMetal-only regression must red in this module's own suites. + CPU-oracle test compares the GPU result against a CPU-computed expectation. A regression in + dasMetal alone must fail this module's own tests. - **A change visible only in the emitted text ships a `tests/msl/` (repo root) fixture.** The fixture asserts the emitted text that the change alters. @@ -33,15 +34,14 @@ the diff puts it.** An emitted-text fixture answers to `tests/msl/REVIEW.md` (re - **A new or changed host extern under `modules/dasMetal/src/` ships a host-side test under `tests/metal/` (repo root) in the same change.** A changed public function in `metal/das_metal_boost.das` fires this rule too. The test `feint`s when no Metal device is - present. A dasMetal-only regression must red here, not in a consumer module's suite. + present. A regression in dasMetal alone must fail here, not in a consumer module's tests. -- **Weakening `REVIEW.das`'s descriptor cell is a defect.** That cell checks every - `matmul2d_descriptor` that `metal/msl_emit.das` writes, and each one sets - `relaxed_precision = true`. Narrowing the file the cell scans weakens it. So does making the - cell's finding text stop naming what failed. `relaxed_precision = false` keeps the op off - the tensor-unit fast path. +- **Weakening `REVIEW.das`'s descriptor check is a defect.** That check requires every + `matmul2d_descriptor` written in `metal/msl_emit.das` to set `relaxed_precision = true`. + Narrowing the file it scans weakens it. So does making its finding text stop naming what + failed. `relaxed_precision = false` keeps the op off the tensor-unit fast path. -- **A cooperative tensor that a `matmul2d` `run` accumulates into gets no zero-init element - walk.** `get_destination_cooperative_tensor` hands the tensor back already zeroed. The walk +- **Never zero a cooperative tensor element by element before a `matmul2d` `run` accumulates + into it - `get_destination_cooperative_tensor` already hands it back zeroed.** That walk forces every element into real storage before the K loop, and that costs the op its fast path for the whole loop. diff --git a/modules/dasSpirv/REVIEW.md b/modules/dasSpirv/REVIEW.md index 8aa10e806d..85fa264da4 100644 --- a/modules/dasSpirv/REVIEW.md +++ b/modules/dasSpirv/REVIEW.md @@ -4,15 +4,19 @@ emitter rules: `modules/REVIEW_SHADER_EMITTERS.md` - apply that list with this one.** Architecture doc: `MASTERPLAN.md`. -- **A new emitter capability ships its word-level tests in `tests/spirv/` (repo root):** a - `_golden/` disassembly or equivalence fixture, and presence in the opcode census. +- **A diff that adds an emitter capability also adds its tests under `tests/spirv/` (repo + root), in the same change:** a `_golden/` disassembly or equivalence fixture, and presence + in the opcode census. -- **A new rejection path ships its `_fail_closed/` fixture,** with its error needle asserted - in `tests/spirv/test_fail_closed.das`. +- **A diff that adds a rejection path also adds its fixture under `tests/spirv/_fail_closed/` + (repo root) and asserts that fixture's error text in `tests/spirv/test_fail_closed.das`, in + the same change.** A rejection path is emitter code that refuses a construct with a compile + error. -- **A capability that serves a downstream consumer lands with a behavioral arm there** - - dasVulkan `tests/integration` or the dasLLAMA vulkan kernel suite - where the kernel runs - on a device against its CPU oracle. +- **A diff that adds a capability a downstream consumer uses also adds a test there, in the + same change** - in dasVulkan `tests/integration` or the dasLLAMA vulkan kernel suite - that + runs the kernel on a device and compares the result against the same computation run on the + CPU. -- **The opcode census is two-directional:** every opcode a fixture emits is declared, and - every declared opcode is emitted by some fixture. +- **A diff that leaves a fixture emitting an opcode the opcode census does not declare, or + leaves the census declaring an opcode no fixture emits, is a defect.** diff --git a/modules/dasVulkan/REVIEW.md b/modules/dasVulkan/REVIEW.md index 9978a67537..6da7269d11 100644 --- a/modules/dasVulkan/REVIEW.md +++ b/modules/dasVulkan/REVIEW.md @@ -7,10 +7,9 @@ checklist. A generator change - any `generator/*.das` emitter or `generator/skip answers to the `generator/` subfolder's checklist. **Weakening `REVIEW.das` (beside this file) is a defect:** dropping a check, adding a name to -its ignore set, or a finding text that no longer names what failed. What the gate enforces is -read from the gate itself. +its ignore set, or changing a finding text so it no longer names what failed. Read the gate +itself to see what it checks. -**Generated sources are never hand-edited - change `generator/` and regenerate instead.** A diff -touching any `src/dasVULKAN.gen*` file, or any `daslib/*.das` whose first line reads -`// generated by generator/generate.das -- DO NOT EDIT`, with no accompanying `generator/` change -is a defect. +**A diff touching any `src/dasVULKAN.gen*` file, or any `daslib/*.das` whose first line reads +`// generated by generator/generate.das -- DO NOT EDIT`, with no accompanying `generator/` +change is a defect - change `generator/` and regenerate instead.** diff --git a/modules/dasVulkan/generator/REVIEW.md b/modules/dasVulkan/generator/REVIEW.md index 64a9f19313..cd8da21ceb 100644 --- a/modules/dasVulkan/generator/REVIEW.md +++ b/modules/dasVulkan/generator/REVIEW.md @@ -3,6 +3,6 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `../CLAUDE.md`. -**A diff that grows `skip_report.txt` states in its commit message why the newly skipped surface -is acceptable.** CI checks only that the committed report matches a fresh run; whether losing that -bound surface is intended is the reviewer's call. +**A diff that grows `skip_report.txt` states in its commit message why losing the newly skipped +structs and commands is acceptable.** CI checks only that the committed report matches a fresh +run; it does not judge whether the loss is intended. diff --git a/modules/dasVulkan/tutorials/REVIEW.md b/modules/dasVulkan/tutorials/REVIEW.md index 5ddd677537..69014576f0 100644 --- a/modules/dasVulkan/tutorials/REVIEW.md +++ b/modules/dasVulkan/tutorials/REVIEW.md @@ -5,7 +5,7 @@ doc: `../CLAUDE.md`. A recording-pipeline change - a `record_*.das` driver, wher it, or the shared `recording/` harness - is reviewed with `skills/internal/vulkan_recording.md` (repo root). -**A change that adds a tutorial folder ships its `/window/show_.das` viewer in the -same change** (`` = the folder name without the numeric prefix) - a real GLFW window -presenting the live animation. The offscreen pixel-oracle test proves correctness; nothing else -lets a human see the result. +**A diff that adds a tutorial folder also adds its `/window/show_.das` viewer, in +the same change** (`` is the folder name without the numeric prefix) - a GLFW window that +shows the animation as it runs. The offscreen test proves correctness; only the window lets a +human see the result. diff --git a/nano/REVIEW.md b/nano/REVIEW.md index 679809a4ef..8b76b66b38 100644 --- a/nano/REVIEW.md +++ b/nano/REVIEW.md @@ -3,40 +3,42 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `ARCHITECTURE.md`. -**A shared source added to `NANO_SHARED_SRC` compiles with no edit to the file itself.** -A source that needs an edit to build here is split upstream instead - its runtime half moved -into a file of its own - because an edited copy is a fork that drifts silently. - -**A source under `src/` that nano compiles must not include `daScript/ast/ast.h` for anything -beyond `Annotation`, `TypeAnnotation` and `Module::name`.** Those three are all nano's -`include/daScript/ast/ast.h` provides; anything else compiles here and means something -different than it does upstream. - -**A new stub in `src/nano_stubs.cpp` either implements the behaviour or stops the program.** -A stub that returns a default and continues turns a missing feature into a wrong answer on a -target with no debugger attached. - -**A directory holding a target that links `libDaScriptNano` clears its inherited include -directories.** CMake searches a directory's `include_directories()` before any target's own, -so without `set_property(DIRECTORY PROPERTY INCLUDE_DIRECTORIES "")` the target compiles -against the full runtime's headers while linking nano's library. - -**A name added to `include/daScript/simulate/simulate.h` here is a name something in the reuse -set or in generated code refers to.** This header is a subset, not a copy: a name added -speculatively is one nobody will know to remove. - -**A member kept in this folder's `Context` keeps the name and type it has in -`include/daScript/simulate/simulate.h` at the repo root.** Generated code and the reused -headers are written against those names, so a renamed member is a compile error at best and a -different field at worst. - -**A change to what nano leaves out updates the tier table in `ARCHITECTURE.md` and the -"What it leaves out" list in `README.md`.** Both are read by embedders deciding whether their -script fits. - -**A new tier or a new fail-closed seam ships an example under `examples/standalone/` and a -case in `tests-cpp/big/nano_ctx/test_nano_ctx.cpp`.** A tier with no linked program is a tier -that stops working without anything turning red. - -**A number in `README.md` was measured, not estimated.** The table says which toolchain and -which targets produced it, so a reader can reproduce it. +**Never edit a source to make it build under `NANO_SHARED_SRC` - split the file in the full +runtime's tree instead, moving its runtime half into a file of its own.** An edited copy is a +fork that drifts silently. + +**Never use a name from `daScript/ast/ast.h` beyond `Annotation`, `TypeAnnotation` and +`Module::name` in a source under `src/` that nano compiles.** Any other name compiles here and +means something different than it does in the full runtime. + +**Never add a stub to `src/nano_stubs.cpp` that returns a default and continues - implement +the behaviour or stop the program instead.** On a target with no debugger attached, a default +return turns a missing feature into a wrong answer. + +**Never link `libDaScriptNano` from a directory whose `CMakeLists.txt` lacks +`set_property(DIRECTORY PROPERTY INCLUDE_DIRECTORIES "")` - add that line there instead.** +CMake searches a directory's `include_directories()` before any target's own, so without the +line the target compiles against the full runtime's headers while linking nano's library. + +**Never add a name to `include/daScript/simulate/simulate.h` unless a source nano compiles +unchanged from the full runtime, or generated code, already refers to it.** This header is a +subset, not a copy: a name added for later use is one nobody will know to remove. + +**Never rename a member of this folder's `Context` or change its type - it keeps the name and +the type it has in `include/daScript/simulate/simulate.h` at the repo root.** Generated code +and the reused headers are written against those names, so a rename is a compile error at best +and a different field at worst. + +**A diff that changes what nano leaves out also updates the tier table in `ARCHITECTURE.md` +and the "What it leaves out" list in `README.md`, in the same change.** Both are read by +embedders deciding whether their script fits. + +**A diff that adds a tier (a group of script features nano supports as a unit) or a +fail-closed seam (a place where nano stops the program instead of returning a default) also +adds an example under `examples/standalone/` and a case in +`tests-cpp/big/nano_ctx/test_nano_ctx.cpp`, in the same change.** A tier with no program +linking it stops working with nothing turning red. + +**Never put an estimated number in `README.md` - measure it, and name in the table the +toolchain and the targets that produced it.** A reader reproduces the number from those two +facts. diff --git a/site-dasllama/REVIEW.md b/site-dasllama/REVIEW.md index c8ec0f0545..f30bc0d3d1 100644 --- a/site-dasllama/REVIEW.md +++ b/site-dasllama/REVIEW.md @@ -3,44 +3,40 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `README.md`. Planned work: `plans/dasllama_io_site.md`. -**Every shell command shown on a page runs verbatim and produces the result the page shows.** -A command, flag, or output line invented for illustration is a defect; if the implementation -does not exist yet, the page does not show the command. - -**Every code sample shown on a page compiles and runs with the current toolchain.** Same -standard as shell commands: no pseudo-code presented as code. - -**Every number shown is either rendered from live data or copied from a checked-in -measurement record; anything else is a placeholder and carries an HTML comment naming it as -one.** A placeholder that could be mistaken for a fact is a defect. - -**No engine ranking on index.html - a claim in words that one engine beats another -("beats", "Nx faster than", "fastest") is a defect, the generated news region included.** -Figures copied from a checked-in record stand side by side, ours and another engine's; the -arithmetic a reader does is not the page's claim, and ranking in words is -daslang.io/dasllama.html's job. - -**An empty ratio cell always reads "not raced".** Copy or rendering that lets a das-only row -imply parity with a reference is a defect. - -**The `dl-*` measurement-table language lives only in `site/files/dasllama-table.css`.** -New `dl-*` rules in any file under this directory are a defect; site chrome uses the `dio-` -prefix in `files/dasllama-io.css`. - -**Every claim in a news entry is true at publish time - including a claim about what a -page here renders, checked against that page's markup and `files/dasllama-io.js`.** An -entry for something not yet shipped is a defect; entries live in `_news/*.md`. - -**The region between the `news:begin`/`news:end` markers in index.html is `build_news.py` -output.** A hand-edit inside the markers is a defect - edit `_news/` and re-run the -generator; the regenerated index.html, feed.xml and sitemap.xml land in the same change. - -**Every value `files/dasllama-io.js` takes from `/api/*` reaches markup only through the -context-correct escape: `esc()` (which escapes `< > & " '`) for text AND for quoted-attribute -values, `Number(...)` for anything numeric, and `safeApiHref()` (an `^/api/...` allow-list) for -anything landing in `href`/`src` or fed to `fetch`.** A submission-derived value concatenated -into markup with the wrong escape - a bare `esc()`-less number, an unfiltered URL, or a -value inside `"..."` - is a defect. - -**Every page keeps ``, meta description, OpenGraph tags, and the Atom `<link>`.** -A new or renamed page without them is a defect. +**A shell command, flag, or output line shown on a page is a defect unless the command runs +verbatim and produces the result the page shows.** Never show a command for an +implementation that does not exist yet. + +**A code sample shown on a page that does not compile and run with the current toolchain is +a defect.** + +**A number shown on a page is a defect unless it is rendered from live data, copied from a +checked-in measurement record, or carries an HTML comment naming it a placeholder.** + +**A claim in words on index.html that one engine beats another ("beats", "Nx faster than", +"fastest") is a defect, the generated news region included.** Figures copied from a +checked-in record may stand side by side, ours and another engine's - the arithmetic a +reader does is not the page's claim. + +**Copy or rendering that lets a row with no reference engine's figure imply parity with a +reference engine is a defect - an empty ratio cell reads "not raced".** + +**A new `dl-*` CSS rule in any file under this directory is a defect** - the `dl-*` +measurement-table styles live only in `site/files/dasllama-table.css`, and site chrome here +uses the `dio-` prefix in `files/dasllama-io.css`. + +**A claim in a `_news/*.md` entry that is not true at publish time is a defect, including a +claim about what a page here renders - check that one against the page's markup and +`files/dasllama-io.js`.** An entry for something not yet shipped is a defect. + +**A hand-edit between the `news:begin` and `news:end` markers in index.html is a defect - +edit `_news/` and re-run `build_news.py` instead.** A diff that changes `_news/` also lands +the regenerated index.html, feed.xml and sitemap.xml, in the same change. + +**A value `files/dasllama-io.js` takes from `/api/*` and puts into markup without the +context-correct escape is a defect: `esc()` (which escapes `< > & " '`) for text and for +quoted-attribute values, `Number(...)` for anything numeric, and `safeApiHref()` (an +`^/api/...` allow-list) for anything landing in `href`/`src` or fed to `fetch`.** + +**A page that lacks `<title>`, a meta description, OpenGraph tags, or the Atom `<link>` is a +defect - a new or renamed page included.** diff --git a/site/REVIEW.md b/site/REVIEW.md index e8ecc76b66..8fe1134854 100644 --- a/site/REVIEW.md +++ b/site/REVIEW.md @@ -3,73 +3,77 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `README.md`. -**Every hand-written shell command shown on a page runs verbatim and produces the result -the page shows.** A command, flag, or output line invented for illustration is a defect; if -the implementation does not exist yet, the page does not show the command. +**Never show on a page a hand-written shell command, flag, or output line invented for +illustration - show only a command that runs verbatim and produces the result the page +shows.** Never show a command whose implementation does not exist yet. -**The `cmd` field of any run in `files/dasllama/bench_records.json` is the exact argv -`modules/dasLLAMA/performance/gen_bench_records.das` (or the released bench exe it spawns) -ran.** A receipt is per run - one `cmd` + `date` run object - never per rendered row. +**A diff that writes a `cmd` field in `files/dasllama/bench_records.json` that is not the +exact argv `modules/dasLLAMA/performance/gen_bench_records.das` (or the released bench exe it +spawns) ran is a defect.** Never put a `cmd` + `date` pair on a rendered row - the pair sits +on the run object that produced the rows. -**`files/dasllama.js` shows a run's receipt line on every row that run produced.** One run's -`cmd` + `date` covers several rendered rows; a row rendered without its run's receipt line is -a defect. +**A row `files/dasllama.js` renders without its run's receipt line is a defect - render that +line on every row the run produced.** One run's `cmd` + `date` covers several rendered rows. -**`files/dasllama/bench_records.json` is generator output: `modules/dasLLAMA/performance/gen_site_records.das` -merges every `modules/dasLLAMA/performance/records/<box>.json` and applies -`modules/dasLLAMA/performance/records/annotations.json`, so re-running the generator leaves -the file byte-identical.** A diff where it does not is a hand edit and a defect. +**A diff that leaves `files/dasllama/bench_records.json` differing from what re-running +`modules/dasLLAMA/performance/gen_site_records.das` writes is a hand edit and a defect - +change the generator inputs and re-run it instead.** The generator merges every +`modules/dasLLAMA/performance/records/<box>.json` and applies +`modules/dasLLAMA/performance/records/annotations.json`. -**`files/performance_bench.json` is generator output: `benchmarks/sql/_update_results.das --site-json` -writes it from the same sweep output that regenerates the tables in `benchmarks/sql/results.md`.** -A diff that changes `files/performance_bench.json` without changing `benchmarks/sql/results.md` in -the same change, or that leaves any cell in the record differing from the same family-and-lane cell -in the results.md tables, is a defect. +**A diff that changes `files/performance_bench.json` without changing `benchmarks/sql/results.md` +in the same change, or that leaves any cell in the record differing from the same +family-and-lane cell in the `benchmarks/sql/results.md` tables, is a defect.** +`benchmarks/sql/_update_results.das --site-json` writes the record from the same sweep output +that regenerates those tables. -**`files/performance_engines.json` is generator output: `examples/benchmarks/sql/_update_results.das ---site-json` writes it from the same sweep output that regenerates the tables in -`examples/benchmarks/sql/results.md`.** A diff that changes `files/performance_engines.json` without -changing `examples/benchmarks/sql/results.md` in the same change, or that leaves any cell in the -record differing from the same family-and-lane cell in those results.md tables, is a defect. +**A diff that changes `files/performance_engines.json` without changing +`examples/benchmarks/sql/results.md` in the same change, or that leaves any cell in the record +differing from the same family-and-lane cell in the `examples/benchmarks/sql/results.md` +tables, is a defect.** `examples/benchmarks/sql/_update_results.das --site-json` writes the +record from the same sweep output that regenerates those tables. -**`benchmarks.html` renders only the vendored dasProfile records - the same -`files/profile_results_<platform>.json` files pages.yml fetches from borisbat/dasProfile at -deploy time and `files/forge.js`'s front-page cycler reads.** A benchmark number hand-carried into the page -or a second record file for the same data is a defect; provenance (the `das_capture` / -`ext_capture` stamps) travels in the dasProfile files themselves. +**A benchmark number hand-carried into `benchmarks.html`, or a second record file for data a +dasProfile record already holds, is a defect - render the page only from the vendored +dasProfile records, the `files/profile_results_<platform>.json` files pages.yml fetches from +borisbat/dasProfile at deploy time and `files/forge.js`'s front-page cycler reads.** +Provenance - the `das_capture` and `ext_capture` stamps - travels in the dasProfile files +themselves. -**Every code sample shown on a page is a full program - it compiles and runs with the -current toolchain, and the page links it to "try it on playground".** daslang samples are -gen2; a partial snippet shown as a code card, a sample without its playground link, or -pseudo-code presented as code is a defect. +**A partial snippet shown as a code card, a code sample without its "try it on playground" +link, or pseudo-code presented as code is a defect - show every sample as a full program that +compiles and runs with the current toolchain, linked to "try it on playground".** A daslang +sample not written in gen2 is a defect. -**A test under `tests/playground/` (this folder) that needs the daslang runtime carries `@wasm` in its -title.** The per-PR lane stages the site without WASM artifacts and runs the suite with +**A diff that adds a test under `tests/playground/` (this folder) needing the daslang runtime, +or that makes an existing test there need it, puts `@wasm` in that test's title, in the same +change.** The per-PR lane stages the site without WASM artifacts and runs the suite with `--grep-invert '@wasm'` (`playground-e2e.yml`), so an untagged runtime-dependent test fails every PR. -**A change to the deployed playground sources - `playground/` (this folder), `web/examples/ui/src/`, or -`web/examples/ui/samples/` - ships with a stated run of the WASM-staged Playwright suite** -(`tests/playground/`, this folder), naming the result - passes and any failures - in the PR body or -commit message. The run also names its runtime artifacts: built from this change when the -change feeds `web/output/daslang_static.*`, the deployed ones otherwise. The no-WASM lane -cannot see a broken runtime path, and every sample on the page runs through that path. +**A diff that changes the deployed playground sources - `playground/` (this folder), +`web/examples/ui/src/`, or `web/examples/ui/samples/` - states a run of the WASM-staged +Playwright suite (`tests/playground/`, this folder) in its PR body or commit message, naming +the passes and any failures, in the same change.** The stated run also names the runtime +artifacts it used: built from this change when the change feeds `web/output/daslang_static.*`, +the deployed ones otherwise. The no-WASM lane cannot see a broken runtime path, and every +sample on the page runs through that path. -**Every number on a page that reports a measurement - a rate, a duration, a size, a score -some run produced - is rendered from live data, copied from a checked-in measurement record, -or carries a source comment naming it a placeholder.** A placeholder that could be mistaken -for a fact is a defect. A number stating a fixture or a run parameter (an image's dimensions, -a token budget) reports no measurement and is outside this rule. +**A number on a page that reports a measurement - a rate, a duration, a size, a score some run +produced - is a defect when it is not rendered from live data, not copied from a checked-in +measurement record, and not named a placeholder by a source comment.** A placeholder that +could be mistaken for a fact is a defect. A number stating a fixture or a run parameter (an +image's dimensions, a token budget) reports no measurement and is outside this rule. -**A `dl-*` selector defined in BOTH `files/dasllama-table.css` (the file dasllama.io loads) -and `dasllama.html`'s inline `<style>` copy keeps identical bodies.** Changing one copy and -not the other is a defect. +**A diff that changes one copy of a `dl-*` selector defined in BOTH +`files/dasllama-table.css` (the file dasllama.io loads) and `dasllama.html`'s inline `<style>` +copy, and not the other, is a defect - give both copies the same body.** -**A `dl-*` selector is defined where its markup lives - the markup a page renders, including -what `files/dasllama.js` injects: a selector `dasllama.html` renders is defined in -`dasllama.html`, and in `files/dasllama-table.css` too when `site-dasllama/`'s pages (repo -root - the dasllama.io site) render it as well; a selector only `site-dasllama/`'s pages -render is defined in `files/dasllama-table.css` alone.** +**A diff that adds or moves a `dl-*` selector defines it where its markup lives - the markup a +page renders, counting what `files/dasllama.js` injects: in `dasllama.html` when +`dasllama.html` renders it, in `files/dasllama-table.css` as well when `site-dasllama/`'s +pages (repo root - the dasllama.io site) render it too, and in `files/dasllama-table.css` +alone when only `site-dasllama/`'s pages render it.** -**News entries state real, shipped events.** An entry in `_news/*.md` for something not yet -true at publish time is a defect. +**An entry in `_news/*.md` for something not yet true at publish time is a defect - a news +entry states a real, shipped event.** diff --git a/src/builtin/REVIEW.md b/src/builtin/REVIEW.md index d826fcb7bd..272a9928ea 100644 --- a/src/builtin/REVIEW.md +++ b/src/builtin/REVIEW.md @@ -3,19 +3,18 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `ARCHITECTURE.md`. -- **A bind added or changed under this folder binds by module.** The Inline modules are - `$` (builtin), `math`, `strings` and `jit`. In those, a bind whose result is a plain - value - not a reference, and not a result the callee writes into the caller's result - slot - and that is not an interop bind (`addInterop`) uses `addExternInline` or - `addExternInlineEx`; every other module uses an `addExtern...` entry point whose name - does not contain `Inline`. Binds of the generic container and equality helpers declared - in `include/daScript/ast/ast_handle.h` (repo root) - the `das_vector_*`, `das_equ*`, - `das_nequ*`, `das_handle_equ*` and `das_handle_nequ*` families - belong to that header, - not to the module that instantiates them, and this rule does not decide their flavor. - Binds in the Inline modules sit on interpreted inner loops, so each gets its own node the - callee inlines into, while elsewhere binds share one node per signature to keep binaries - small. +- **A bind added or changed under this folder uses `addExternInline` or + `addExternInlineEx` when its module is `$` (builtin), `math`, `strings` or `jit`, its + result is a plain value - not a reference, and not a result the callee writes into the + caller's result slot - and it is not an interop bind (`addInterop`); a bind in any other + module uses an `addExtern...` entry point whose name does not contain `Inline`.** This + rule does not decide the flavor of binds of the generic container and equality helpers + declared in `include/daScript/ast/ast_handle.h` (repo root) - the `das_vector_*`, + `das_equ*`, `das_nequ*`, `das_handle_equ*` and `das_handle_nequ*` families. Binds in + `$`, `math`, `strings` and `jit` run inside interpreted inner loops, so each gets its own + node the callee inlines into, while binds elsewhere share one node per signature to keep + binaries small. -- **Weakening `src/builtin/REVIEW.das`'s bind-flavor scan is a defect** - a bind the gate - reports is fixed by switching the bind, never by editing the gate and never by dropping - a module from `review_nttp.das`'s coverage requires, which are the scan's reach. +- **Weakening `src/builtin/REVIEW.das`'s bind-flavor scan is a defect** - fix a bind the + gate reports by switching the bind, never by editing the gate and never by dropping a + module from `review_nttp.das`'s `require` list, which sets the modules the scan covers. diff --git a/src/parser/REVIEW.md b/src/parser/REVIEW.md index c0856d0da1..cfdf6910b6 100644 --- a/src/parser/REVIEW.md +++ b/src/parser/REVIEW.md @@ -3,7 +3,7 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `skills/internal/cpp_codebase_notes.md` (repo root). -**New surface syntax in `ds2_parser.ypp` or `ds2_lexer.lpp` ships the matching -`tree-sitter-daslang/grammar.js` rule (repo root).** The editor and the MCP search tools parse -with the tree-sitter grammar, not with bison - syntax the grammar lacks disappears from folds, -outline and `grep_usage` the moment it is used. +**A diff that lets `ds2_parser.ypp` or `ds2_lexer.lpp` accept new syntax also adds the matching +rule to `tree-sitter-daslang/grammar.js` (repo root), in the same change.** The editor and the +MCP search tools parse with the tree-sitter grammar, not with bison, so syntax missing from +`grammar.js` does not appear in code folding, outline or `grep_usage`. diff --git a/tests-cpp/big/REVIEW.md b/tests-cpp/big/REVIEW.md index 944e12ca7c..50955ff462 100644 --- a/tests-cpp/big/REVIEW.md +++ b/tests-cpp/big/REVIEW.md @@ -5,5 +5,5 @@ doc: `skills/internal/writing_cpp_tests.md` (repo root). **A diff that touches a test under this folder whose ctest labels include `big` says in the PR that the test ran and passed on the author's machine, naming the command - `ctest -L big` -or the test's own binary.** CI runs `ctest -L small` only, so nothing else proves a -big-labelled test. +or the test's own binary.** CI runs `ctest -L small` only, so no other run shows that a +big-labelled test passes. diff --git a/tests-cpp/small/REVIEW.md b/tests-cpp/small/REVIEW.md index eab6fda763..137bee5dee 100644 --- a/tests-cpp/small/REVIEW.md +++ b/tests-cpp/small/REVIEW.md @@ -3,8 +3,6 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `skills/internal/writing_cpp_tests.md` (repo root). -**A pin test is weakened only by the source change that forces it.** A pin test (`*_pin.cpp`) -asserts that a compiled-in layout, offset, or watched file set stays put. A diff that loosens -or deletes a pin assertion carries, in the same PR, a change to what that assertion watches - -the struct, offset, or file named in the pin's own assertion text; a pin relaxed with no such -source change is a defect. +**A diff that loosens or deletes a pin assertion also changes, in the same PR, what that +assertion watches - the struct, offset, or file named in the pin's own assertion text.** A pin +test (`*_pin.cpp`) asserts that a compiled-in layout, offset, or watched file set stays put. diff --git a/tests/metal/REVIEW.md b/tests/metal/REVIEW.md index 8f44e26599..3dfe07a1b1 100644 --- a/tests/metal/REVIEW.md +++ b/tests/metal/REVIEW.md @@ -3,8 +3,8 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `modules/dasMetal/MASTERPLAN.md`. -**A CPU-oracle test builds a fresh instance of the kernel class for every thread id it -drives.** A CPU-oracle test produces its expected values by running the kernel method on the -CPU. Its driver loop sets `gl_GlobalInvocationID` once per thread id. Kernel members are -per-thread state, so one instance reused across ids carries the previous thread's values into -the next. Sharing one instance is a defect of the test, not of the kernel. +**A CPU-oracle test that reuses one kernel-class instance across thread ids is a defect of the +test, not of the kernel - build a fresh instance for every thread id.** A CPU-oracle test +produces its expected values by running the kernel method on the CPU, in a driver loop that +sets `gl_GlobalInvocationID` once per thread id. Kernel members are per-thread state, so a +reused instance carries the previous thread's values into the next. diff --git a/tests/msl/REVIEW.md b/tests/msl/REVIEW.md index 9dee9d1d80..dbec5b445e 100644 --- a/tests/msl/REVIEW.md +++ b/tests/msl/REVIEW.md @@ -3,9 +3,10 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `modules/dasMetal/MASTERPLAN.md`. -**Weakening the census check is a defect.** The census check runs in two directions: every -kind in `declared_msl_census` is emitted by some fixture, and every kind any fixture emits is -declared there. A kind is the construct label the emitter records at emit time. Dropping -either direction's assertion in `test_msl_census.das` weakens the check. So does dropping a -kind from `declared_msl_census`, or dropping a fixture's census from `all_msl_censuses` (both -in `_msl_common.das`). Text has no disassembler, so the census is the only coverage proxy. +**Weakening the census check is a defect.** The check runs both directions: every kind in +`declared_msl_census` is emitted by some fixture, and every kind any fixture emits is declared +there. A kind is the construct label the emitter records at emit time. Dropping either +direction's assertion in `test_msl_census.das` weakens the check, and so does dropping a kind +from `declared_msl_census` or a fixture's census from `all_msl_censuses` (both in +`_msl_common.das`). Emitted text cannot be read back into constructs, so the census is the +only measure of coverage. diff --git a/tree-sitter-daslang/REVIEW.md b/tree-sitter-daslang/REVIEW.md index 81f739c4cc..38dd22066f 100644 --- a/tree-sitter-daslang/REVIEW.md +++ b/tree-sitter-daslang/REVIEW.md @@ -3,14 +3,15 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `CLAUDE.md`. -**Weakening `REVIEW.das` (beside this file) is a defect** - it byte-diffs `src/parser.c`, -`src/grammar.json`, `src/node-types.json` and `src/tree_sitter/parser.h` against a fresh -`tree-sitter generate` and runs the `test/corpus` suite. +**Weakening `REVIEW.das` (beside this file) is a defect** - it compares `src/parser.c`, +`src/grammar.json`, `src/node-types.json` and `src/tree_sitter/parser.h` byte for byte against a +fresh `tree-sitter generate`, and runs the tests in `test/corpus`. **A `grammar.js` change ships a `test/corpus` case for the syntax it adds or changes.** -**A `grammar.js` change ships a green `modules/dasImgui/tests/test_grammar_canary.das` (repo root) -with a section for the syntax it adds** - the canary only protects syntax it pins. +**A `grammar.js` change ships `modules/dasImgui/tests/test_grammar_canary.das` (repo root) passing, +with a section for the syntax it adds** - that file only protects syntax it pins. -**A red canary is repaired in `grammar.js` and regenerated - never by deleting or loosening a -canary section.** +**A failing `modules/dasImgui/tests/test_grammar_canary.das` (repo root) is fixed by changing +`grammar.js` and re-running `tree-sitter generate` - never by deleting or loosening a section of +that file.** diff --git a/utils/REVIEW.md b/utils/REVIEW.md index 913badeda5..c74f9d013c 100644 --- a/utils/REVIEW.md +++ b/utils/REVIEW.md @@ -3,29 +3,26 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `CLAUDE.md` (repo root). -**A tool's own files answer to that tool's `REVIEW.md` where one exists, keyed by what the file -IS, not where it sits: a file belonging to a tool is reviewed with that tool's checklist as well -as this one.** +**A file that belongs to a tool is reviewed with that tool's own `REVIEW.md`, where one exists, +as well as with this checklist - the tool it belongs to decides, not the directory it is in.** -**Weakening `REVIEW.das` (beside this file) is a defect.** The gate machine-checks this -folder's mechanical rules - what it enforces is read from it, each check's finding text -stating its rule. +**Weakening `REVIEW.das` (beside this file) is a defect.** What the gate checks is read from the +script itself, and each check's finding text states its rule. -**A tool REMOVED from `DAS_UTILS_SHIPPED_EXES` (`CMakeLists.txt`, beside this file) records -the decision in that tool's own `REVIEW.md`, wherever the tool's directory sits, in the same -change** - a removed entry's decision record is the one thing the gate cannot see. A tool -deleted outright records it beside the list in `CMakeLists.txt` instead. +**A tool removed from `DAS_UTILS_SHIPPED_EXES` (`CMakeLists.txt`, beside this file) records +the decision in that tool's own `REVIEW.md`, wherever that tool's directory is, in the same +change** - the gate cannot see a removed entry. A tool deleted outright records the decision +beside the list in `CMakeLists.txt` instead. **A new or changed test for a `utils/` tool whose load-bearing assertions a CI lane can run - -the assertions that prove the change, not a skip-path assertion - is covered, wherever the -diff puts it, by a CI row that executes them, added in the same change if none already -covers it.** A row that only compile-checks the test (`dastest --compile-only`) does not -execute them, and a test whose assertions no row runs passes review once and never runs -again. +the assertions that prove the change, not a skip-path assertion - ships with a CI row that +executes those assertions, wherever the diff puts the test, added in the same change if no +row already covers it.** A row that only compile-checks the test (`dastest --compile-only`) +does not execute them. A test whose assertions no row executes never runs again. **A test for a `utils/` tool whose load-bearing assertions no CI lane can run - they need something no CI machine has - ships with a CI row that compile-checks it.** -**A new or changed test for a `utils/` tool that takes a compile-only row ships with its -executed run recorded in the PR description**: the machine the assertions ran on, what that -machine had that CI lacks, and the pass count. +**A new or changed test for a `utils/` tool that gets a compile-only row records its executed +run in the PR description**: the machine the assertions ran on, what that machine had that CI +lacks, and the pass count. diff --git a/utils/benchctl/REVIEW.md b/utils/benchctl/REVIEW.md index 6cc3725c02..70aa8ba682 100644 --- a/utils/benchctl/REVIEW.md +++ b/utils/benchctl/REVIEW.md @@ -3,6 +3,6 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `README.md`. -**benchctl ships as a prebuilt exe when sqlite is enabled: it stays in `DAS_UTILS` and -`DAS_UTILS_SHIPPED_EXES` (`utils/CMakeLists.txt`)** - dropping either entry breaks the release -bundle. +**A diff that drops `benchctl` from `DAS_UTILS` or from `DAS_UTILS_SHIPPED_EXES` +(`utils/CMakeLists.txt`) is a defect** - those two entries are what puts the prebuilt +`benchctl` exe in the release bundle when sqlite is enabled. diff --git a/utils/das-fmt/REVIEW.md b/utils/das-fmt/REVIEW.md index eae45c4877..bbe95f4637 100644 --- a/utils/das-fmt/REVIEW.md +++ b/utils/das-fmt/REVIEW.md @@ -3,5 +3,6 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `README.md`. -**das-fmt ships as a prebuilt exe: it stays in `DAS_UTILS` and `DAS_UTILS_SHIPPED_EXES` -(`utils/CMakeLists.txt`)** - dropping either entry breaks the release bundle. +**A diff that drops `das-fmt` from `DAS_UTILS` or `DAS_UTILS_SHIPPED_EXES` in +`utils/CMakeLists.txt` (repo root) is a defect** - das-fmt ships as a prebuilt exe, and dropping +either entry breaks the release bundle. diff --git a/utils/dascov/REVIEW.md b/utils/dascov/REVIEW.md index 8eee8205b8..f32a236bd1 100644 --- a/utils/dascov/REVIEW.md +++ b/utils/dascov/REVIEW.md @@ -3,5 +3,6 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `README.md`. -**dascov ships as a prebuilt exe: it stays in `DAS_UTILS` and `DAS_UTILS_SHIPPED_EXES` -(`utils/CMakeLists.txt`)** - dropping either entry breaks the release bundle. +**A diff that removes `dascov` from `DAS_UTILS` or `DAS_UTILS_SHIPPED_EXES` in +`utils/CMakeLists.txt` is a defect.** Either entry missing drops the prebuilt dascov exe from +the release bundle. diff --git a/utils/dasllama-server/REVIEW.md b/utils/dasllama-server/REVIEW.md index 089a18c796..caee42c2ca 100644 --- a/utils/dasllama-server/REVIEW.md +++ b/utils/dasllama-server/REVIEW.md @@ -6,22 +6,22 @@ doc: `README.md`. **A diff to a file that requires a `dasllama/*` module applies `modules/dasLLAMA/REVIEW.md` (repo root) too** - its gate polices the family seams here. -**A fixture under `tests/fixtures/` carries the bytes of a real server run with machine-local -paths normalized, and is never otherwise written or edited by hand** - the capture rail is -`tests/fixtures/README.md`. A diff that changes fixture VALUES without a capture behind it is -a defect. +**Never write or edit a fixture under `tests/fixtures/` by hand - capture the bytes of a real +server run and normalize the machine-local paths instead.** `tests/fixtures/README.md` gives +the capture steps. -**Weakening `REVIEW.das` (beside this file) is a defect** - dropping a check, narrowing what -one scans, adding a name to a check's licensed set (the names that check does not flag), or a -finding text that no longer names what failed. What the gate enforces is read from the gate -itself; each check's finding text states its own rule. +**Weakening `REVIEW.das` (beside this file) is a defect** - dropping a check, narrowing what a +check scans, adding a name to a check's licensed set (the names that check does not flag), or +changing a finding text so it no longer names what failed. What the gate enforces is read from +the gate itself. -**A route added or removed, or its path, method, or a request field the server reads changed, -updates every surface that documents or consumes it in the same change**: the `README.md` -route row, the endpoint table in `doc/source/reference/utils/dasllama_server.rst` (repo root), -and `control.html` where the page consumes it. A surface still describing the old path, -method, or shape is a defect of the change. +**A diff that adds or removes a route, or changes its path, method, or a request field the +server reads, updates every surface that documents or uses that route, in the same change**: +the `README.md` route row, the endpoint table in +`doc/source/reference/utils/dasllama_server.rst` (repo root), and `control.html` where the +page uses it. -**A response wire key added, removed, or renamed re-captures its fixtures under -`tests/fixtures/` in the same change and lands its `README.md` route-row mention** - the -fixtures are the recorded wire shape, and the row is where a consumer learns the key exists. +**A diff that adds, removes, or renames a key in a response body re-captures its fixtures +under `tests/fixtures/` and mentions the key in its `README.md` route row, in the same +change** - the fixtures are the recorded response shape, and the row is where a consumer learns +the key exists. diff --git a/utils/daspkg/REVIEW.md b/utils/daspkg/REVIEW.md index 4862c473f2..69a3c72dcd 100644 --- a/utils/daspkg/REVIEW.md +++ b/utils/daspkg/REVIEW.md @@ -3,40 +3,44 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `README.md`. -**Run the unit suite on every change to a `.das` file in this folder:** `bin/daslang -dastest/dastest.das -- --test utils/daspkg/test_daspkg.das` - fast, no network, interpreted. -Such a change without a green unit run is a defect. +**A change to a `.das` file in this folder without a green unit run is a defect.** The unit +suite is `bin/daslang dastest/dastest.das -- --test utils/daspkg/test_daspkg.das` - fast, no +network, interpreted. -**The integration suite - `bin/daslang dastest/dastest.das -- --test -utils/daspkg/test_daspkg_git.das` - runs on any edit to `commands.das`, `index.das`, or -`utils.das` whose hunks reach a function that runs a git command** - a behavior-preserving -refactor still counts. Needs network (the `borisbat/daspkg-test-*` fixture repos). +**A diff whose hunks in `commands.das`, `index.das`, or `utils.das` reach a function that runs +a git command also runs the integration suite, in the same change** - a behavior-preserving +refactor still counts. The integration suite is `bin/daslang dastest/dastest.das -- --test +utils/daspkg/test_daspkg_git.das`, and it needs network (the `borisbat/daspkg-test-*` fixture +repos). **A change to `cmd_release`, `cmd_release_wasm`, or a `release_*` helper is verified on macOS, -or the review says it was not.** The release layout forks per platform (`.app` bundle vs flat -directory), and no per-PR CI runs these suites. +or the review says it was not.** The release layout differs per platform (`.app` bundle vs flat +directory), and no per-PR CI runs the daspkg test suites. -**A new command or flag lands with its test cell, its `print_usage` line, and its README table -row in the same change.** +**A diff that adds a command or a flag also adds its test, its `print_usage` line, and its +`README.md` table row, in the same change.** -**daspkg ships as a prebuilt exe: it stays in `DAS_UTILS` and `DAS_UTILS_SHIPPED_EXES` -(`utils/CMakeLists.txt`)** - dropping either entry breaks the release bundle. +**A diff that drops `daspkg` from `DAS_UTILS` or `DAS_UTILS_SHIPPED_EXES` in +`utils/CMakeLists.txt` (repo root) is a defect** - daspkg ships as a prebuilt exe, and dropping +either entry breaks the release bundle. -**A `cmd_release` bundle always mints the tune sidecar.** A `cmd_release` bundle that ships an -exe without a sidecar beside it is a defect. +**A `cmd_release` bundle that ships an exe without a tune sidecar beside it is a defect** - the +tune sidecar is the `<bundle>.tune.json` file of measured kernel choices the exe reads at run +time. -**`--quick` is the only path that inherits a prior sidecar, and it refuses an incomplete one** - - incomplete meaning missing any scope key the exe's deps JSON reports, the same completeness -the release build itself checks. +**A diff that lets a release path other than `--quick` inherit a prior sidecar, or lets +`--quick` accept an incomplete one, is a defect** - incomplete meaning missing any scope key the +exe's deps JSON reports, the same completeness the release build itself checks. -**`release_include_if_missing` files are user-owned after initialization.** A release path that -overwrites or deletes one, on any platform, is a defect. +**A release path that overwrites or deletes a `release_include_if_missing` file, on any +platform, is a defect.** -**`.daspkg_release.manifest` is written on every platform.** +**A release path that finishes a bundle without writing `.daspkg_release.manifest`, on any +platform, is a defect.** -**Unit cells touch only local fixtures.** A `test_daspkg.das` cell that reaches the network is -a defect - network coverage belongs in `test_daspkg_git.das`. +**A test in `test_daspkg.das` that reaches the network is a defect** - network coverage belongs +in `test_daspkg_git.das`. -**A package, bundle, or app name that reaches a shell command is validated by -`is_safe_pkg_name` first, and only `commands.das` may build such a command** (the validator is -private to it); an interpolation site without the check, or outside `commands.das`, is a defect. +**A shell command built from a package, bundle, or app name outside `commands.das`, or without +an `is_safe_pkg_name` check on that name first, is a defect** - `is_safe_pkg_name` is private to +`commands.das`. diff --git a/utils/detect-dupe/REVIEW.md b/utils/detect-dupe/REVIEW.md index 4f0f6dd011..f1c32f196a 100644 --- a/utils/detect-dupe/REVIEW.md +++ b/utils/detect-dupe/REVIEW.md @@ -3,6 +3,6 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `README.md`. -**Decision record: detect-dupe's `DAS_UTILS_SHIPPED_EXES` membership is deliberate** - it -ships as a prebuilt exe. `utils/REVIEW.das` machine-checks the membership itself; a diff -removing the entry records the removal decision here, per `utils/REVIEW.md`. +**A diff removing detect-dupe from `DAS_UTILS_SHIPPED_EXES` records the removal decision in +this file, per `utils/REVIEW.md`.** detect-dupe ships as a prebuilt exe, and `utils/REVIEW.das` +machine-checks the membership itself. diff --git a/utils/internal/dasllama-ladder/REVIEW.md b/utils/internal/dasllama-ladder/REVIEW.md index 5e0720558d..9f132ddef7 100644 --- a/utils/internal/dasllama-ladder/REVIEW.md +++ b/utils/internal/dasllama-ladder/REVIEW.md @@ -3,103 +3,104 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `README.md`. Planned work: `plans/dasllama_io_site.md`. -**Every route, every store operation, and every config or limit behavior has a dastest test in -this directory** - `main.das` and `admin.das` stay argv/dispatch glue over tested modules, so -they need none of their own. - -**`[test]` files live in this directory and require siblings by bare name** - never under the -global `tests/` tree, and never registered in any `CMakeLists.txt`. - -**HTTP tests go through the local `with_ladder_server` harness (`test_ladder_server.das`) on -this directory's reserved test port 19015; store tests call `ladder_store` directly with no -server.** A store behavior proven only through HTTP, or an HTTP behavior proven only against -the store, is a defect. - -**A test that touches the filesystem uses `temp_directory`-rooted paths and deletes what it -creates; store tests run against `:memory:`.** A test writing into the repo tree is a defect. - -**Operator routes (`/admin/*`, `/shutdown`) never appear in `caddy.snippet` - or in any -Caddyfile route reaching this service, a catch-all included.** Caddy proxies from the same box, -so `is_loopback_peer` sees `127.0.0.1` for every proxied request - the Caddyfile is the only -real boundary. Operators reach these routes on the service port itself, over the ssh tunnel; -no Caddyfile entry exists for them. - -**`is_operator_caller` requires ALL THREE of a loopback transport peer, a loopback `Host` -authority, and same-origin-or-headerless (the request carries no `Origin` header, or its -`Origin` names the same authority as its `Host`), and every operator route gates through it.** -Dropping any of the three, or gating an operator route on the peer alone, is a defect. The -`Host` and `Origin` halves together are the CSRF guard for the `/admin/` page over an ssh -tunnel: same-origin refuses a plain cross-origin fetch, and the loopback-`Host` requirement -refuses DNS rebinding (comparing `Origin` to `Host` alone does not - a rebound name owns both). - -**`caddy.snippet` is the authoritative copy of the public route boundary: the deployed -Caddyfile is edited to match it, never the reverse, and a route a public caller needs lands in -it in the same change as its handler.** - -**Every route that serves board data, mutates the store, or refuses a caller logs one -`ladder.req` line through `log_request`, including the refusals.** A response path on such a -route that skips the log is a defect. (The bare liveness probe `GET /healthz` serves no data -and is silent by design.) - -**Every handler that consumes a request body checks `body_is_byte_faithful` before using the -string view.** A body used without the NUL guard is a defect. - -**A store-backed handler does exactly four things - validate transport shape, gate the request -(operator gate, submit-open, attempt-limit), make one store call, format the response.** SQL, -hashing, and store policy live in `ladder_store.das`, and HTTP never does - no `dashv` require -there. (The `/admin/` page and `/healthz` reach no store and are not store-backed handlers.) - -**Public submissions default closed: `submit_open` is false in both `LadderArgs` and -`LadderPolicy`, the `/api/submit/sidecar` and `/api/submit/records` handlers return 403 while it -is false, and only the loopback `/admin/submit` route flips it.** A default of true, or an opener -reachable from any non-loopback path, is a defect. - -**In `caddy.snippet` the large-body allowance appears on the `/api/submit/records` and -`/api/submit/sidecar` matcher only; every other proxied route carries the small read cap.** A -large cap on any other route, or a read cap on a submit route, is a defect. - -**Nothing in this directory requires the dasLLAMA engine, dasLLVM, or any model machinery; the -only dasLLAMA module required is the engine-free `dasllama/dasllama_exchange_schema` (a public -entry of that module's facade lint), and only `ladder_store.das` requires it** (`README.md` -sec.3). Its `dasllama_lint` carrier is a compile-time macro, not engine code. - -**Every sidecar document - community or planted - passes `validate_sidecar_submission` before -any row is written (a sidecar without the version stamp can never be served, so storing one is -always an error); every community record store passes `validate_record_submission`; -`import_official_store` validates with `validate_record_store` (shape only - official record -history predates the release counter).** A write path that skips validation is a defect. - -**`Source` and `Verified` are written only from store code, and no value from a public -(proxied) request reaches either column.** The loopback operator surface chooses `Verified` -by design - through store calls, never through SQL in a handler. - -**Every sidecar - community or planted - is privacy-cleaned through `exchange_strip_private` -before validation, hashing, and storage, and a community record store through -`redact_record_paths`; the cleaned text IS the document.** Hashing the cleaned text on both -sidecar paths is what makes re-plant promote instead of duplicate. `import_official_store` -alone stores its document as-is. - -**After insert a document is never mutated.** Derived columns may be recomputed; the document -itself may only be inserted, or deleted with its submission. - -**A shipped `[sql_migration]` body is never edited.** Schema change means a new, higher -version in the same stream. - -**Sidecars are content-addressed: the row key is the sha256 of the stored document.** A -sidecar row whose `Sha` is not the hash of its `Doc` is a defect. - -**A new operator-edited file the box runs from is added to `.das_package`'s `release()` in the -same change.** - -**Every privileged (root) deploy action goes through `dasllama-deploy.sh`, and -`dasllama-deploy.sudoers` grants NOPASSWD for exactly `/usr/local/sbin/dasllama-deploy.sh` and -nothing else** - a second command, a wildcard target, a bare `ALL`, or a shell is a defect. - -**The systemd unit that `provision` writes runs the service sandboxed** (`ProtectSystem=strict`, emptied -`CapabilityBoundingSet`, `ReadWritePaths` limited to the data dir and the release tree). A diff -that changes where the service or watchdog writes at runtime - log path, working directory, -database location - without a matching `ReadWritePaths` entry is a defect; so is relaxing -`ProtectSystem` or restoring a capability. +**A diff that adds or changes a route, a store operation, or a config or limit behavior also +adds a dastest test for it in this directory, in the same change.** `main.das` and `admin.das` +stay argv/dispatch glue over tested modules, so they need no test of their own. + +**Never put a `[test]` file for this folder under the repo-root `tests/` tree, and never +register one in any `CMakeLists.txt` - a `[test]` file lives in this directory and requires its +siblings by bare name.** + +**Never reach the service from a test except through the local `with_ladder_server` harness +(`test_ladder_server.das`) on this directory's reserved test port 19015, and never start a +server in a store test - a store test calls `ladder_store` directly.** A store behavior proven +only through HTTP, or an HTTP behavior proven only against the store, is a defect. + +**Never let a test write into the repo tree or anywhere outside `temp_directory` - root every +path the test creates under `temp_directory` and delete it before the test ends; a store test +opens `:memory:`, never a file.** + +**Operator routes (`/admin/*`, `/shutdown`) never appear in `caddy.snippet`, or in any +Caddyfile route that reaches this service, a catch-all included - operators reach them on the +service port itself, over the ssh tunnel.** Caddy proxies from the same box, so +`is_loopback_peer` sees `127.0.0.1` for every proxied request, which leaves the Caddyfile as +the only real boundary. + +**Never drop one of the three checks in `is_operator_caller` - a loopback transport peer, a +loopback `Host` authority, and same-origin-or-headerless (the request carries no `Origin` +header, or its `Origin` names the same authority as its `Host`) - and never gate an operator +route on the peer alone: every operator route gates through `is_operator_caller`.** The `Host` +and `Origin` checks together are the CSRF guard for the `/admin/` page over an ssh tunnel - +same-origin refuses a cross-origin fetch, and the loopback `Host` refuses DNS rebinding. + +**A diff that adds a route a public caller needs also adds it to `caddy.snippet`, in the same +change as its handler; never edit `caddy.snippet` to match the deployed Caddyfile - edit the +deployed Caddyfile to match `caddy.snippet` instead.** + +**On a route that serves board data, mutates the store, or refuses a caller, a response path +that does not log one `ladder.req` line through `log_request` is a defect - refusals log too.** +The bare liveness probe `GET /healthz` is none of these and logs nothing. + +**A handler that reads a request body without first checking `body_is_byte_faithful` is a +defect.** + +**A store-backed handler that does anything beyond four things - check transport shape, gate +the request (operator gate, submit-open, attempt-limit), make one store call, format the +response - is a defect.** SQL, hashing, and store policy go in `ladder_store.das`, which never +requires `dashv`. The `/admin/` page and `/healthz` reach no store and are not store-backed +handlers. + +**A diff that defaults `submit_open` to true in `LadderArgs` or `LadderPolicy`, lets +`/api/submit/sidecar` or `/api/submit/records` answer anything but 403 while `submit_open` is +false, or adds an opener reachable from a non-loopback path, is a defect** - only the loopback +`/admin/submit` route flips the gate. + +**In `caddy.snippet` every proxied route other than the `/api/submit/records` and +`/api/submit/sidecar` matcher carries the small read cap; a large-body allowance on any other +route, or a read cap on either submit matcher, is a defect.** + +**Never require the dasLLAMA engine, dasLLVM, or any model machinery from a file in this +directory - the one dasLLAMA module allowed is the engine-free +`dasllama/dasllama_exchange_schema` (a public entry of that module's facade lint), required +from `ladder_store.das` and nowhere else** (`README.md` sec.3). Its `dasllama_lint` carrier is +a compile-time macro, not engine code. + +**A write path that stores a document without validating it first is a defect: every sidecar - +community or planted - passes `validate_sidecar_submission`, every community record store +passes `validate_record_submission`, and `import_official_store` passes `validate_record_store` +(shape only - official record history predates the release counter).** A sidecar without the +version stamp can never be served, so storing one is always an error. + +**Never write `Source` or `Verified` outside store code, and never let a value from a public +(proxied) request reach either column - the loopback operator surface sets `Verified` through +store calls, never through SQL in a handler.** + +**Never validate, hash, or store sidecar text - community or planted - that has not been +through `exchange_strip_private`, and never store a community record store that has not been +through `redact_record_paths`.** Hashing the cleaned text on both sidecar paths is what makes +re-plant promote instead of duplicate. `import_official_store` stores its document as-is. + +**Never update a document after its insert - derived columns may be recomputed, and a document +leaves only by deleting its submission.** + +**Never edit a shipped `[sql_migration]` body - a schema change adds a new, higher version in +the same stream.** + +**A sidecar row whose `Sha` is not the sha256 of its stored `Doc` is a defect.** + +**A diff that adds an operator-edited file the box runs from also adds it to `.das_package`'s +`release()`, in the same change.** + +**A privileged (root) deploy action that does not go through `dasllama-deploy.sh` is a defect, +and so is any `dasllama-deploy.sudoers` grant beyond NOPASSWD for exactly +`/usr/local/sbin/dasllama-deploy.sh`** - a second command, a wildcard target, a bare `ALL`, or +a shell. + +**A diff that changes where the service or watchdog writes at runtime - log path, working +directory, database location - also adds a matching `ReadWritePaths` entry to the systemd unit +`provision` writes, in the same change.** Relaxing that unit's sandbox is a defect: +`ProtectSystem=strict`, an emptied `CapabilityBoundingSet`, and `ReadWritePaths` no wider than +the data dir and the release tree. **Placement - one file, one line: a diff keeps each file inside its line, and a new file adds its line here, with its tests, in the same change.** diff --git a/utils/internal/dasweb-buildd/REVIEW.md b/utils/internal/dasweb-buildd/REVIEW.md index 90301728ab..2b2115301c 100644 --- a/utils/internal/dasweb-buildd/REVIEW.md +++ b/utils/internal/dasweb-buildd/REVIEW.md @@ -3,86 +3,91 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `README.md`. Planned work: `plans/dasweb_wasm_pipeline.md`. -**Every config, core, and client behavior has a dastest test in this directory** - `main.das` -and `buildd_service.das` stay glue and orchestration over tested modules; the whole loop is -proven by the end-to-end checkpoint on the real boxes. - -**`[test]` files live in this directory and require siblings by bare name** - never under the -global `tests/` tree, and never registered in any `CMakeLists.txt`. - -**HTTP tests run against the stub playground in `test_buildd_client.das` on this directory's -reserved test port 19014; core tests touch no network.** The playground's real endpoints are +**A diff that adds or changes behavior in `buildd_config.das`, `buildd_core.das`, or +`buildd_client.das` ships a dastest test in this directory, in the same change.** A diff that +puts behavior beyond glue and orchestration over those modules into `main.das` or +`buildd_service.das` is a defect. The whole loop is proven by the end-to-end checkpoint on the +real boxes. + +**Never put a `[test]` file under the global `tests/` tree, and never register one in a +`CMakeLists.txt` - a `[test]` file for this directory lives here and requires its siblings by +bare name.** + +**A test that reaches any host or port other than the stub playground in +`test_buildd_client.das` on this directory's reserved test port 19014 is a defect, and so is a +test of `buildd_core.das` that reaches the network at all.** The playground's real endpoints are proven in its own directory. -**A test that touches the filesystem uses `temp_directory`-rooted paths and deletes what it -creates.** A test writing into the repo tree is a defect. +**A diff that adds or changes a test that touches the filesystem roots that test's paths in +`temp_directory` and deletes what the test creates.** -**The health server binds loopback only** - `set_bind_host("127.0.0.1")` between `init` and -`start`. A diff that removes, reorders past `start`, or conditionalizes the bind is a defect. +**Never let the health server bind an address other than loopback - it calls +`set_bind_host("127.0.0.1")` between `init` and `start`.** A diff that removes that call, moves +it past `start`, or makes it conditional is a defect. -**`POST /shutdown` verifies the transport peer is loopback.** No header can carry that proof. +**Never let `POST /shutdown` act on a request without checking that the transport peer address +is loopback.** No header can carry that proof. -**The bearer token never appears in a log line, an error message, or the startup banner.** The -banner logs set/unset and provenance only. +**Never write the bearer token into a log line, an error message, or the startup banner - the +banner reports set or unset and the provenance instead.** -**Every file name that reaches the filesystem - bundle sources and build outputs alike - passes -this directory's name validation first.** A path assembled from an unvalidated request- or -build-derived name is a defect. +**A diff that assembles a filesystem path from a request- or build-derived name - a bundle +source or a build output alike - without passing that name through this directory's name +validation first is a defect.** -**A build publishes exactly the file set its mode declares, by name.** The build executes the -user's own compile-time code and can write anything into the output directory, so a -suffix filter, a glob, or any rule that lets the build widen its own output set is a defect. +**A diff that lets a build publish a file its mode does not declare by name - through a suffix +filter, a glob, or any rule that widens the build's own output set - is a defect.** The build +runs the user's own compile-time code and can write anything into the output directory. -**Builds run in the container sandbox defined by `Containerfile` and `run_build.sh`, and there -is no unsandboxed path.** Every host path the build may touch is an explicit mount in that -script; a change that adds a mount without a stated reason, or that reintroduces a -run-outside-the-sandbox fallback, is a defect. Mounting anything that holds a secret, a key, or -another service's data is a defect. +**A diff that adds a mount to `run_build.sh` without a stated reason, that gives a build a host +path which is not an explicit mount in that script, or that adds a way to run a build outside +the container sandbox defined by `Containerfile` and `run_build.sh`, is a defect.** Mounting +anything that holds a secret, a key, or another service's data is a defect. -**The sandbox is the boundary, and daslang's own policies are not.** A change justified by -`no_unsafe`, `no_init`, or any compile-time policy flag standing in for isolation is a defect: -compile-time code reads files with no `unsafe` at all. +**A change justified by `no_unsafe`, `no_init`, or any compile-time policy flag standing in for +the sandbox's isolation is a defect.** Compile-time code reads files with no `unsafe` at all. -**Processes are spawned via `popen_argv` (no shell).** A build or git invocation through a -shell-interpreted string is a defect. +**A diff that spawns a process through a shell-interpreted string is a defect - spawn it with +`popen_argv` instead.** -**An empty configured token refuses startup, never starts an unauthenticated poller.** +**Never let the service start with an empty configured token - refuse startup instead.** -**Every claimed job logs its start and its outcome (done/failed, duration, exit code) as -structured lines.** A job path that can end without a log line is a defect. +**A diff that leaves a claimed job able to end without structured log lines for its start and +its outcome (done or failed, duration, exit code) is a defect.** -**Startup logs the full effective config with per-key provenance before the first poll.** +**A diff that leaves a config key out of the startup log, drops that key's provenance, or moves +that log after the first poll is a defect.** -**Lifecycle-owned state is module-global; no collectable value lives in a `main`-loop local -across `maybe_collect_gc()`.** +**Never put lifecycle-owned state anywhere but a module-global, and never hold a collectable +value - one the GC can free - in a `main`-loop local across a call to `maybe_collect_gc()`.** -**Every claimed job is resolved by an upload - success or failure - on every code path.** A -path that drops a claim for the stale-requeue sweep to mop up is a defect. +**A diff that adds a code path where a claimed job ends without an upload - success or failure - +is a defect.** Leaving the claim for the stale-requeue sweep is not a resolution. -**Per-job scratch directories are removed when the job resolves.** +**Never let a job resolve without removing its per-job scratch directory.** -**Route callbacks are retained with `push`, never `emplace`.** +**Never retain a route callback with `emplace` - use `push` instead.** -**A behavior change in a box-side file - `run_build.sh`, `roll_toolchain.sh`, `.das_package`, -`watchdog.json`, `dasweb-buildd.toml` - lands with its note in the matching `README.md` -section** (The sandbox / The toolchain-bump protocol / Run). +**A diff that changes the behavior of a box-side file - `run_build.sh`, `roll_toolchain.sh`, +`.das_package`, `watchdog.json`, `dasweb-buildd.toml` - also updates the matching `README.md` +section (The sandbox / The toolchain-bump protocol / Run), in the same change.** -**A toolchain roll that moves the worktree rebuilds both the cross-compile host and the -runtime archive.** +**Never let a toolchain roll move the worktree without rebuilding both the cross-compile host +and the runtime archive.** -**A `Containerfile` change lands with an image-tag bump in `run_build.sh` (`IMAGE=`), same -commit.** +**A diff that changes `Containerfile` also bumps the image tag in `run_build.sh` (`IMAGE=`), in +the same change.** -**The wasm-archive step's build command and archive list mirror -`modules/dasImgui/.das_package`, minus `liblibDasModuleClipboard.a`:** a change to either side -lands with the other. +**A diff that changes the wasm-archive step's build command or archive list, or +`modules/dasImgui/.das_package`, also changes the other to match, in the same change - the two +differ only in `liblibDasModuleClipboard.a`, which the wasm-archive list leaves out.** -**A file an operator edits on the box is preserved across upgrades:** shipped -`release_include_if_missing`, never plain `release_include`. +**Never ship a file an operator edits on the box with plain `release_include` - use +`release_include_if_missing`, so an upgrade keeps the operator's edits.** -**Checked-in config holds development values.** The config file is discovered automatically -beside the module, so a production token or URL in it would make an in-repo run claim -production jobs. +**Never commit a production value to `dasweb-buildd.toml` - the checked-in file holds +development values.** The config file is discovered automatically beside the module, so a +production token or URL there would make an in-repo run claim production jobs. **Placement - one file, one line: a diff keeps each file inside its line, and a new file adds its line here, with its tests, in the same change.** diff --git a/utils/internal/dasweb-playground/REVIEW.md b/utils/internal/dasweb-playground/REVIEW.md index 9be74ba0c8..51c1fedf8c 100644 --- a/utils/internal/dasweb-playground/REVIEW.md +++ b/utils/internal/dasweb-playground/REVIEW.md @@ -3,115 +3,115 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `README.md`. Planned work: `plans/dasweb_backend.md`. -**Every route, every store operation, and every config or limit behavior has a dastest test in -this directory** - `main.das` and `admin.das` stay argv/dispatch glue over tested modules, so -they need none of their own. +**A diff that adds or changes a route, a store operation, or a config or limit behavior covers +it with a dastest test in this directory, in the same change.** `main.das` and `admin.das` stay +argv/dispatch glue over tested modules, so they need no test of their own. -**`[test]` files live in this directory and require siblings by bare name** - never under the -global `tests/` tree, and never registered in any `CMakeLists.txt`. +**Never register a `[test]` file in a `CMakeLists.txt`, and never put one under the repo-root +`tests/` tree - `[test]` files live in this directory and require their siblings by bare name.** -**HTTP tests go through a local `with_*_server` harness (`test_playground_server.das`, -`test_build_endpoints.das`) on this directory's reserved test ports 19011, 19012, and 19013; -store tests call `samples_store` or `build_queue` directly with no server.** A store behavior -proven only through HTTP, or an HTTP behavior proven only against the store, is a defect. +**A diff that adds an HTTP test drives it through a local `with_*_server` harness +(`test_playground_server.das`, `test_build_endpoints.das`) on this directory's reserved test +ports 19011, 19012, and 19013; a store test calls `samples_store` or `build_queue` directly, +with no server.** A store behavior proven only through HTTP, or an HTTP behavior proven only +against the store, is a defect. **A test that touches the filesystem uses `temp_directory`-rooted paths and deletes what it -creates.** A test writing into the repo tree is a defect. +creates.** -**The server binds loopback only** - `set_bind_host("127.0.0.1")` between `init` and `start`. -A diff that removes, reorders past `start`, or conditionalizes the bind is a defect. +**Never remove `set_bind_host("127.0.0.1")`, move it past `start`, or make it conditional - it +stays between `init` and `start`, so the server binds loopback only.** -**Every SQL statement goes through the `daslib/sql_linq` rail (`_sql` / `insert` / -`_sql_update`) or bound parameters.** A query assembled by string interpolation or `format` -from any request-derived value is a defect. +**A diff that adds a SQL statement puts it through the `daslib/sql_linq` rail (`_sql` / +`insert` / `_sql_update`) or bound parameters.** A query assembled by string interpolation or +`format` from any request-derived value is a defect. -**Every request body is bounds-checked against the configured cap before it is hashed or -stored, and the transport cap that precedes it lives in `caddy.snippet`.** The service can only -check a body already buffered in full, so a route accepting a body without a `request_body` -limit in front of it is a defect, as is a hash or insert reachable without the size check. +**A route that accepts a body without a `request_body` limit in `caddy.snippet` in front of it +is a defect, as is a hash or insert reachable without the size check against the configured +cap.** The service can only check a body already buffered in full. -**A request body is read as bytes, and a byte count that disagrees with its string length is -rejected.** A das string ends at the first NUL, so a body used without that guard can be stored -truncated under a hash that does not describe it. +**Never use a request body whose byte count disagrees with its string length - read the body as +bytes and reject the mismatch.** A das string ends at the first NUL, so a body used without that +guard can be stored truncated under a hash that does not describe it. -**Client identity comes from the LAST `X-Forwarded-For` hop and is treated as data, never -parsed into behavior beyond the rate ceiling.** A proxy appends the peer it accepted; every -earlier hop is client-authored, so keying anything on one is a defect. Logging it is required. +**Never derive client identity from any `X-Forwarded-For` hop but the last, and never parse +that hop into behavior beyond the rate ceiling - log it, and otherwise treat it as data.** A +proxy appends the peer it accepted, so every earlier hop is client-authored. -**Operator routes (`POST /shutdown`, `/admin/*`) verify the transport peer is loopback.** No -header can carry that proof - a browser on any page can drive a cross-origin POST to a proxied -route. A new operator route without the check is a defect. +**An operator route (`POST /shutdown`, `/admin/*`) that does not verify the transport peer is +loopback is a defect.** No header can carry that proof - a browser on any page can drive a +cross-origin POST to a proxied route. -**Remote-builder routes (`/api/build/toolchain`, `/api/build/next`, `/api/build/result`) -authenticate by the bearer token from config, compared with `constant_time_equal`.** A route -that logs, echoes, or short-circuit-compares the token is a defect; an empty configured token -must disable the surface, never open it. +**A remote-builder route (`/api/build/toolchain`, `/api/build/next`, `/api/build/result`) that +does not authenticate by the config bearer token compared with `constant_time_equal` is a +defect, as is one that logs, echoes, or short-circuit-compares the token.** An empty configured +token disables the surface, never opens it. -**Every artifact-cache path is assembled only from components validated in -`build_artifacts`, and an upload reaches the served tree only through a stage directory -renamed into place after every file's sha256 verifies.** A served path built from request -data anywhere else, or a write landing directly in the served tree, is a defect. +**Never assemble an artifact-cache path from anything but components validated in +`build_artifacts`, and never land an upload in the served tree other than by renaming a stage +directory into place after every file's sha256 verifies.** -**An upload publishes exactly the file set its job's mode declares, checked here and not -taken on the builder's word.** A build runs the user's own compile-time code, so accepting -whatever set the builder sends would let a build put an extra file on this origin. +**A diff that publishes an upload checks its file set against the set its job's mode declares, +instead of taking the builder's word.** A build runs the user's own compile-time code, so +accepting whatever set the builder sends would let a build put an extra file on this origin. -**A handler does exactly three things - validate transport shape, make one store call, format -the response.** SQL, hashing, and policy (size, rate, listing) live in `samples_store.das`, -and HTTP never does - no `dashv` require there. +**Never put SQL, hashing, or policy (size, rate, listing) in a route handler, and never require +`dashv` in `samples_store.das` - a handler validates transport shape, makes one store call, and +formats the response, and those three live in `samples_store.das`.** -**No route enables CORS.** The middleware reflects the caller's `Origin` on every route at once, -which would make stored samples and the operator surface cross-origin readable. +**Never enable CORS on any route.** The middleware reflects the caller's `Origin` on every route +at once, which would make stored samples and the operator surface cross-origin readable. -**Responses that echo stored user input carry `X-Content-Type-Options: nosniff`.** +**Never send a response that echoes stored user input without `X-Content-Type-Options: +nosniff`.** -**No shell-out anywhere in the service.** +**Never shell out anywhere in the service.** -**No filesystem path derived from request data, and a manifest-supplied path is checked to stay -under its configured directory before it is read.** `path_join` discards the base when the right -side is absolute, so an unchecked manifest path reads any file the service user can - and the -importer publishes what it reads. +**Never derive a filesystem path from request data, and never read a manifest-supplied path +without checking that it stays under its configured directory.** `path_join` discards the base +when the right side is absolute, so an unchecked manifest path reads any file the service user +can - and the importer publishes what it reads. -**No `unsafe` in any route handler.** An `unsafe` elsewhere carries a reason comment on the -unsafe operation's own line; one without it is a defect. +**Never write `unsafe` in a route handler, and never write one elsewhere without a reason +comment on the unsafe operation's own line.** -**`POST /shutdown` and every admin operation stay unrouted in `caddy.snippet`.** Forwarding one -there, or adding an admin endpoint to the public route set, is a defect. +**Never forward `POST /shutdown` or any admin operation in `caddy.snippet`, and never add an +admin endpoint to the public route set.** -**Secrets (tokens, credentials) never appear in a log line, a response body, or an error +**Never put a secret - a token or a credential - in a log line, a response body, or an error message.** The config banner logs key names and provenance, never values marked secret. -**Every request but `GET /healthz` emits one structured line: method, path, status, duration, -client ip, bytes.** A route without it is a defect; the watchdog polls `/healthz` every few +**A route that does not emit one structured line - method, path, status, duration, client ip, +bytes - is a defect; `GET /healthz` emits none.** The watchdog polls `/healthz` every few seconds, and logging it would bury the signal. -**Every store mutation and every job/state transition emits a structured line.** A failure path -that can trigger without leaving a log line is a defect. +**A diff that adds a store mutation or a job/state transition also emits a structured line for +it.** A failure path that can trigger without leaving a log line is a defect. -**Startup logs the full effective config with per-key provenance before the first request is -served.** +**Never serve the first request before the startup banner has logged the full effective config +with per-key provenance.** -**Lifecycle-owned state is module-global; no collectable value lives in a `main`-loop local -across `maybe_collect_gc()`.** +**Never keep state the `init`/`update`/`shutdown` lifecycle owns anywhere but a module global, +and never leave a collectable value in a `main`-loop local across `maybe_collect_gc()`.** -**The store opens inside the thread that serves it.** A `SqlRunner` created on one thread and -used on another is a defect. +**Never create a `SqlRunner` on one thread and use it on another - the store opens inside the +thread that serves it.** -**Route callbacks are retained with `push`, never `emplace`.** +**Never retain a route callback with `emplace` - use `push`.** **A behavior change in a box-side file - `.das_package`, `watchdog.json`, `dasweb-playground.toml`, `deploy.sh`, `caddy.snippet` - lands with its note in the `README.md` Run section.** -**A file an operator edits on the box is preserved across upgrades**: shipped -`release_include_if_missing` and carried forward by `deploy.sh`. Shipping one with plain -`release_include`, or adding one `deploy.sh` does not carry, is a defect. +**Never ship a file an operator edits on the box with plain `release_include` - ship it with +`release_include_if_missing` and have `deploy.sh` carry it forward across upgrades.** -**Checked-in config holds development values.** The config file is discovered automatically -beside the module, so a production path in it makes an in-repo run open the live database. +**Never put a production value in the checked-in `dasweb-playground.toml` - it holds +development values.** The config file is discovered automatically beside the module, so a +production path in it makes an in-repo run open the live database. -**Migrations are append-only.** A diff that edits a shipped `[sql_migration]` body is a defect; -schema change means a new version. +**A diff that edits a shipped `[sql_migration]` body is a defect - a schema change adds a new +version instead.** **Placement - one file, one line: a diff keeps each file inside its line, and a new file adds its line here, with its tests, in the same change.** diff --git a/utils/internal/dasweb-verify/REVIEW.md b/utils/internal/dasweb-verify/REVIEW.md index f02d06da97..470995dde9 100644 --- a/utils/internal/dasweb-verify/REVIEW.md +++ b/utils/internal/dasweb-verify/REVIEW.md @@ -4,30 +4,29 @@ `README.md`. A `.mjs` file or `expectations.json`, wherever the diff puts it, answers to `browser/REVIEW.md`. -**Every core behavior has a dastest test in this directory** - `main.das` stays thin argv/exit -glue over tested pieces, so it needs none of its own. +**A diff that adds or changes a core behavior also adds a dastest test for it in this +directory, in the same change.** `main.das` is thin argv and exit-code glue over tested +pieces, so it needs no test of its own. -**`[test]` files live in this directory and require siblings by bare name** - never under the -global `tests/` tree, and never registered in any `CMakeLists.txt`. +**Never put a `[test]` file outside this directory, and never register one in any +`CMakeLists.txt` - keep it here and require its siblings by bare name.** -**A test that touches the filesystem uses `temp_directory`-rooted paths and deletes what it -creates.** A test writing into the repo tree is a defect. +**Never add a test that touches the filesystem outside a `temp_directory`-rooted path, or +that leaves behind what it creates.** -**Which samples exist comes only from the manifest the playground ships -(`web/examples/ui/samples/data.json`, repo root), never a hand-maintained copy.** A file -under this tool that lists, adds, or omits samples is a defect; a file carrying per-sample -data keyed by manifest name is not one. +**A file under this tool that lists, adds, or omits samples is a defect - take the sample list +only from the manifest the playground ships (`web/examples/ui/samples/data.json`, repo +root).** A file carrying per-sample data keyed by manifest name is not such a list. -**The generated-sample mapping in `verify_core.das` mirrors -`web/stage_playground_imgui_samples.cmake`:** a change to either lands with the other in the -same change. +**A diff that changes either the generated-sample mapping in `verify_core.das` or +`web/stage_playground_imgui_samples.cmake` also updates the other, in the same change.** **Weakening the fail-closed cases in `test_verify_core.das` is a defect** - a missing, -unparseable, or degenerate manifest or sample entry stays a named error returning no +unparseable, or degenerate manifest or sample entry still raises a named error and returns no entries, and `main.das` still maps that to a non-zero exit. -**Every failure line names the sample and carries the underlying message.** A failure a reader -cannot act on from the log alone is a defect. +**A diff that adds or changes a failure line puts the sample name and the underlying message +in that line.** A failure a reader cannot act on from the log alone is a defect. **Placement - one file, one line: a diff keeps each file inside its line, and a new file adds its line here, with its tests, in the same change.** diff --git a/utils/internal/dasweb-verify/browser/REVIEW.md b/utils/internal/dasweb-verify/browser/REVIEW.md index 21fe4e9ce0..ae5cc72686 100644 --- a/utils/internal/dasweb-verify/browser/REVIEW.md +++ b/utils/internal/dasweb-verify/browser/REVIEW.md @@ -3,29 +3,31 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `../README.md`. -**Every pure helper in this folder - data in, data out, no playwright, no DOM, no network - -has a `node:test` case in `protocol.test.mjs`, wherever the diff puts the helper.** -`runner.mjs` and `probe.mjs` hold only browser- and playwright-bound code, which the nightly -proves. +**A diff that leaves a pure helper in this folder - data in, data out, no playwright, no DOM, +no network - without a `node:test` case in `protocol.test.mjs` is a defect, wherever the diff +puts the helper.** `runner.mjs` and `probe.mjs` hold only browser- and playwright-bound code, +which the nightly CI run proves. -**Expectations live in `expectations.json`, never in sample sources.** A budget or pattern -spelled as a `// verify:` line in a sample would change its content hash and throw away the -build-cache entry the nightly depends on. +**Never put a budget or a pattern in a sample source - put it in `expectations.json` +instead.** A `// verify:` line in a sample changes the sample's content hash and throws away +the build-cache entry the nightly depends on. **Weakening the fail-closed checks in `protocol.test.mjs` is a defect** - an unknown sample stays a FAIL, manifest drift stays a WARN, and the manifest-coverage case keeps reading the shipped manifest `web/examples/ui/samples/data.json` (repo root). -**The verifier never cache-busts a build.** A cached FAILED build for identical content and -toolchain is a correct verdict; re-minting the hash to "get a fresh result" hides the bug. +**Never make the verifier change a build's hash to force a rebuild - report the cached result +instead.** A cached FAILED build for identical content and toolchain is a correct verdict, so +a fresh hash only hides the bug. -**The wasm build's artifact page - the HTML page a graphics or audio build returns, which the -playground embeds as a cross-origin frame - is never probed through that frame.** It is -cross-origin by design; verification opens the artifact URL as a top-level page instead. +**Never probe the wasm build's artifact page - the HTML page a graphics or audio build +returns - through the frame the playground embeds it in; open the artifact URL as a top-level +page instead.** That frame is cross-origin by design, so nothing inside it can be read. -**Exactly one glGetError poller per context.** `getError` clears the flag, so `probe.mjs` -polls only where the page has no watcher of its own (artifact pages, `pollGl: true`) - never -on the playground, where `site/playground/run-frame.html` (repo root) already polls. +**Never make `probe.mjs` poll `glGetError` on a page that has its own GL-error watcher - poll +only where the page has none (artifact pages, `pollGl: true`), never on the playground, where +`site/playground/run-frame.html` (repo root) already polls.** `getError` clears the flag, so +only one poller per context can see an error. **Placement - one file, one line: a diff keeps each file inside its line, and a new file adds its line here, with its tests, in the same change.** diff --git a/utils/lint/REVIEW.md b/utils/lint/REVIEW.md index caefee7f33..6297fe9f9e 100644 --- a/utils/lint/REVIEW.md +++ b/utils/lint/REVIEW.md @@ -3,11 +3,11 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `README.md`. -**lint ships as a prebuilt exe: it stays in `DAS_UTILS` and `DAS_UTILS_SHIPPED_EXES` -(`utils/CMakeLists.txt`)** - dropping either entry breaks the release bundle. +**A diff that removes `lint` from `DAS_UTILS` or `DAS_UTILS_SHIPPED_EXES` +(`utils/CMakeLists.txt`) is a defect** - dropping either entry breaks the release bundle. -**The set of rule ids `REVIEW.das` (beside this file) discovers never shrinks** - the gate -checks that every rule id emitted by a lint module has a fixture and a section in -`doc/source/reference/language/lint.rst` (repo root). A change that drops an id from the -scan - an edit to the gate, or a source change that removes the id's only scannable -spelling - silently retires that id's fixture-and-rst check. +**A diff that drops a rule id from the set `REVIEW.das` (beside this file) finds - by +editing the gate, or by removing the id's only spelling the gate can find in the source - +is a defect.** The gate checks that every rule id a lint module emits has a fixture and a +section in `doc/source/reference/language/lint.rst` (repo root), so a dropped id retires +that check with no warning. diff --git a/utils/lsp/REVIEW.md b/utils/lsp/REVIEW.md index 01e13d64ac..965f9d0bf5 100644 --- a/utils/lsp/REVIEW.md +++ b/utils/lsp/REVIEW.md @@ -3,6 +3,6 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `README.md`. -**lsp never enters `DAS_UTILS` or `DAS_UTILS_SHIPPED_EXES` (`utils/CMakeLists.txt`) - adding it -to either is a defect.** Development runs it through the python keep-alive supervisor, so an -exe form would never be dogfooded before shipping. +**Never add `lsp` to `DAS_UTILS` or `DAS_UTILS_SHIPPED_EXES` in `utils/CMakeLists.txt` - clients +start the server through `lsp_supervisor.py` instead.** Development always starts it through that +python script, so an exe form would ship without anyone having used it. diff --git a/utils/mcp/REVIEW.md b/utils/mcp/REVIEW.md index 68ec21e379..6ab030c6eb 100644 --- a/utils/mcp/REVIEW.md +++ b/utils/mcp/REVIEW.md @@ -3,6 +3,6 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `README.md`. -**mcp never enters `DAS_UTILS` or `DAS_UTILS_SHIPPED_EXES` (`utils/CMakeLists.txt`) - adding it -to either is a defect.** Development runs it through the python keep-alive supervisor, so an -exe form would never be dogfooded before shipping. +**Never add `mcp` to `DAS_UTILS` or `DAS_UTILS_SHIPPED_EXES` (`utils/CMakeLists.txt`).** +Development runs the server through the python keep-alive supervisor, so an exe form would +never be used in development before it ships. diff --git a/web/examples/ui/REVIEW.md b/web/examples/ui/REVIEW.md index 07a0c521b4..736305a72f 100644 --- a/web/examples/ui/REVIEW.md +++ b/web/examples/ui/REVIEW.md @@ -5,8 +5,8 @@ **`src/` and `samples/` are the deployed playground's UI source - every rule of `site/REVIEW.md` binds a change under them as if the files lived under `site/playground/`.** -The deploy vendors them into the site (`pages.yml`), so `site/playground/` never shows the -change and the walk from here is the only way that checklist fires. +The deploy copies them into the site (`pages.yml`), so `site/playground/` never shows the +change and only this line makes that checklist apply. **A file of the boulder-dash sample - its `.das` sources, its dastest suites, its `ARCHITECTURE.md` - answers to `samples/examples/boulder-dash/REVIEW.md` wherever the diff --git a/web/examples/ui/samples/examples/boulder-dash/REVIEW.md b/web/examples/ui/samples/examples/boulder-dash/REVIEW.md index 13c9694ab0..f54459d19d 100644 --- a/web/examples/ui/samples/examples/boulder-dash/REVIEW.md +++ b/web/examples/ui/samples/examples/boulder-dash/REVIEW.md @@ -3,32 +3,33 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `ARCHITECTURE.md`. -**Everything that reads a tick's outgoing state sits between `cave_tick` and the next tick** - - scoring from `collected`, `play_tick_sfx`, and `cave_consume_input` run immediately after -the tick, in that order. The per-tick event fields are cleared at the top of every tick, and -consuming input before the tick applies the input one tick late. +**Never read the state a tick produced anywhere but between that `cave_tick` call and the +next tick - scoring from `collected`, `play_tick_sfx`, and `cave_consume_input` run +immediately after the tick, in that order.** The per-tick event fields are cleared at the top +of every tick, and consuming input before the tick applies the input one tick late. -**`skip_draw` is sized and filled only in `mark_movers`, and `mark_movers` runs before -`draw_static_tiles` on every frame** - `draw_static_tiles` indexes the mask without a -length check. +**Never size or fill `skip_draw` anywhere but in `mark_movers`, and never call +`draw_static_tiles` in a frame that has not already called `mark_movers`** - +`draw_static_tiles` indexes the mask without a length check. -**A `Tile` member added or reordered updates the `tile_color` and `mat_for` arms in -`main.das` in the same change** - a tile without those arms renders floor-colored, i.e. -invisibly. +**A diff that adds or reorders a `Tile` member also updates the `tile_color` and `mat_for` +arms in `main.das`, in the same change** - a tile with no arm renders floor-colored, so it +is invisible. -**`tile_color` and `mat_for` classify every tile identically** - a tile matched by one -family test and not the other draws with one family's color and another's material. +**A diff that changes a tile family test in `tile_color` or `mat_for` makes the matching +change to the other, in the same change** - a tile matched by one family test and not the +other draws with one family's color and another's material. **Weakening or deleting a check in `test_cave_rules.das` or `test_sfx_gen.das` is a -defect** - those two suites are the only enforcement of the couplings this checklist -names, and relaxing one silently un-enforces a rule above. +defect** - those two suites are the only enforcement of the couplings this checklist names. -**When one tick moves two objects through a shared cell, the move whose SOURCE is the -shared cell runs first** - `move_tile` clears source before writing destination, so the -reversed order erases the object written there (`try_push`: boulder first, then player). +**A diff that adds or reorders two moves that share a cell in one tick puts the move whose +SOURCE is the shared cell first** - `move_tile` clears the source cell before writing the +destination, so the reversed order erases the object written there (`try_push`: boulder +first, then player). -**A new `Cave` field that records something that happened during a tick gains an -assignment in `reset_tick_events` and an arm in `play_tick_sfx` in the same change** - +**A diff that adds a `Cave` field recording something that happened during a tick also adds +an assignment in `reset_tick_events` and an arm in `play_tick_sfx`, in the same change** - missing the first latches the event across ticks; missing the second makes it silent. **A diff that adds, removes, or reorders a random draw on `generate_cave`'s path - From c52c584ee0708f426ea4d14e284d899bfa3bd3a7 Mon Sep 17 00:00:00 2001 From: Boris Batkin <bbatkin@gmail.com> Date: Thu, 27 Aug 2026 01:09:37 -0700 Subject: [PATCH 07/15] janitor sweep rulings: 25 queued items resolved and applied In-place term definitions, arch-doc moves (statements land in their present-tense home), verdict/head alignment, APIs named where rules gestured, one gate-covered rule collapsed to its weakening ban. The two daspkg forks stay held for a human ruling. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014fKJGuUL8tP58Y6NvoVeSn --- daslib/ARCHITECTURE.md | 7 +++++++ daslib/REVIEW_LINQ.md | 9 ++++----- modules/REVIEW_SHADER_EMITTERS.md | 7 +++++-- modules/dasLLAMA/ARCHITECTURE.md | 2 ++ modules/dasLLAMA/REVIEW.md | 2 -- modules/dasLLAMA/REVIEW_GPU.md | 10 +++++----- modules/dasLLAMA/REVIEW_VISION.md | 7 +++++-- modules/dasLLAMA/performance/REVIEW.md | 4 ++-- modules/dasLLAMA/tests/REVIEW.md | 3 +-- nano/REVIEW.md | 10 ++++------ utils/dasllama-server/REVIEW.md | 3 ++- utils/daspkg/REVIEW.md | 5 +++-- utils/detect-dupe/REVIEW.md | 4 ++-- utils/internal/dasweb-buildd/README.md | 4 ++++ utils/internal/dasweb-buildd/REVIEW.md | 18 +++++++++--------- utils/internal/dasweb-playground/REVIEW.md | 2 +- utils/mcp/REVIEW.md | 3 ++- web/examples/ui/REVIEW.md | 7 +++---- 18 files changed, 61 insertions(+), 46 deletions(-) diff --git a/daslib/ARCHITECTURE.md b/daslib/ARCHITECTURE.md index d55132a129..a6a980e957 100644 --- a/daslib/ARCHITECTURE.md +++ b/daslib/ARCHITECTURE.md @@ -444,6 +444,9 @@ an entry lands here only when no name, shape, or test can carry it. (`from x in [c.id]`, a function result) is safe because daslang heap-allocates arrays and does not finalize them at scope exit - the backing storage outlives the borrow and is reclaimed by GC. +- **A `FromInMacro` reject returns null so infer can stabilize.** Errors clear on every pass, + so the not-yet-inferred-source arm's error survives only when the source never infers at + all - that is what makes that arm a deferral rather than a final verdict. - **`from_in` identifies typed sources by matching module + type NAME as strings**, not by requiring the provider modules: `sqlite_boost::SqlRunner` -> `select_from`, pugixml `xml_node` -> `from_xml_node`, `json::JsonValue?` -> `from_json` - linq_das stays @@ -463,6 +466,10 @@ an entry lands here only when no name, shape, or test can carry it. - **`hoist_prelude` hand-builds a flat block on purpose.** The materializer prelude - the hoisted default-row declaration on the field-prune path - must be a SIBLING of the walk; a `$b(prelude)` splice nests it in its own scope, where the per-element reads cannot see it. +- **One `at` can reach two materializers.** A `group_by` chain closed by + `first`/`order`/`distinct`/`take` emits through both the array-walk materializer and the + handle materializer at the same source location, so `qn` - which keys on (prefix, at) - + keeps the two sites' binds apart only while each site carries its own prefix. - **An `ExprFor` the typer has not re-inferred carries an EMPTY `iteratorVariables`.** A decs bridge feeding a no-chain terminator (`from_decs_template(...).count()`) has no chain op to force the second inference pass, so the iterator names are recovered from the push diff --git a/daslib/REVIEW_LINQ.md b/daslib/REVIEW_LINQ.md index 31e3fa09c3..89bdaea5ea 100644 --- a/daslib/REVIEW_LINQ.md +++ b/daslib/REVIEW_LINQ.md @@ -74,11 +74,10 @@ a projected adapter is a defect - skip the row instead.** The row's emit assumes element, so running it on the un-projected adapter orders or dedups raw rows - a wrong result, not a missed splice. -**The call order in `register_all_linq_fold_rows` IS pattern priority.** Rows land in -`splice_patterns` in call order and the walker takes the first match, so reordering the -`build_*_rows()` calls changes which arm claims a chain. Separate modules run their -`[_macro]`s in separate macro contexts that cannot coordinate - that single registrar is the -only place the order exists. +**A diff that reorders the `build_*_rows()` calls in `register_all_linq_fold_rows` changes +which arm claims a chain - review it as a pattern-priority change, never as a cleanup.** Rows +land in `splice_patterns` in call order, the walker takes the first match, and that registrar +is the only place the order exists. **A sql_linq column-ref arm that accepts a `$e(recv).$f(field)` qmatch without first proving `recv is ExprVar` is a defect.** A nested receiver otherwise matches under a foreign name and diff --git a/modules/REVIEW_SHADER_EMITTERS.md b/modules/REVIEW_SHADER_EMITTERS.md index 9a16761b3d..80e4b0baf3 100644 --- a/modules/REVIEW_SHADER_EMITTERS.md +++ b/modules/REVIEW_SHADER_EMITTERS.md @@ -30,10 +30,13 @@ that module applies this file together with the module's own list. cannot have one, the same diff records that in the module's `MASTERPLAN.md`. - **Never check a claim about emitted shape against the das source - check it in the emitted - words or text.** + words or text.** Emitted shape is the structure of the emitted kernel: its signature, its + parameter attributes, and its statement forms. - **A diff that adds a kernel-model capability to one emitter adds it to the other, or records - the asymmetry in the shared ledger (`modules/dasMetal/MASTERPLAN.md`).** + the asymmetry in the shared ledger (`modules/dasMetal/MASTERPLAN.md`).** A kernel-model + capability is a way of authoring or lowering kernels: class members, multiple kernels per + class, method flattening, tensor-GEMM builtins. - **An emitter diff that uses a `daslib/shader_lingua_franca` declaration this emitter does not handle ships that emitter's lowering of the declaration, in the same change.** The diff may diff --git a/modules/dasLLAMA/ARCHITECTURE.md b/modules/dasLLAMA/ARCHITECTURE.md index a81a99782e..b40bef5e50 100644 --- a/modules/dasLLAMA/ARCHITECTURE.md +++ b/modules/dasLLAMA/ARCHITECTURE.md @@ -31,6 +31,8 @@ thing live instead. Files with no contested edge get one short line. ## 1. File charters +When placement charters disagree on one function, the sec.1 charter line decides. + Every file under `dasllama/` appears here. Two carry generated CONTENT rather than being generated themselves: `dasllama_env.das` holds the `[EnvConfig]` knob declarations (`ENVIRONMENT.md` is generated FROM it by `harness/gen_env_doc.das`), and `dasllama_unicode.das` is hand-written diff --git a/modules/dasLLAMA/REVIEW.md b/modules/dasLLAMA/REVIEW.md index 77356d9871..6867d5ce84 100644 --- a/modules/dasLLAMA/REVIEW.md +++ b/modules/dasLLAMA/REVIEW.md @@ -273,8 +273,6 @@ file is the checklist's own. **A disk-order -> compute-order transform lands per scope: kernel-layout in `dasllama/dasllama_repack.das`, load-scope in `dasllama/dasllama_layout.das`.** -**When placement rules disagree on one function, `ARCHITECTURE.md` sec.1's charter decides.** - **A CPU KV-cache store, read, score dot, or V-accumulate lands in `dasllama/dasllama_kv_codec.das`, its format family kept whole.** GPU twins land in their backend kernel file. diff --git a/modules/dasLLAMA/REVIEW_GPU.md b/modules/dasLLAMA/REVIEW_GPU.md index 922fc77743..e173d7af9f 100644 --- a/modules/dasLLAMA/REVIEW_GPU.md +++ b/modules/dasLLAMA/REVIEW_GPU.md @@ -56,8 +56,8 @@ One shared scratch serializes the whole chain through its write-after-read hazar work size, in the same change.** The gate's threshold is measured at both ends of the size ladder. The small-work regression hides behind the big-work win. -**A shape claim is settled at the one site that is authoritative for that kind of constant, -never by tracing the das that computes the value.** An in-body tile constant is confirmed +**A diff that changes a tile, grid, threadgroup, or uniform constant shows the value at that +constant's authoritative site, in the same change.** An in-body tile constant is confirmed literal in the generated `*_msl` global or the SPIR-V dump. A grid or threadgroup constant is read off the class's `[metal_dispatch]` / `[vk_dispatch]` `grid=`/`tg=` spec, whose `"n/c"` form is a CEIL-divide; the spec alone decides, and no builder read is needed. A uniform's @@ -161,9 +161,9 @@ class one of them dispatches, that class's builder, the servability gates is `harness/parity.das`, or the in-suite instruments `tests/test_metal_decode_parity.das` / `tests/test_metal_prefill_parity.das` through `tests/run.das`. -**A `harness/parity.das` run arms its backend: the Metal arm is `--ngl`; the Vulkan arm is -`DASLLAMA_GPU=1`, never `--ngl`, and its log shows `resident driver armed`.** The Vulkan -driver declines codec-mismatched sessions silently. +**Parity evidence counts only when its backend was armed: the Metal arm ran with `--ngl`; the +Vulkan arm ran with `DASLLAMA_GPU=1` - never `--ngl` - and its log shows `resident driver +armed`.** The Vulkan driver declines codec-mismatched sessions silently. **A change to `dasllama/dasllama_metal_tower.das`, to the `AttnArgs` kargs struct, to any kernel class the tower dispatches or builder the tower borrows, or to state the whole diff --git a/modules/dasLLAMA/REVIEW_VISION.md b/modules/dasLLAMA/REVIEW_VISION.md index 2d1d803757..0058575957 100644 --- a/modules/dasLLAMA/REVIEW_VISION.md +++ b/modules/dasLLAMA/REVIEW_VISION.md @@ -36,8 +36,11 @@ is a defect.** **A family file that holds a second copy of code two tower families both need - compute, stage/read, or load-orchestration code that names no family type - is a defect; that code -lives in `dasllama/dasllama_tower.das`, the encoder-tower home.** Per-family serving state - a -family's exported runtime setter and the module global it writes - stays in the family file. +lives in `dasllama/dasllama_tower.das`, the encoder-tower home.** + +**A diff that puts a family's exported runtime setter, or the module global it writes, into +`dasllama/dasllama_tower.das` is a defect** - per-family serving state stays in the family +file. **Splitting a media splice's rows across more than one `forward_prefill_embd` call, or letting a driver chunk them by row, is a defect** - the span bounds are counted from the start of the diff --git a/modules/dasLLAMA/performance/REVIEW.md b/modules/dasLLAMA/performance/REVIEW.md index 59aca89004..ffb6312466 100644 --- a/modules/dasLLAMA/performance/REVIEW.md +++ b/modules/dasLLAMA/performance/REVIEW.md @@ -8,8 +8,8 @@ validate through `../dasllama/dasllama_exchange_schema.das` instead.** The engin `dasllama/` require beyond the lint macro module) is `REVIEW.das`'s to enforce; weakening that gate is a defect. -**A diff that adds a row to `records/` whose `hardware.remote_desktop` is `parsec`, or a -sidecar whose `provenance.noise` is not `ok`, is a defect - re-mint on a box with no +**A diff that adds a row to `records/` whose `hardware.remote_desktop` is anything but `off`, +or a sidecar whose `provenance.noise` is not `ok`, is a defect - re-mint on a box with no remote-desktop session.** **A diff that adds a commit stamp to `records/` naming a commit the branch under review cannot diff --git a/modules/dasLLAMA/tests/REVIEW.md b/modules/dasLLAMA/tests/REVIEW.md index 1daaf95f4c..95c4e00ec1 100644 --- a/modules/dasLLAMA/tests/REVIEW.md +++ b/modules/dasLLAMA/tests/REVIEW.md @@ -12,8 +12,7 @@ corpus it reads, or a name it asserts on; a comment-only edit reaches none. defect, when its cells still assert what they claim under `DASLLAMA_CPU_PREFILL=1`.** The model suites are every suite but `model-free`, and that variable is what the runner arms for `model-free`. The listing lands in the same change that adds the file, and the file skips -honestly when its models are absent. The per-PR gate then runs it wherever the models are -stocked. +honestly when its models are absent. **Listing a test file that `DASLLAMA_CPU_PREFILL=1` disarms in any `run.das` suite is a defect, and so is leaving that fact out of the file's header.** `DASLLAMA_CPU_PREFILL=1` is what the diff --git a/nano/REVIEW.md b/nano/REVIEW.md index 8b76b66b38..d55d68c8c0 100644 --- a/nano/REVIEW.md +++ b/nano/REVIEW.md @@ -8,17 +8,15 @@ runtime's tree instead, moving its runtime half into a file of its own.** An edi fork that drifts silently. **Never use a name from `daScript/ast/ast.h` beyond `Annotation`, `TypeAnnotation` and -`Module::name` in a source under `src/` that nano compiles.** Any other name compiles here and -means something different than it does in the full runtime. +`Module::name` in a source nano compiles - its own under `src/` here and the reused ones under +`src/` (repo root).** Any other name compiles here and means something different than it does +in the full runtime. **Never add a stub to `src/nano_stubs.cpp` that returns a default and continues - implement the behaviour or stop the program instead.** On a target with no debugger attached, a default return turns a missing feature into a wrong answer. -**Never link `libDaScriptNano` from a directory whose `CMakeLists.txt` lacks -`set_property(DIRECTORY PROPERTY INCLUDE_DIRECTORIES "")` - add that line there instead.** -CMake searches a directory's `include_directories()` before any target's own, so without the -line the target compiles against the full runtime's headers while linking nano's library. +**Weakening the include-clear check in `REVIEW.das` (beside this file) is a defect.** **Never add a name to `include/daScript/simulate/simulate.h` unless a source nano compiles unchanged from the full runtime, or generated code, already refers to it.** This header is a diff --git a/utils/dasllama-server/REVIEW.md b/utils/dasllama-server/REVIEW.md index caee42c2ca..4136e9cbd8 100644 --- a/utils/dasllama-server/REVIEW.md +++ b/utils/dasllama-server/REVIEW.md @@ -4,7 +4,8 @@ doc: `README.md`. **A diff to a file that requires a `dasllama/*` module applies `modules/dasLLAMA/REVIEW.md` -(repo root) too** - its gate polices the family seams here. +(repo root) too** - the server reaches only public dasLLAMA facade verbs, and a diff here can +break that facade. **Never write or edit a fixture under `tests/fixtures/` by hand - capture the bytes of a real server run and normalize the machine-local paths instead.** `tests/fixtures/README.md` gives diff --git a/utils/daspkg/REVIEW.md b/utils/daspkg/REVIEW.md index 69a3c72dcd..6107c41195 100644 --- a/utils/daspkg/REVIEW.md +++ b/utils/daspkg/REVIEW.md @@ -29,8 +29,9 @@ tune sidecar is the `<bundle>.tune.json` file of measured kernel choices the exe time. **A diff that lets a release path other than `--quick` inherit a prior sidecar, or lets -`--quick` accept an incomplete one, is a defect** - incomplete meaning missing any scope key the -exe's deps JSON reports, the same completeness the release build itself checks. +`--quick` accept an incomplete one, is a defect** - incomplete means missing any scope key - a +`tune_scopes` entry the exe's deps JSON reports - the same completeness the release build +itself checks. **A release path that overwrites or deletes a `release_include_if_missing` file, on any platform, is a defect.** diff --git a/utils/detect-dupe/REVIEW.md b/utils/detect-dupe/REVIEW.md index f1c32f196a..16310d2e7f 100644 --- a/utils/detect-dupe/REVIEW.md +++ b/utils/detect-dupe/REVIEW.md @@ -4,5 +4,5 @@ `README.md`. **A diff removing detect-dupe from `DAS_UTILS_SHIPPED_EXES` records the removal decision in -this file, per `utils/REVIEW.md`.** detect-dupe ships as a prebuilt exe, and `utils/REVIEW.das` -machine-checks the membership itself. +this file.** detect-dupe ships as a prebuilt exe, and `utils/REVIEW.das` machine-checks the +membership itself. diff --git a/utils/internal/dasweb-buildd/README.md b/utils/internal/dasweb-buildd/README.md index 77eb99cbff..c8a3dfa18b 100644 --- a/utils/internal/dasweb-buildd/README.md +++ b/utils/internal/dasweb-buildd/README.md @@ -165,3 +165,7 @@ the symlinked-artifact refusal, toolchain derivation), client (claim/announce/re the stub playground on reserved port 19014, including the multipart round trip verified byte-exact on the receiving side). CI runs the suites in `extended_checks.yml` ("Test dasweb-buildd"). + +The end-to-end checkpoint covers what no in-dir suite reaches: one job announced, claimed, +built in the sandbox and uploaded, run against the real playground on the deployed boxes. It +is what proves the whole poll -> build -> upload loop. diff --git a/utils/internal/dasweb-buildd/REVIEW.md b/utils/internal/dasweb-buildd/REVIEW.md index 2b2115301c..57784b20f5 100644 --- a/utils/internal/dasweb-buildd/REVIEW.md +++ b/utils/internal/dasweb-buildd/REVIEW.md @@ -6,8 +6,7 @@ **A diff that adds or changes behavior in `buildd_config.das`, `buildd_core.das`, or `buildd_client.das` ships a dastest test in this directory, in the same change.** A diff that puts behavior beyond glue and orchestration over those modules into `main.das` or -`buildd_service.das` is a defect. The whole loop is proven by the end-to-end checkpoint on the -real boxes. +`buildd_service.das` is a defect. **Never put a `[test]` file under the global `tests/` tree, and never register one in a `CMakeLists.txt` - a `[test]` file for this directory lives here and requires its siblings by @@ -32,8 +31,8 @@ is loopback.** No header can carry that proof. banner reports set or unset and the provenance instead.** **A diff that assembles a filesystem path from a request- or build-derived name - a bundle -source or a build output alike - without passing that name through this directory's name -validation first is a defect.** +source or a build output alike - without passing that name through `is_valid_source_filename`, +`is_valid_asset_path`, or `validate_asset_paths` (`buildd_core.das`) first is a defect.** **A diff that lets a build publish a file its mode does not declare by name - through a suffix filter, a glob, or any rule that widens the build's own output set - is a defect.** The build @@ -72,15 +71,16 @@ is a defect.** Leaving the claim for the stale-requeue sweep is not a resolution `.das_package`, `watchdog.json`, `dasweb-buildd.toml` - also updates the matching `README.md` section (The sandbox / The toolchain-bump protocol / Run), in the same change.** -**Never let a toolchain roll move the worktree without rebuilding both the cross-compile host -and the runtime archive.** +**Never let a toolchain roll (`roll_toolchain.sh`) move the worktree without rebuilding both +the cross-compile host and the runtime archive.** **A diff that changes `Containerfile` also bumps the image tag in `run_build.sh` (`IMAGE=`), in the same change.** -**A diff that changes the wasm-archive step's build command or archive list, or -`modules/dasImgui/.das_package`, also changes the other to match, in the same change - the two -differ only in `liblibDasModuleClipboard.a`, which the wasm-archive list leaves out.** +**A diff that changes the wasm-archive step's build command or archive list in +`roll_toolchain.sh`, or `modules/dasImgui/.das_package`, also changes the other to match, in +the same change - the two differ only in `liblibDasModuleClipboard.a`, which the wasm-archive +list leaves out.** **Never ship a file an operator edits on the box with plain `release_include` - use `release_include_if_missing`, so an upgrade keeps the operator's edits.** diff --git a/utils/internal/dasweb-playground/REVIEW.md b/utils/internal/dasweb-playground/REVIEW.md index 51c1fedf8c..6844bb8ca3 100644 --- a/utils/internal/dasweb-playground/REVIEW.md +++ b/utils/internal/dasweb-playground/REVIEW.md @@ -79,7 +79,7 @@ comment on the unsafe operation's own line.** admin endpoint to the public route set.** **Never put a secret - a token or a credential - in a log line, a response body, or an error -message.** The config banner logs key names and provenance, never values marked secret. +message.** **A route that does not emit one structured line - method, path, status, duration, client ip, bytes - is a defect; `GET /healthz` emits none.** The watchdog polls `/healthz` every few diff --git a/utils/mcp/REVIEW.md b/utils/mcp/REVIEW.md index 6ab030c6eb..d5115a37a7 100644 --- a/utils/mcp/REVIEW.md +++ b/utils/mcp/REVIEW.md @@ -3,6 +3,7 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `README.md`. -**Never add `mcp` to `DAS_UTILS` or `DAS_UTILS_SHIPPED_EXES` (`utils/CMakeLists.txt`).** +**Never add `mcp` to `DAS_UTILS` or `DAS_UTILS_SHIPPED_EXES` (`utils/CMakeLists.txt`) - run it +interpreted through `.mcp.json` instead.** Development runs the server through the python keep-alive supervisor, so an exe form would never be used in development before it ships. diff --git a/web/examples/ui/REVIEW.md b/web/examples/ui/REVIEW.md index 736305a72f..a2ff4b9bd0 100644 --- a/web/examples/ui/REVIEW.md +++ b/web/examples/ui/REVIEW.md @@ -3,10 +3,9 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `site/README.md`. -**`src/` and `samples/` are the deployed playground's UI source - every rule of -`site/REVIEW.md` binds a change under them as if the files lived under `site/playground/`.** -The deploy copies them into the site (`pages.yml`), so `site/playground/` never shows the -change and only this line makes that checklist apply. +**Every rule of `site/REVIEW.md` binds a change under `src/` and `samples/` as if the files +lived under `site/playground/`.** The deploy copies them into the site (`pages.yml`), so +`site/playground/` never shows the change and only this line makes that checklist apply. **A file of the boulder-dash sample - its `.das` sources, its dastest suites, its `ARCHITECTURE.md` - answers to `samples/examples/boulder-dash/REVIEW.md` wherever the From 86ba1e34295352c0dfbccf50fe363cb254e26552 Mon Sep 17 00:00:00 2001 From: Boris Batkin <bbatkin@gmail.com> Date: Thu, 27 Aug 2026 01:26:05 -0700 Subject: [PATCH 08/15] sweep human queue: five flashlight items ruled GPU shape-evidence duty and the daspkg mint deletion stand; the integration-suite trigger spelled as changed-lines-inside-a-git-running -function; the vision serving-state ban widened to anywhere-but-the- family-file; emitted shape widened to include stamped shape values. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014fKJGuUL8tP58Y6NvoVeSn --- modules/REVIEW_SHADER_EMITTERS.md | 5 +++-- modules/dasLLAMA/REVIEW_VISION.md | 5 ++--- utils/daspkg/REVIEW.md | 6 +++--- 3 files changed, 8 insertions(+), 8 deletions(-) diff --git a/modules/REVIEW_SHADER_EMITTERS.md b/modules/REVIEW_SHADER_EMITTERS.md index 80e4b0baf3..b8cddaed28 100644 --- a/modules/REVIEW_SHADER_EMITTERS.md +++ b/modules/REVIEW_SHADER_EMITTERS.md @@ -30,8 +30,9 @@ that module applies this file together with the module's own list. cannot have one, the same diff records that in the module's `MASTERPLAN.md`. - **Never check a claim about emitted shape against the das source - check it in the emitted - words or text.** Emitted shape is the structure of the emitted kernel: its signature, its - parameter attributes, and its statement forms. + words or text.** Emitted shape is the structure of the emitted kernel - its signature, its + parameter attributes, its statement forms - and its stamped shape values (tile, grid, + threadgroup sizes). - **A diff that adds a kernel-model capability to one emitter adds it to the other, or records the asymmetry in the shared ledger (`modules/dasMetal/MASTERPLAN.md`).** A kernel-model diff --git a/modules/dasLLAMA/REVIEW_VISION.md b/modules/dasLLAMA/REVIEW_VISION.md index 0058575957..e6b0685bb4 100644 --- a/modules/dasLLAMA/REVIEW_VISION.md +++ b/modules/dasLLAMA/REVIEW_VISION.md @@ -38,9 +38,8 @@ is a defect.** stage/read, or load-orchestration code that names no family type - is a defect; that code lives in `dasllama/dasllama_tower.das`, the encoder-tower home.** -**A diff that puts a family's exported runtime setter, or the module global it writes, into -`dasllama/dasllama_tower.das` is a defect** - per-family serving state stays in the family -file. +**A diff that puts a family's exported runtime setter, or the module global it writes, +anywhere but that family's file is a defect.** **Splitting a media splice's rows across more than one `forward_prefill_embd` call, or letting a driver chunk them by row, is a defect** - the span bounds are counted from the start of the diff --git a/utils/daspkg/REVIEW.md b/utils/daspkg/REVIEW.md index 6107c41195..859a3fa2d2 100644 --- a/utils/daspkg/REVIEW.md +++ b/utils/daspkg/REVIEW.md @@ -7,9 +7,9 @@ suite is `bin/daslang dastest/dastest.das -- --test utils/daspkg/test_daspkg.das` - fast, no network, interpreted. -**A diff whose hunks in `commands.das`, `index.das`, or `utils.das` reach a function that runs -a git command also runs the integration suite, in the same change** - a behavior-preserving -refactor still counts. The integration suite is `bin/daslang dastest/dastest.das -- --test +**A diff whose changed lines sit inside a function that runs a git command - in +`commands.das`, `index.das`, or `utils.das` - also runs the integration suite, in the same +change** - a behavior-preserving refactor still counts. The integration suite is `bin/daslang dastest/dastest.das -- --test utils/daspkg/test_daspkg_git.das`, and it needs network (the `borisbat/daspkg-test-*` fixture repos). From 55c73ada3a13293482909148b757c360df9b7e27 Mon Sep 17 00:00:00 2001 From: Boris Batkin <bbatkin@gmail.com> Date: Thu, 27 Aug 2026 01:43:24 -0700 Subject: [PATCH 09/15] document grammar linked; MASTERPLANs migrated to ARCHITECTURE.md REVIEW_COMMON names ARCHITECTURE_COMMON as the statements' contract; CLAUDE.md carries the three-mood document system and the write-time rule (read the folder's architecture doc before writing code there); the dragon binds ARCHITECTURE.md to its contract. dasSpirv and dasMetal MASTERPLANs split: implementation logs and plan sections archived verbatim to history/<area>/MASTERPLAN_LOG.md, the present-tense half survives as each module's ARCHITECTURE.md with facts corrected to the state the logs record; every operative reference repo-wide re-pointed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014fKJGuUL8tP58Y6NvoVeSn --- .claude/agents/dragon.md | 10 +- CLAUDE.md | 15 + REVIEW_COMMON.md | 3 +- history/README.md | 9 +- .../dasMetal/MASTERPLAN_LOG.md | 261 +--------------- .../dasSpirv/MASTERPLAN_LOG.md | 117 +------- modules/REVIEW_SHADER_EMITTERS.md | 6 +- modules/dasLLAMA/followup_vulkan.md | 2 +- modules/dasMetal/ARCHITECTURE.md | 279 ++++++++++++++++++ modules/dasMetal/REVIEW.md | 2 +- modules/dasSpirv/ARCHITECTURE.md | 137 +++++++++ modules/dasSpirv/REVIEW.md | 2 +- modules/dasVulkan/ROADMAP.md | 2 +- plans/msl-flatten.md | 4 +- skills/internal/doc_archiving.md | 4 +- skills/review_md.md | 13 +- tests/metal/REVIEW.md | 2 +- tests/metal/test_metal_mul_ab.das | 2 +- tests/msl/REVIEW.md | 2 +- 19 files changed, 482 insertions(+), 390 deletions(-) rename modules/dasMetal/MASTERPLAN.md => history/dasMetal/MASTERPLAN_LOG.md (78%) rename modules/dasSpirv/MASTERPLAN.md => history/dasSpirv/MASTERPLAN_LOG.md (94%) create mode 100644 modules/dasMetal/ARCHITECTURE.md create mode 100644 modules/dasSpirv/ARCHITECTURE.md diff --git a/.claude/agents/dragon.md b/.claude/agents/dragon.md index f23ccd65ce..c9ca8e9e5e 100644 --- a/.claude/agents/dragon.md +++ b/.claude/agents/dragon.md @@ -40,10 +40,12 @@ You audit the document, not the code it governs. Document types and their contra - `skills/*.md` - bound by `skills/internal/writing_skills.md` (audience decision, shipping gate). - `CLAUDE.md` and per-module rule files - bound by the rules-carry-rules-not-history clause: current contract only, no incident anecdotes, no PR numbers, no dated entries. -- Ledger and architecture documents (`ARCHITECTURE.md`, `PROFILE.md`, `followup_*.md`, - `THINKING.md`, `PERF_LEDGER.md`, `MASTERPLAN.md`) are ALLOWED to accrete - growth there is - the document doing its job. If handed one, say it is a ledger and audit only factual-claim - staleness, not size or history. +- `ARCHITECTURE.md` (any folder) - bound by `ARCHITECTURE_COMMON.md` (repo root): present + tense, the mood test (a must/never sentence binding a diff belongs in the folder's + REVIEW.md; a dated or past-tense passage belongs in /history), numbered citable sections. +- Ledger documents (`PROFILE.md`, `followup_*.md`, `THINKING.md`, `PERF_LEDGER.md`) are + ALLOWED to accrete - growth there is the document doing its job. If handed one, say it is + a ledger and audit only factual-claim staleness, not size or history. ## Verdicts diff --git a/CLAUDE.md b/CLAUDE.md index 575b6e7f98..75f7cd4689 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -18,6 +18,21 @@ Per-platform generator commands, build flags, AOT debugging, exit-code meanings, Delete files created for diagnostics, staging, and one-off tests before handing work back - temporary release trees, formatter backups, extracted audio, generated `__pycache__`, ad-hoc stdout/stderr logs. Keep intentional application logs, databases, configs, rollback bundles, and user-owned artifacts. In deployed application directories, do not leave unrelated diagnostic files beside the program. +## The document system + +Three document types split one grammar, each with a repo-root contract: + +- **`REVIEW.md`** (per folder) - bans and duties binding a diff, applied at REVIEW time via + the folder walk. Contract: `REVIEW_COMMON.md`. +- **`ARCHITECTURE.md`** (per folder) - present-tense statements: how the system is and why. + Contract: `ARCHITECTURE_COMMON.md`. **Read the folder's architecture doc BEFORE writing + code there** - charters say which file owns what, mechanisms say why shapes win, and the + exception ledger says what is already ruled. Facts formerly carried by comments live here. +- **`/history`** - past tense: what happened. Nothing current cites it. + +The mood test routes misplaced text: must/never binding a diff -> REVIEW.md; present-tense +fact -> ARCHITECTURE.md; dated or past-tense -> /history. + ## GitHub Operations Use GitHub MCP tools (`mcp__github__*`) for all GitHub operations (PRs, issues, reads) - they avoid shell escaping entirely. Fallback when unavailable: `gh` CLI with `--body-file` for any text containing backticks (shell escape characters in every supported shell). diff --git a/REVIEW_COMMON.md b/REVIEW_COMMON.md index d9191f6223..94337f6e4d 100644 --- a/REVIEW_COMMON.md +++ b/REVIEW_COMMON.md @@ -4,7 +4,8 @@ Every folder-scoped `REVIEW.md` is a binding per-change review checklist for the its folder: run it on every such change before it ships, including a change to the checklist itself. This file is the part all checklists share; a checklist's own text carries only its folder's criteria, and its opening block points here. An external repo that adopts `REVIEW.md` -vendors a copy of this file at its root. +vendors a copy of this file at its root. A statement's home is the folder's architecture doc - +`ARCHITECTURE_COMMON.md` (repo root) is that document type's contract. **What stays in a checklist: rules that can be checked against a diff. Nothing else.** A reader must be able to apply every rule **without reading the code and without prior diff --git a/history/README.md b/history/README.md index 1a770a69da..f2206d71b9 100644 --- a/history/README.md +++ b/history/README.md @@ -3,8 +3,9 @@ Design docs, plans, audits, findings, and session notes from **completed arcs** - kept for the record, no longer load-bearing. A doc belongs here when its remaining value is the record ("why is it built this way"), not operation. Living contracts and durable records -stay next to their code (e.g. `modules/dasSQLITE/PROVIDER_CONTRACT.md`, the `MASTERPLAN.md` -files, `modules/dasLLAMA/*_plan.md` specs still cited from code, `results.md` ledgers). +stay next to their code (e.g. `modules/dasSQLITE/PROVIDER_CONTRACT.md`, the per-module +`ARCHITECTURE.md` files, `modules/dasLLAMA/*_plan.md` specs still cited from code, +`results.md` ledgers). Process and criteria: `skills/internal/doc_archiving.md`. - `compiler/` - shipped compiler-rework docs (fixed-array structural types, annotation-info @@ -22,7 +23,7 @@ Process and criteria: `skills/internal/doc_archiving.md`. `tutorial-mockup/` design artifacts); the living contract stays at `modules/dasSQLITE/PROVIDER_CONTRACT.md` - `dasSpirv/` - the completed PHASE6-9 working plans (content captured in - `modules/dasSpirv/MASTERPLAN.md`) + `modules/dasSpirv/ARCHITECTURE.md` and `history/dasSpirv/MASTERPLAN_LOG.md`) - `dasVulkan/` - the original boost-layer design plan, carried over when the module moved in-tree; the living docs are `modules/dasVulkan/CLAUDE.md` and its `ROADMAP.md` - `linq_fold/` - the linq_fold arc's plans and audits; the living masterplan is @@ -139,3 +140,5 @@ listed in the log below - search it first when hunting for a doc. - 2026-08-24 `modules/dasLLAMA/eseries_bringup_plan.md` -> `history/dasLLAMA/eseries_bringup_plan.md` - the E-series (MatFormer) bringup plan (shipped via `#3791`/`#3796`) - 2026-08-24 `modules/dasLLAMA/tool_formats_plan.md` -> `history/dasLLAMA/tool_formats_plan.md` - the tool-call wire-format plan (shipped via `#3691`) - 2026-08-24 `modules/dasLLAMA/quant_lane_audit_plan.md` -> `history/dasLLAMA/quant_lane_audit_plan.md` - the format x kernel-lane x platform coverage audit (holes plugged; the kq-native and fused-arm work it drove shipped) +- 2026-08-27 `modules/dasSpirv/MASTERPLAN.md` (plan sections + implementation log) -> `history/dasSpirv/MASTERPLAN_LOG.md` - the Phase 0-11 SPIR-V emitter build-out, its phase plans, and the tutorial-prerequisite backlog (the doc's present-tense half stayed as `modules/dasSpirv/ARCHITECTURE.md`) +- 2026-08-27 `modules/dasMetal/MASTERPLAN.md` (plan sections + implementation log) -> `history/dasMetal/MASTERPLAN_LOG.md` - the Phase 0-8 MSL emitter build-out and the dasLLAMA GPU-resident prefill/decode chase (the doc's present-tense half, including the shared cross-backend ledger, stayed as `modules/dasMetal/ARCHITECTURE.md`) diff --git a/modules/dasMetal/MASTERPLAN.md b/history/dasMetal/MASTERPLAN_LOG.md similarity index 78% rename from modules/dasMetal/MASTERPLAN.md rename to history/dasMetal/MASTERPLAN_LOG.md index 0299d1fc3f..8f5b64df3d 100644 --- a/modules/dasMetal/MASTERPLAN.md +++ b/history/dasMetal/MASTERPLAN_LOG.md @@ -1,118 +1,13 @@ -# dasMetal - native Metal compute: platform-gated binding + pure-daslang MSL backend +# dasMetal MASTERPLAN - archived plan material and implementation log -> **This file is the durable masterplan and implementation history for dasMetal.** -> It is the canonical, reviewable record - not Claude memory, not a scratch plan. -> Each phase appends a dated entry to the **Implementation log** at the bottom as it lands. +Archived 2026-08-27 from `modules/dasMetal/MASTERPLAN.md`, which became +`modules/dasMetal/ARCHITECTURE.md` and kept only present-tense reference content +(including the shared cross-backend asymmetry ledger). Everything below is that +document's plan sections and its implementation log, verbatim. -## Why - -dasVulkan owns graphics on every platform (including macOS via MoltenVK). What it cannot -give on Apple Silicon is the **native compute fast path**: no Vulkan-emulation layer under -every dispatch, unified-memory `MTLBuffer`s that are plain pointers on both CPU and GPU, -and Metal-only features (simdgroup reductions, `simdgroup_matrix`). The target profile is -dasLLAMA-class GPU compute offload on the M-boxes. Scope is **compute only** - graphics -stays dasVulkan's job. - -The shader-language facts that shape the design: **LLVM cannot emit MSL** (no backend; AIR, -Metal's binary form, is a private version-unstable LLVM-IR dialect) and **Metal does not -ingest SPIR-V** - its sanctioned inputs are MSL source and DXIL. Translation libraries -exist (SPIRV-Cross is MoltenVK's own shader stage), but vendoring one is a third-party C++ -dependency that contradicts this module's thesis: *Metal.framework is just there*. - -So dasMetal uses the house pattern, already proven three times: dasGlsl emits **GLSL text** -from the daslang AST (`GlslExport`), dasSpirv emits **SPIR-V words**, and `daslib/aot_cpp.das` -emits **C++ for the entire language**. MSL is a C++14 dialect; a compute-only MSL text -emitter is squarely inside the established pattern. Just as `[compute_shader]` lowers -daslang->SPIR-V->Vulkan, `[metal_kernel]` lowers daslang->MSL->Metal. - -What makes it cheap: Metal's in-process runtime compiler (`newLibraryWithSource`) does all -GPU optimization; text is the easiest backend to emit and debug; and the authoring frontend -(module globals + `@ssbo`/`@binding` field annotations + builtin globals) already exists in -dasSpirv and is reused as-is. - -## Settled decisions - -1. **Lives in the main tree as `modules/dasMetal`.** The C++ binding is APPLE-gated; the - `metal/` daslang files (emitter + annotation) are **pure das and registered on ALL - platforms** - emitter/text tests run on every CI lane; only GPU execution is Apple-only. -2. **Compute only.** No render pipelines, no drawables, no CAMetalLayer. Graphics = dasVulkan. -3. **Zero third-party code.** No metal-cpp, no SPIRV-Cross, no MoltenVK, no committed - `.msl`/`.metallib`, no external SDK. The binding is a hand-written Obj-C++ shim - (`src/dasMetal.mm`) over the system Metal.framework - ~18 externs for the compute subset. - (metal-cpp + dasClangBind rejected: vendored headers and binder churn for an ~18-extern - surface; revisit only if the surface outgrows hand maintenance.) -4. **Class-based authoring: kernels are class methods, resources are class members.** A - compute pass is a `class` whose `@ssbo @binding = N` members declare the buffers and - whose `[metal_kernel]` methods are the kernels - no module-scope resource globals (they - pollute the namespace and don't scale to two kernels with different buffer sets in one - file), and members->kernel-parameters is exactly MSL's own model. Multiple kernels over - one buffer set = multiple methods in one class. Builtin globals - (`gl_GlobalInvocationID`, ...) are still reused via `require spirv/spirv_builtins public` - (dasSpirv is unconditional in-tree pure das - always resolves). Because the body is - ordinary daslang, the same method also executes on the CPU - the primary correctness - oracle (Test architecture). **Zero edits to shipped dasSpirv/dasGlsl.** -5. **Naming.** C++ module `das_metal` (`Module("das_metal")`, class `Module_DasMetal`). das - files under `metal/`: `require metal/msl_shader`, `require metal/das_metal_boost`. - Consumers guard: `require ?das_metal metal/das_metal_boost` + - `static_if (typeinfo builtin_module_exists(das_metal))` (the sql_boost provider pattern). - If a top-level registration proves workable in Phase 0, the spelling may shorten to a - bare `das_metal_boost`; the two-segment guarded form is the safe default. -6. **Test-per-construct is a hard requirement**, enforced by a construct census (gate B) + - the real-frontend compile gate. LCOV covers runtime files only - the emitter runs at - compile time, invisible to line coverage (dasSpirv finding; census is the proxy). -7. **PoC before breadth.** The earliest milestone is `c[i] = a[i]*b[i]` through the REAL - emitter on a real M-series GPU. The only hand-written MSL ever permitted is the Phase-0 - dev-scaffold **inline string** proving the binding, deleted when Phase 1 lands (the - `_handbuild_square.das` pattern). **No `.msl` file is ever committed.** -8. **`fastmath` is a `[metal_kernel]` property, ON by default.** daslang's own posture is - fastmath-on, so we are not chasing bit-exact float parity; Metal's runtime compiler also - defaults fast-math ON and we keep it. Float oracles compare with tolerance (ints - bit-exact). `[metal_kernel(fastmath=false)]` per kernel when isolating a divergence - needs strict IEEE. -9. **Every `matmul2d_descriptor` sets `relaxed_precision = true`.** RP = false keeps the - op off the M5 tensor unit's fast path - measured 2-3x across the tmm2d families; the - cooperative-tensor register layout the fast path uses is what RP licenses. `REVIEW.das` - enforces the emitter's descriptor sites. - -## Architecture - -`modules/dasMetal` = one APPLE-gated Obj-C++ shim + pure-daslang emitter files: - -| File | Gen/Hand | Purpose | -|---|---|---| -| `src/dasMetal.mm` | hand | `Module("das_metal")` - Obj-C++ shim over Metal.framework. Opaque annotated handles (device, queue, command buffer, compute encoder, pipeline state, library, function, buffer) + the extern surface below. Compiled with ARC; handles cross to das as `__bridge_retained void*`; `metal_release` = `__bridge_transfer`. Shim-side live-object counter for the leak gate. APPLE-only; links `-framework Metal -framework Foundation`. | -| `metal/msl_types.das` | hand | daslang `TypeDecl` -> MSL type name (32-bit scalars/bool + the 16/8-bit lattice - MSL has native `half`/`short`/`char` - and their 2/3/4 vectors, classified via the shared `daslib/shader_block_layout` rails). `msl_buffer_elem_name` gives the layout-bearing spelling: 3-lane elements take MSL's `packed_T3` (das packs tightly; unified memory means the das array IS the buffer). | -| `metal/metal_builtins.das` | hand | Metal-only builtin surface over the shared lingua franca (the spirv_builtins pattern): re-exports `daslib/shader_lingua_franca` (whence `gl_WorkGroupSize` and the four `gl_Subgroup*` IDs, shared with dasSpirv) and adds the `simd_sum`/`simd_shuffle*` intrinsics (Metal spellings, float/int/uint). Identity stub bodies = width-1 simdgroup CPU semantics. | -| `metal/msl_emit.das` | hand | The text emitter: `[macro_function] generate_msl(fn, var errors, cfg, var census) : string`. Manual recursion (`emit_value`/`emit_stmt`, mirroring `spirv_emit`). Kernel-signature synthesis from `@ssbo` globals (the one structural novelty - below). Records the construct census at every emit site. | -| `metal/msl_shader.das` | hand | `[metal_kernel]` function-macro (`MetalKernel : AstFunctionAnnotation`, modeled on `SpirvShader`), applied to a **class method**; args: `name`, `fastmath` (default **true**, surfaced to the host via a companion ``{name}`msl_fastmath : bool`` global feeding the pipeline-compile options). `apply` declares the public ``{name}`msl : string`` global; **`fixup` fills `glob.init = new ExprConstString(...)`** - string capture is call-free, so fixup suffices (dasGlsl precedent; the patch/astChanged dance dasSpirv needed for `array<uint>` does not apply). Does `require spirv/spirv_builtins public`. | -| `metal/das_metal_boost.das` | hand | Host sugar over `das_metal`: `with_metal_device`, `pipeline_from_kernel` (compile + error surfacing), unified-memory buffer helpers, `run_compute_1d` one-liner, live-object leak assert. `require das_metal` -> usable only where the C++ module exists. | -| `CMakeLists.txt` | hand | `ADD_MODULE_DAS_FROM_DESCRIPTOR(metal metal)` unconditional (rows derived from `.das_module`, emitter everywhere); `IF(APPLE)`: `ADD_MODULE_CPP(DasMetal)` + `ADD_MODULE_LIB` + frameworks. Install rule mirrors dasSpirv's. | - -**Extern surface (PoC-complete).** `metal_create_system_default_device`; device name + -unified-memory query; `metal_new_command_queue`; `metal_new_library_from_source(dev, src, -fastmath; var error)`; `metal_new_function(lib, name)`; `metal_new_compute_pipeline(dev, -fn; var error)`; `metal_new_buffer(dev, bytes)` (storageModeShared); -`metal_buffer_contents(buf) : void?` (the unified-memory pointer - host reads/writes it -directly, no map/unmap); `metal_new_command_buffer(q)`; `metal_new_compute_encoder(cb)`; -`metal_set_pipeline`; `metal_set_buffer(enc, buf, offset, index)`; -`metal_dispatch_threadgroups(enc, groups, threads_per_group)` (+`metal_dispatch_threads` -as the Apple-silicon exact-grid fast path); `metal_end_encoding`; `metal_commit`; -`metal_wait_until_completed`; `metal_command_buffer_error`; `metal_release`; -`metal_live_object_count`. +--- -**Kernel-signature lowering (the one structural novelty).** MSL has no module-scope device -globals - and the authoring class's members map onto its model exactly: each `@ssbo` -member lowers to a kernel parameter `device T* name [[buffer(N)]]` (`device const T*` when -no kernel body writes it; member write-set collected in a pre-scan), and each referenced -builtin global lowers to a builtin-attributed parameter (`gl_GlobalInvocationID` -> -`uint3 gl_GlobalInvocationID [[thread_position_in_grid]]`). Member access in the method -body (bare `a` / `self.a`) emits as the bare parameter name, so the body needs no other -rewriting. `@binding` = the flat `[[buffer(N)]]` index; `@set` must be absent or 0 (clean -error - Metal has no descriptor sets); duplicate bindings within one class are a clean -error. Identifiers colliding with MSL keywords (`kernel`, `device`, `constant`, `thread`, -`half`, ...) are mangled. The exact AST shape of method-member access (`ExprField` over -`self`, constness, `ExprRef2Value` wrapping at fixup time) is dumped and recorded as the -first Phase-1 step - the same discipline as dasSpirv's square AST dump. +## Capture + cache (lifted from the Architecture section - the cache half was never built) **Capture + cache.** Capture mirrors dasGlsl exactly (`ExprConstString` in fixup). A companion reflection global is deferred until the boost needs auto-binding (mirror @@ -156,41 +51,6 @@ Constructs: preamble; kernel signature (device-pointer param, const-ness inferen `[[buffer(N)]]`, builtin param); value-`let`; single-component swizzle; array index -> pointer index; `ExprOp2 *` (float); `ExprCopy` -> assignment. -## Test architecture - "every emitted construct has a test" - -Three behavioral layers + enforcement gates: - -1. **Text-assertion units** (`tests/msl/`, ALL platforms). Each test compiles a tiny - `[metal_kernel]` fixture, calls `generate_msl`, asserts structural facts (signature - shape, attribute per binding, const-ness, statement forms), plus a golden snapshot of - *our own* emitted text as a forward regression guard (dasSpirv's byte-snapshot amendment). -2. **Compile gate** (the spirv-val analog). Every emitted MSL must compile through the - **real Metal frontend**: in-process `metal_new_library_from_source` where `das_metal` - exists (no Xcode required - the OS MTLCompilerService), soft-skip elsewhere, - hard-required on the macOS CI lane. (`xcrun metal` offline compile is a secondary local - oracle only - it needs full Xcode, not CLT.) -3. **Real-GPU behavioral gate vs the CPU-reference oracle - as early as Phase 1** - (`tests/metal/`, Apple-only). The kernel body is ordinary daslang, so the **same method - runs on the CPU**: a driver loop sets `gl_GlobalInvocationID` and calls the method on a - class instance whose members are plain arrays - that CPU run (interp/JIT) produces the - expected buffer contents with zero second-source effort. GPU results compare against - it: ints bit-exact, floats with tolerance (fastmath on both sides - settled decision 8; - the PoC's `a[i]*b[i]` on small ints is exact regardless). Files are `require ?das_metal` - + `static_if builtin_module_exists` guarded, so they compile and no-op cleanly on - non-Apple lanes. Primary gate = **local M-boxes**; CI = the macOS lane's paravirtual - device (Phase-0 probe decides behavioral vs compile-only). - -- **Gate A - LCOV** on runtime-reached files (`das_metal_boost`, `msl_types` where runtime - code exists). The emitter is compile-time -> census is its coverage proxy. -- **Gate B - construct census.** `generate_msl` records every construct kind it emits - (`table<string>` set: `"kernel_sig"`, `"param.buffer"`, - `"param.builtin.thread_position_in_grid"`, `"op.mul.f32"`, ...). A meta-test unions the - census across all fixtures and asserts equality with the declared supported set, both - directions. Census-record and emit are fused in one helper so they cannot drift; the - golden-snapshot layer catches emissions that bypass it. -- **Leak gate.** Every `tests/metal/` file asserts `metal_live_object_count() == 0` at - exit - Metal objects are invisible to all six das leak detectors, so the shim counts. - ## Phasing (independently verifiable; no sizing) - **Phase 0 - binding bring-up + probes.** This file; `src/dasMetal.mm` (module + @@ -263,50 +123,6 @@ Three behavioral layers + enforcement gates: shapes. Gates: kernel unit parity tests, 40/40 greedy parity (tolerance path), then the clean round (Parsec OFF - Boris window): CPU vs hybrid vs resident vs lcpp `-ngl 99`. -## Cross-backend parity - the kernel-model asymmetry ledger - -The mirror rule (REVIEW.md, both emitters) records deliberate or pending kernel-model -asymmetries HERE - one ledger for both backends (dasSpirv's MASTERPLAN points at this -section). The primary correctness oracle remains the CPU-reference run of the same das body, -cheaper and stricter than a second GPU; cross-GPU parity of one source is a nice-to-have now -that both emitters read class members. Current entries: - -- **Class-member authoring is symmetric now.** dasSpirv gained `[spirv_kernel]` classes (this - emitter's model, ported wholesale in the vulkan class-kernel arc) - the original blocker on - stacking both shader annotations on one body is gone. Free-function callees and - devirtualized overrides are symmetric too (msl_emit's devirtualization table was the donor). -- **Multi-kernel-per-class is symmetric now; `family=` surface sharing stays Vulkan-only.** - Both lenses accept N kernel methods per class with `kernel=` picking per instance - (`_mtl_toy.das` / `_vkd_toy.das` are the sibling fixtures). `family=` remains Vulkan's - alone - Metal's `enc_*` builder is the whole generated surface, so there is nothing for a - family to share. -- **Method flattening, scope splicing, and plain-member thread-local STATE are Metal-only.** - `msl_emit` splices method bodies inline (statically devirtualized, value and statement - position) and lowers plain class members to kernel-entry thread-locals (scalar-with-init, - fixed array, carried pointer walk - the state contract is per-thread); `spirv_emit` still - rejects an unannotated member and lowers method calls as calls. If a joined or - state-carrying class ever crosses backends, the same design ports - deliberate asymmetry - until then. -- **The tmm2d tensor-GEMM builtin family is Metal-only (pending, not deliberate).** The - whole-GEMM helpers (`tmm2d_f32_bf16_f32`/`_f16w_f32`/`_q8*_f32` with the bk staging - depths) and the staged tg protocol (`begin/step` in `deva`/`nt`/`devant` flavors, - `tile_tg`, `rowscale`, the `store`/`store_half`/`store_bias` tails, `sgmat_to_half`) - lower MPP `matmul2d` cooperative tensors; the Vulkan analogue would sit on the coopmat2 - classes when a consumer needs it. Until then the CPU-replay bodies are the only - cross-checked twin. -- **Inheritance in the kernel corpus: Vulkan leans on it, Metal does not yet.** The vulkan - classes are base+leaf families (kq GEMV/batch, flash attention, deltanet); the metal - classes are flat, with at least one base duplicated inline (MetalMoeMulMmK6). Dedup - opportunity, not a correctness gap. -- **Grid-literal validation: Vulkan rejects non-int64 ceil-div grid params at macro apply; - the metal lens still carries that silent infer trap.** Pending metal-side fix. -- **cm2 decode-in-load (and its class-method decode form) is Vulkan-only by hardware.** - `[spirv_decode]` - including the method form, where a das-level self is erased from the - rigid 3-param signature so the decode body reads class members (a separate scale plane, - push constants, @workgroup staging) - lowers SPV_NV_cooperative_matrix2 tensor loads. - Metal-4 tensors have no decode-callback analogue; the Metal quant GEMMs stage dequant - through threadgroup memory instead. Deliberate, target-specific - not a pending port. - ## Top risks 1. **macOS CI paravirtual Metal.** GitHub's macOS runners expose an "Apple Paravirtual @@ -1114,66 +930,3 @@ Census grows matching kinds (gate B both-directions discipline). Tests: tgmem param / builtin-using helper / uniform-reading method; negative gates for recursion, method-field reassignment, unsupported param types. First consumer: the dasLLAMA SqAttn family dedup (16-of-20 twins share one skeleton; stage helpers + one uniform struct per family). - -## @uniform structs - the kargs form - -A kernel with a dozen scalars used to cost a dozen binds. `@uniform @binding = N ka : KArgs`, -where `KArgs` is a plain das struct, costs one: the struct definition emits into the MSL -preamble, the member becomes `constant KArgs& ka [[buffer(N)]]`, and `ka.field` is one hop. - -**Fields are `int`/`uint`/`float` only.** That restriction is the whole feature: a struct of -4-byte scalars is the same bytes in das and in MSL, with no padding rules to keep in sync, so -the host writes its own struct straight through `setBytes` (`metal_set_bytes`, already bound - -Metal caps it at 4KB and a kargs struct is well under). A `float4` field would be 16-byte -aligned in MSL and shift every field after it; a nested struct brings its own alignment. Both -are refused (`tests/msl/_fail_closed/_fc_ustruct_{field,nested}.das`). - -Host side: `run_compute_1d_kargs` for tests, and dasLLAMA's `kn_bytes` / `kn_kargs` on the -capture rail - under graph capture the bytes are COPIED into a pool, since the caller's kargs is -a stack local long gone by replay time. - -**It costs the GPU nothing.** Every field was already a runtime `constant uint&`; the fold trades -N bindings for one argument-buffer read and the kernel branches on exactly the values it did -before. Nothing about the *shape* of a kernel may become a kargs field, though: a block stride, a -lane width, a codec selector belongs in a per-codec overload or a monomorphized generic, where it -stays a literal in the emitted MSL. Passing one as a value and trusting Metal to inline-and-fold -it back is an assumption, not a guarantee - and the assumption is worth nothing in the kernels -that matter. - -Tests: `tests/msl/test_msl_uniform_struct.das` (definition placement, the single `constant&` -parameter, no per-scalar parameter survives) and `tests/metal/test_metal_uniform_struct.das` -(GPU vs CPU-oracle - the only thing that can prove the layout claim; mutation-verified by -inserting one pad field into the emitted struct). - -## Phase 0 follow-on: pointer parameters - -A helper takes a raw pointer: `def stage(var p : half4 const?; n : uint)`. This is what lets the -kernels that stream `unsafe(addr(buf[i]))` through a loop - the split-K "D" attention family, the -K-quant mul_mm trio - share a skeleton at all, since the advancing pointer is the shape and -rewriting it as index math changes AGX register allocation. - -**MSL requires an address space in the signature** (`device half4*` vs `threadgroup half4*`) and -the das type `half4?` carries none - the same type can name an @ssbo interior or a @workgroup -interior. The parameter declares it and the call site proves it: - -- **Unmarked is `device`.** Every pointer stream in the zoo is device, so the common case is - plain das with no marker to learn. -- **`@threadgroup p : T?`** opts into threadgroup memory. Parameter annotations already parse - and keep their values, so this needs no grammar. -- **The emitter derives the space from the argument's provenance and cross-checks it** - an - inline `addr(member[i])`, a pointer local, a pointer parameter being forwarded, or `p + n` - over any of those. Disagreement is a das error naming both sides; provenance it cannot trace - (a thread-space local) is refused rather than guessed. The declared space is never trusted on - its own, so a wrong marker cannot reach the runtime MSL compile as a null pipeline. -- **The POINTEE's const is the MSL const.** `T const?` lowers `device const T*` and reads only; - `T?` lowers `device T*` and marks the source member written, exactly as a `var array<T>` - parameter does. (Writing through it in das additionally needs the handle non-const - `var o : - T?` - because das flows the handle's const onto the dereference.) - -Long-term this is a shortcut for **pointer families** as a language-level feature - a real type -axis rather than an annotation. Parked on `modules/dasLLAMA/followup_general.md`. - -Tests: `tests/msl/_msl_common.das` + `test_msl_functions.das` (signatures, forwarding, inline -`addr` arguments, census), `tests/metal/test_metal_functions.das` (GPU advancing dot + threadgroup -slab max vs a directly-computed expectation), `tests/msl/_fail_closed/_fc_ptr_space_{tg,dev}.das` -and `_fc_ptr_untraceable.das` (both mismatch directions + the untraceable case). diff --git a/modules/dasSpirv/MASTERPLAN.md b/history/dasSpirv/MASTERPLAN_LOG.md similarity index 94% rename from modules/dasSpirv/MASTERPLAN.md rename to history/dasSpirv/MASTERPLAN_LOG.md index d49d373bc1..76d2a6a326 100644 --- a/modules/dasSpirv/MASTERPLAN.md +++ b/history/dasSpirv/MASTERPLAN_LOG.md @@ -1,80 +1,12 @@ -# dasSpirv - pure-daslang daslang->SPIR-V shader backend - -> **This file is the durable masterplan and implementation history for dasSpirv.** -> It is the canonical, reviewable record - not Claude memory, not a scratch plan. -> Each phase appends a dated entry to the **Implementation log** at the bottom as it lands. - -## Why - -dasVulkan's shaders are authored as hand-written GLSL `.vert`/`.frag`/`.comp`, compiled -offline with glslangValidator into `.spv` blobs, *both* committed, and loaded at runtime. -Two languages, an external SDK, committed binaries, no code-sharing with the host. - -dasGlsl already eliminated exactly this for OpenGL: shaders written in daslang, annotated -`[..._program]`, an `AstVisitor` (`GlslExport`) emits GLSL at compile time into a global -captured by `@@fn`, fed to `glShaderSource`. dasSpirv does the same for Vulkan, but emits -**SPIR-V binary directly from the daslang AST** - no GLSL/glslang intermediary, no LLVM. - -SPIR-V is an SSA IR (typed results, basic blocks, structured control flow, a logical -builder) - the same shape `llvm_jit` already lowers daslang to. So dasSpirv is -**dasGlsl's frontend pattern + llvm_jit's SSA-backend pattern, fused into a SPIR-V emitter**. -Just as `[jit]` lowers daslang->LLVM->native, `[compute_shader]` lowers daslang->SPIR-V->GPU. - -What makes it cheap: drivers do all GPU optimization (we emit naive *valid* SPIR-V - so does -glslang); SPIR-V binary is a self-delimiting word stream (trivial to emit and disassemble); -the backend-agnostic AST tooling (`collect_dependencies`, `collect_used_types`, -`make_visitor`) is reusable verbatim; and dasVulkan already has a GPU-verified `out[i]==i*i` -compute test as a ready-made end-to-end gate. - -## Settled decisions - -1. **Lives in the main tree as `modules/dasSpirv`** - sibling to dasGlsl/dasLLVM, in - PR-protected GaijinEntertainment/daScript, sharing main-tree CI + `daslib/coverage` + - `tests/`. SPIR-V is a general daslang capability (also useful for GL4.6/WebGPU), not - Vulkan-specific. -2. **Fresh frontend** - new `[compute_shader]`/`[vertex_shader]`/`[fragment_shader]` - annotations; dasGlsl is the *design map*, not a code dependency. Reuse only the generic - AST tooling (none of which lives in dasGlsl). **Zero edits to the shipped dasGlsl/dasOpenGL.** -3. **Test-per-instruction is a hard requirement**, enforced by both an opcode census and the - LCOV coverage gate. -4. **SPIR-V target 1.3 / Vulkan 1.1** - StorageBuffer storage class + `Block` (not the - deprecated 1.0 BufferBlock+Uniform path). lavapipe advertises >=1.2; 1.3 also dodges the - 1.4 rule that the entry-point interface must list *all* globals (<=1.3 lists only - Input/Output). An annotation arg `spirv_version` leaves 1.0/1.4 selectable later; the - interface builder must be made version-aware before any bump. - -## Architecture - -`modules/dasSpirv` is **pure daslang** (mirrors dasGlsl: a `spirv/` subdir of `.das` files + -CMake resolver rows derived from `.das_module`; no `.shared_module`, no C++). dasVulkan -consumes it via `require spirv/...` and feeds the emitted `array<uint>` (SPIR-V words) to `create_shader_module`. - -| File | Gen/Hand | Purpose | -|---|---|---| -| `spirv/spirv_grammar.das` | **generated** | Opcode + GLSL.std.450 enums + StorageClass/Decoration/BuiltIn/ExecutionMode/Capability token constants, generated from the vendored, pinned SPIRV-Headers grammar JSON. Single source of numeric truth (+ opcode->name table for the disassembler). | -| `spirv/spirv_builder.das` | hand | `SpirvModule` (per-section append buffers), monotonic id allocator, type/constant dedup pools, `emit(section, opcode, ...)`, `get_type_id`, `get_const_id`, `finalize() : array<uint>`. AST-agnostic. | -| `spirv/spirv_types.das` | hand | daslang `TypeDecl` -> SPIR-V type-id (scalars, vec/mat, fixed + runtime arrays, structs, pointers) with layout decorations. Highest-detail correctness file. | -| `spirv/spirv_emit.das` | hand | `SpirvEmit : AstVisitor` codegen visitor + `[macro_function] generate_spirv(fn, var errors) : array<uint>`. Override set mirrors `GlslExport`; rejection list mirrors its error list. | -| `spirv/spirv_shader.das` | hand | `[compute_shader]`/`[vertex_shader]`/`[fragment_shader]` function-macros (subclass `SpirvShader : AstFunctionAnnotation`, modeled on dasGlsl's `GlslShader`). `apply` reserves an `array<uint>` global; `fixup` runs dependency collection, calls `generate_spirv`, sets `glob.init`. | -| `spirv/spirv_dis.das` | hand | Minimal disassembler + opcode-census helper (self-delimiting walk: word0 = `(wordCount<<16)\|opcode`). Symbolic via `spirv_grammar`'s opcode->name table. | -| `generator/gen_spirv_grammar.das` | hand | The mini-generator: reads vendored grammar JSON -> emits `spirv/spirv_grammar.das`. | -| `spirv_headers/*.json` | vendored | Pinned `spirv.core.grammar.json` + `extinst.glsl.std.450.grammar.json` (provenance + commit recorded below). | -| `CMakeLists.txt` | hand | `ADD_MODULE_DAS_FROM_DESCRIPTOR(spirv spirv)` (rows derived from `.das_module`) + install rule, modeled on `modules/dasGlsl/CMakeLists.txt`. | - -**SSA backend (llvm_jit template).** `SpirvEmit` carries `e2v : table<Expression?;uint>` -(Expression->result-id), `v2v : table<Variable?;uint>` (Variable->pointer-id), -`ite2blocks`/`loop_stack` for control flow, and a hand-maintained `cur_block_terminated` -(SPIR-V has no builder to query - set on every terminator, cleared on `OpLabel`; guards -mirror llvm_jit's `current_block_terminates()`). Section buffers concatenate at `finalize` -in SPIR-V's mandated order: header(5) -> capabilities -> extensions -> ext-imports -> -memory-model -> entry-points -> exec-modes -> debug -> decorations -> types+constants+global-vars -(interleaved, define-before-use, deduplicated) -> functions. - -**Capture mechanism (dasGlsl analog).** dasGlsl's `fixup` sets -`glob.init = new ExprConstString(value := text)`; ours builds an `ExprMakeArray` of -`ExprConstUInt` (one per SPIR-V word) for an `array<uint>` global named `"{func.name}`spirv"`. -`generate_spirv` is a standalone `[macro_function]` called by **both** `fixup` and the unit -tests - so opcode assertions hit the real codegen path without macro plumbing. +# dasSpirv MASTERPLAN - archived plan material and implementation log + +Archived 2026-08-27 from `modules/dasSpirv/MASTERPLAN.md`, which became +`modules/dasSpirv/ARCHITECTURE.md` and kept only present-tense reference content. +Everything below is that document's plan sections and its implementation log, verbatim. + +--- + +## Cache (planned; lifted from the Architecture section - no such constant was ever built) **Cache.** Mirror llvm_jit's content-addressed pattern: a `SPIRV_CODEGEN_VERSION` constant FNV-folded with `get_function_aot_hash(fn)` (and the pinned SPIRV-Headers version), so @@ -96,31 +28,6 @@ Replace dasVulkan's `examples/shaders/square.comp` (+committed `.spv`) with a da - **Body:** `OpFunction`/`OpLabel`; `OpAccessChain`; `OpLoad`; `OpIMul`; `OpStore`; `OpReturn`; `OpFunctionEnd`. -## Test architecture - "every emitted instruction has a test" - -Three behavioral layers + two enforcement gates (all in main-tree `tests/spirv/` except GPU): - -1. **Opcode-assertion units.** Each test compiles a tiny `[compute_shader]` fixture, calls - `generate_spirv`, runs `spirv_dis` to a structured instruction list, and asserts the - expected numeric opcode present/absent/operand-at-offset/decoration present. One test per - emittable instruction kind. -2. **`spirv-val` gate.** Every blob the suite produces is validated via SPIRV-Tools - `spirv-val` (`C:\VulkanSDK\1.4.350.0\Bin`). Soft-skip if absent locally, hard-required in - CI - the real correctness oracle for structured-CFG and define-before-use bugs. -3. **Real-driver behavioral regression - as early as Phase 1.** A one-call framework in - dasVulkan, `run_compute_spirv(words, n) : array<uint>` over the existing `compute_boost`, - runs any emitted blob in ~2 lines. Primary gate = **local real GPU** (SDK 1.4.350), - `out[i]==i*i`, every iteration. CI = lavapipe software (dasVulkan CI is lavapipe-only on - `ubuntu-latest`; GitHub-hosted runners have no GPU, so no real-GPU CI lane - covered - locally; a self-hosted GPU runner would wire straight in). Triangle pixels at Phase 3. - -- **Gate A - LCOV coverage.** Run layer 1 under `dastest --cov-path spirv.lcov` - (`daslib/coverage.das`); require ~100% of emission-dispatch lines in `spirv_emit.das` + - `spirv_builder.das`. -- **Gate B - opcode census (stronger).** `spirv_dis` aggregates the opcode set across *all* - fixtures; a meta-test asserts it equals the declared "supported opcodes" set. Catches the - gap LCOV can't - an opcode whose emit line ran but was never *asserted*. - ## Phasing (independently verifiable; no sizing) - **Phase 0 - scaffolding + grammar + builder + disassembler.** This file; the generator -> @@ -150,12 +57,6 @@ Three behavioral layers + two enforcement gates (all in main-tree `tests/spirv/` point `test_compute` at the emitted blob. - Phase 3: port `triangle.vert`/`.frag`, delete their `.spv`. -## Cross-backend parity - the kernel-model asymmetry ledger - -The mirror rule (REVIEW.md, both emitters) keeps the kernel model symmetric with the MSL -emitter. The ledger of deliberate/pending asymmetries is shared - ONE list, not two - and -lives in `modules/dasMetal/MASTERPLAN.md` under the same heading. - ## Top risks 1. **Type/constant dedup correctness** - keys must be injective + complete (pointer key diff --git a/modules/REVIEW_SHADER_EMITTERS.md b/modules/REVIEW_SHADER_EMITTERS.md index b8cddaed28..9fe53fe7a9 100644 --- a/modules/REVIEW_SHADER_EMITTERS.md +++ b/modules/REVIEW_SHADER_EMITTERS.md @@ -1,7 +1,7 @@ # Shared emitter rules - dasSpirv and dasMetal **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture -docs: `modules/dasMetal/MASTERPLAN.md` and `modules/dasSpirv/MASTERPLAN.md`. +docs: `modules/dasMetal/ARCHITECTURE.md` and `modules/dasSpirv/ARCHITECTURE.md`. **These rules bind both emitters.** Each module's `REVIEW.md` routes here, and a change under that module applies this file together with the module's own list. @@ -27,7 +27,7 @@ that module applies this file together with the module's own list. - **A diff that adds a kernel capability needing a runtime shape value ships a specialization path.** A specialization path is one compiled variant per constant shape. When the capability - cannot have one, the same diff records that in the module's `MASTERPLAN.md`. + cannot have one, the same diff records that in the module's `ARCHITECTURE.md`. - **Never check a claim about emitted shape against the das source - check it in the emitted words or text.** Emitted shape is the structure of the emitted kernel - its signature, its @@ -35,7 +35,7 @@ that module applies this file together with the module's own list. threadgroup sizes). - **A diff that adds a kernel-model capability to one emitter adds it to the other, or records - the asymmetry in the shared ledger (`modules/dasMetal/MASTERPLAN.md`).** A kernel-model + the asymmetry in the shared ledger (`modules/dasMetal/ARCHITECTURE.md`).** A kernel-model capability is a way of authoring or lowering kernels: class members, multiple kernels per class, method flattening, tensor-GEMM builtins. diff --git a/modules/dasLLAMA/followup_vulkan.md b/modules/dasLLAMA/followup_vulkan.md index a341eeec21..31b789919d 100644 --- a/modules/dasLLAMA/followup_vulkan.md +++ b/modules/dasLLAMA/followup_vulkan.md @@ -76,7 +76,7 @@ Ordered roughly by user-visible value; re-rank against zen2 measurements before `ffn_vs_ref` red bisected per-kernel via the model-less kernel-unit suite, portability_subset enabled at device create (landed dasVulkan-side); correctness only, Metal stays the fast path on that box. (b) Close the kernel-model asymmetry ledger - (`modules/dasMetal/MASTERPLAN.md` sec.Cross-backend parity): relax `[metal_dispatch]` to + (`modules/dasMetal/ARCHITECTURE.md` sec.5 Cross-backend parity): relax `[metal_dispatch]` to multi-kernel + `family=` like `[vk_dispatch]`, adopt inheritance in the metal kernel corpus where families exist, fix the metal lens's grid-literal infer trap. (c) Hoist the ~80 lines of dispatch-lens micro-grammar/validation the `[vk_dispatch]` and diff --git a/modules/dasMetal/ARCHITECTURE.md b/modules/dasMetal/ARCHITECTURE.md new file mode 100644 index 0000000000..08bec0370d --- /dev/null +++ b/modules/dasMetal/ARCHITECTURE.md @@ -0,0 +1,279 @@ +# dasMetal - native Metal compute: platform-gated binding + pure-daslang MSL backend + +**Read `ARCHITECTURE_COMMON.md` (repo root) first - its contract binds this document.** The +checklists that bind a diff here are `REVIEW.md` (this folder) and +`modules/REVIEW_SHADER_EMITTERS.md`. The plan sections and the implementation log this +document grew from are archived at `history/dasMetal/MASTERPLAN_LOG.md`. + +## 1. Why + +dasVulkan owns graphics on every platform (including macOS via MoltenVK). What it cannot +give on Apple Silicon is the **native compute fast path**: no Vulkan-emulation layer under +every dispatch, unified-memory `MTLBuffer`s that are plain pointers on both CPU and GPU, +and Metal-only features (simdgroup reductions, `simdgroup_matrix`). The target profile is +dasLLAMA-class GPU compute offload on the M-boxes. Scope is **compute only** - graphics +stays dasVulkan's job. + +The shader-language facts that shape the design: **LLVM cannot emit MSL** (no backend; AIR, +Metal's binary form, is a private version-unstable LLVM-IR dialect) and **Metal does not +ingest SPIR-V** - its sanctioned inputs are MSL source and DXIL. Translation libraries +exist (SPIRV-Cross is MoltenVK's own shader stage), but vendoring one is a third-party C++ +dependency that contradicts this module's thesis: *Metal.framework is just there*. + +So dasMetal uses the house pattern, already proven three times: dasGlsl emits **GLSL text** +from the daslang AST (`GlslExport`), dasSpirv emits **SPIR-V words**, and `daslib/aot_cpp.das` +emits **C++ for the entire language**. MSL is a C++14 dialect; a compute-only MSL text +emitter is squarely inside the established pattern. Just as `[compute_shader]` lowers +daslang->SPIR-V->Vulkan, `[metal_kernel]` lowers daslang->MSL->Metal. + +What makes it cheap: Metal's in-process runtime compiler (`newLibraryWithSource`) does all +GPU optimization; text is the easiest backend to emit and debug; and the authoring frontend +(`@ssbo`/`@binding` field annotations + builtin globals) already exists in dasSpirv and is +reused as-is. + +## 2. Settled decisions + +1. **Lives in the main tree as `modules/dasMetal`.** The C++ binding is APPLE-gated; the + `metal/` daslang files (emitter + annotation) are **pure das and registered on ALL + platforms** - emitter/text tests run on every CI lane; only GPU execution is Apple-only. +2. **Compute only.** No render pipelines, no drawables, no CAMetalLayer. Graphics = dasVulkan. +3. **Zero third-party code.** No metal-cpp, no SPIRV-Cross, no MoltenVK, no committed + `.msl`/`.metallib`, no external SDK. The binding is a hand-written Obj-C++ shim + (`src/dasMetal.mm`) over the system Metal.framework - the compute subset only (sec.3). + (metal-cpp + dasClangBind rejected: vendored headers and binder churn for a surface this + small; revisit only if it outgrows hand maintenance.) +4. **Class-based authoring: kernels are class methods, resources are class members.** A + compute pass is a `class` whose `@ssbo @binding = N` members declare the buffers and + whose `[metal_kernel]` methods are the kernels - no module-scope resource globals (they + pollute the namespace and don't scale to two kernels with different buffer sets in one + file), and members->kernel-parameters is exactly MSL's own model. Multiple kernels over + one buffer set = multiple methods in one class. Builtin globals + (`gl_GlobalInvocationID`, ...) come from the shared `daslib/shader_lingua_franca`, which + `metal/metal_builtins.das` re-exports. Because the body is ordinary daslang, the same + method also executes on the CPU - the primary correctness oracle (sec.4). **Zero edits to + shipped dasSpirv/dasGlsl.** +5. **Naming.** C++ module `das_metal` (`Module("das_metal")`, class `Module_DasMetal`). das + files under `metal/`: `require metal/msl_shader`, `require metal/das_metal_boost`. + Consumers guard: `require ?das_metal metal/das_metal_boost` + + `static_if (typeinfo builtin_module_exists(das_metal))` (the sql_boost provider pattern). +6. **Test-per-construct is a hard requirement**, enforced by a construct census (gate B) + + the real-frontend compile gate. LCOV covers runtime files only - the emitter runs at + compile time, invisible to line coverage (dasSpirv finding; census is the proxy). +7. **No `.msl` file is committed and no MSL is hand-written.** MSL exists only as text the + emitter produces at compile time. +8. **`fastmath` is a `[metal_kernel]` property, ON by default.** daslang's own posture is + fastmath-on, so we are not chasing bit-exact float parity; Metal's runtime compiler also + defaults fast-math ON and we keep it. Float oracles compare with tolerance (ints + bit-exact). `[metal_kernel(fastmath=false)]` per kernel when isolating a divergence + needs strict IEEE. +9. **Every `matmul2d_descriptor` sets `relaxed_precision = true`.** RP = false keeps the + op off the M5 tensor unit's fast path - measured 2-3x across the tmm2d families; the + cooperative-tensor register layout the fast path uses is what RP licenses. `REVIEW.das` + enforces the emitter's descriptor sites. + +## 3. Files and lowering mechanism + +`modules/dasMetal` = one APPLE-gated Obj-C++ shim + pure-daslang emitter files: + +| File | Gen/Hand | Purpose | +|---|---|---| +| `src/dasMetal.mm` | hand | `Module("das_metal")` - Obj-C++ shim over Metal.framework. Opaque annotated handles (device, queue, command buffer, compute encoder, pipeline state, library, function, buffer) + the extern surface below. Compiled with ARC; handles cross to das as `__bridge_retained void*`; `metal_release` = `__bridge_transfer`. Shim-side live-object counter for the leak gate. APPLE-only; links `-framework Metal -framework Foundation`. | +| `metal/msl_types.das` | hand | daslang `TypeDecl` -> MSL type name (32-bit scalars/bool + the 16/8-bit lattice - MSL has native `half`/`short`/`char` - and their 2/3/4 vectors, classified via the shared `daslib/shader_block_layout` rails). `msl_buffer_elem_name` gives the layout-bearing spelling: 3-lane elements take MSL's `packed_T3` (das packs tightly; unified memory means the das array IS the buffer). | +| `metal/metal_builtins.das` | hand | Metal-only builtin surface over the shared lingua franca (the spirv_builtins pattern): re-exports `daslib/shader_lingua_franca` (whence `gl_WorkGroupSize` and the four `gl_Subgroup*` IDs, shared with dasSpirv) and adds the `simd_sum`/`simd_shuffle*` intrinsics (Metal spellings, float/int/uint). Identity stub bodies = width-1 simdgroup CPU semantics. | +| `metal/msl_emit.das` | hand | The text emitter: `generate_msl(fn, var errors, cfg, var census, var tgmem) : string`. Manual recursion (`emit_value`/`emit_stmt`, mirroring `spirv_emit`). Kernel-signature synthesis from `@ssbo` globals (the one structural novelty - below). Records the construct census at every emit site. | +| `metal/msl_shader.das` | hand | `[metal_kernel]` function-macro (`MetalKernel : AstFunctionAnnotation`, modeled on `SpirvShader`), applied to a **class method**; args: `name`, `fastmath` (default **true**). `apply` declares the public MSL-text global - the `name=` argument, or `<Class>_<method>_msl` derived from the method - plus a `<name>_fastmath : bool` companion feeding the pipeline-compile options; **`fixup` fills `glob.init = new ExprConstString(...)`** - string capture is call-free, so fixup suffices (dasGlsl precedent). Does `require msl_emit public` + `require metal_builtins public`. | +| `metal/das_metal_boost.das` | hand | Host sugar over `das_metal`: `with_metal_device`, `pipeline_from_kernel` (compile + error surfacing), unified-memory buffer helpers, `run_compute_1d` one-liner, live-object leak assert. `require das_metal` -> usable only where the C++ module exists. | +| `CMakeLists.txt` | hand | `ADD_MODULE_DAS_FROM_DESCRIPTOR(metal metal)` unconditional (rows derived from `.das_module`, emitter everywhere); `IF(APPLE)`: `ADD_MODULE_CPP(DasMetal)` + `ADD_MODULE_LIB` + frameworks. Install rule mirrors dasSpirv's. | + +**Extern surface.** `src/dasMetal.mm` binds the compute subset of Metal.framework, in +families: + +- **Device + queue:** `metal_create_system_default_device`, `metal_device_name`, + `metal_device_has_unified_memory`, `metal_max_buffer_length`, `metal_new_command_queue`. +- **Library + pipeline:** `metal_new_library_from_source(dev, src, fastmath; var error)`, + `metal_new_function`, `metal_new_compute_pipeline`, `metal_pipeline_max_total_threads`, + `metal_pipeline_thread_execution_width`. +- **Buffers:** `metal_new_buffer` (storageModeShared) and its `_untracked` / `_no_copy` / + `_no_copy_untracked` variants; `metal_buffer_contents(buf) : void?` - the unified-memory + pointer the host reads and writes directly, with no map/unmap. +- **Encoding + dispatch:** `metal_new_command_buffer` (+ `_unretained`), + `metal_new_compute_encoder` (+ `_concurrent`), `metal_set_pipeline`, `metal_set_buffer`, + `metal_set_bytes`, `metal_set_threadgroup_memory_length`, `metal_dispatch_threadgroups`, + `metal_dispatch_threads` (the Apple-silicon exact-grid fast path), `metal_memory_barrier`, + `metal_end_encoding`, `metal_commit`, `metal_wait_until_completed`, + `metal_dispatch_call_count`. +- **Errors + timing:** `metal_command_buffer_error`, `metal_command_buffer_failed`, + `metal_command_buffer_gpu_start_time` / `_end_time`, `metal_command_buffer_kernel_start_time` + / `_end_time`. +- **Events:** `metal_new_shared_event`, `metal_shared_event_signal`, `metal_cb_wait_for_event`. +- **Residency:** `metal_new_residency_set`, `metal_residency_set_add_buffer`, + `metal_residency_set_commit`, `metal_residency_set_request`, `metal_residency_set_heartbeat`, + `metal_residency_heartbeat_live` / `_sets` / `_ticks`. +- **Lifetime:** `metal_release` (one overload per handle type), `metal_live_object_count`, + `metal_live_object_report`. + +**Kernel-signature lowering (the one structural novelty).** MSL has no module-scope device +globals - and the authoring class's members map onto its model exactly: each `@ssbo` +member lowers to a kernel parameter `device T* name [[buffer(N)]]` (`device const T*` when +no kernel body writes it; member write-set collected in a pre-scan), and each referenced +builtin global lowers to a builtin-attributed parameter (`gl_GlobalInvocationID` -> +`uint3 gl_GlobalInvocationID [[thread_position_in_grid]]`). Member access in the method +body (bare `a` / `self.a`) emits as the bare parameter name, so the body needs no other +rewriting. `@binding` = the flat `[[buffer(N)]]` index; `@set` must be absent or 0 (clean +error - Metal has no descriptor sets); duplicate bindings within one class are a clean +error. Identifiers colliding with MSL keywords (`kernel`, `device`, `constant`, `thread`, +`half`, ...) are mangled. A `@uniform` member lowers to one `constant KArgs&` parameter +(sec.7) and a `@workgroup` member to a threadgroup pointer derived from the entry's `_tgmem` +companion. + +## 4. Test architecture - "every emitted construct has a test" + +Three behavioral layers + enforcement gates: + +1. **Text-assertion units** (`tests/msl/`, ALL platforms). Each test compiles a tiny + `[metal_kernel]` fixture, calls `generate_msl`, asserts structural facts (signature + shape, attribute per binding, const-ness, statement forms), plus a golden snapshot of + *our own* emitted text as a forward regression guard (dasSpirv's byte-snapshot amendment). +2. **Compile gate** (the spirv-val analog). Every emitted MSL must compile through the + **real Metal frontend**: in-process `metal_new_library_from_source` where `das_metal` + exists (no Xcode required - the OS MTLCompilerService), soft-skip elsewhere, + hard-required on the macOS CI lane. (`xcrun metal` offline compile is a secondary local + oracle only - it needs full Xcode, not CLT.) +3. **Real-GPU behavioral gate vs the CPU-reference oracle** (`tests/metal/`, Apple-only). + The kernel body is ordinary daslang, so the **same method runs on the CPU**: a driver loop + sets `gl_GlobalInvocationID` and calls the method on a class instance whose members are + plain arrays - that CPU run (interp/JIT) produces the expected buffer contents with zero + second-source effort. GPU results compare against it: ints bit-exact, floats with tolerance + (fastmath on both sides - sec.2 decision 8). Files are `require ?das_metal` + + `static_if builtin_module_exists` guarded, so they compile and no-op cleanly on non-Apple + lanes. Primary gate = **local M-boxes**; the macOS CI lane runs the compile gate, because + its Apple Paravirtual device is unproven for execution. + +- **Gate A - LCOV** on runtime-reached files (`das_metal_boost`, `msl_types` where runtime + code exists). The emitter is compile-time -> census is its coverage proxy. +- **Gate B - construct census.** `generate_msl` records every construct kind it emits + (`table<string>` set: `"kernel_sig"`, `"param.buffer"`, + `"param.builtin.thread_position_in_grid"`, `"op.mul.f32"`, ...). A meta-test unions the + census across all fixtures and asserts equality with the declared supported set, both + directions. Census-record and emit are fused in one helper so they cannot drift; the + golden-snapshot layer catches emissions that bypass it. +- **Leak gate.** Every `tests/metal/` file asserts `metal_live_object_count() == 0` at + exit - Metal objects are invisible to all six das leak detectors, so the shim counts. + +## 5. Cross-backend parity - the kernel-model asymmetry ledger + +The mirror rule (REVIEW.md, both emitters) records deliberate or pending kernel-model +asymmetries HERE - one ledger for both backends (dasSpirv's `ARCHITECTURE.md` points at this +section). The primary correctness oracle remains the CPU-reference run of the same das body, +cheaper and stricter than a second GPU; cross-GPU parity of one source is a nice-to-have now +that both emitters read class members. Current entries: + +- **Class-member authoring is symmetric now.** dasSpirv gained `[spirv_kernel]` classes (this + emitter's model, ported wholesale in the vulkan class-kernel arc) - the original blocker on + stacking both shader annotations on one body is gone. Free-function callees and + devirtualized overrides are symmetric too (msl_emit's devirtualization table was the donor). +- **Multi-kernel-per-class is symmetric now; `family=` surface sharing stays Vulkan-only.** + Both lenses accept N kernel methods per class with `kernel=` picking per instance + (`_mtl_toy.das` / `_vkd_toy.das` are the sibling fixtures). `family=` remains Vulkan's + alone - Metal's `enc_*` builder is the whole generated surface, so there is nothing for a + family to share. +- **Method flattening, scope splicing, and plain-member thread-local STATE are Metal-only.** + `msl_emit` splices method bodies inline (statically devirtualized, value and statement + position) and lowers plain class members to kernel-entry thread-locals (scalar-with-init, + fixed array, carried pointer walk - the state contract is per-thread); `spirv_emit` still + rejects an unannotated member and lowers method calls as calls. If a joined or + state-carrying class ever crosses backends, the same design ports - deliberate asymmetry + until then. +- **The tmm2d tensor-GEMM builtin family is Metal-only (pending, not deliberate).** The + whole-GEMM helpers (`tmm2d_f32_bf16_f32`/`_f16w_f32`/`_q8*_f32` with the bk staging + depths) and the staged tg protocol (`begin/step` in `deva`/`nt`/`devant` flavors, + `tile_tg`, `rowscale`, the `store`/`store_half`/`store_bias` tails, `sgmat_to_half`) + lower MPP `matmul2d` cooperative tensors; the Vulkan analogue would sit on the coopmat2 + classes when a consumer needs it. Until then the CPU-replay bodies are the only + cross-checked twin. +- **Inheritance in the kernel corpus: Vulkan leans on it, Metal does not yet.** The vulkan + classes are base+leaf families (kq GEMV/batch, flash attention, deltanet); the metal + classes are flat, with at least one base duplicated inline (MetalMoeMulMmK6). Dedup + opportunity, not a correctness gap. +- **Grid-literal validation: Vulkan rejects non-int64 ceil-div grid params at macro apply; + the metal lens still carries that silent infer trap.** Pending metal-side fix. +- **cm2 decode-in-load (and its class-method decode form) is Vulkan-only by hardware.** + `[spirv_decode]` - including the method form, where a das-level self is erased from the + rigid 3-param signature so the decode body reads class members (a separate scale plane, + push constants, @workgroup staging) - lowers SPV_NV_cooperative_matrix2 tensor loads. + Metal-4 tensors have no decode-callback analogue; the Metal quant GEMMs stage dequant + through threadgroup memory instead. Deliberate, target-specific - not a pending port. + +## 6. Verification + +- **Standing per-change gate (main tree):** + `daslang dastest/dastest.das -- --test tests/msl --isolated-mode` green on every platform; + `tests/metal` green on Apple; census == declared set both directions; every emitted kernel + compile-gate clean; no `GC APP LEAK`; `metal_live_object_count()` 0. +- **M-box run book (behavioral + parity):** `bin/daslang tests/metal/...` on the M-boxes; the + zen2 side runs the same parity fixtures through dasVulkan. +- **Lint/format:** MCP `format_file` + `lint` on every new `.das`; `cpp_format_file` on the + `.mm`. PR-mode throughout. + +## 7. `@uniform` structs - the kargs form + +A kernel with a dozen scalars used to cost a dozen binds. `@uniform @binding = N ka : KArgs`, +where `KArgs` is a plain das struct, costs one: the struct definition emits into the MSL +preamble, the member becomes `constant KArgs& ka [[buffer(N)]]`, and `ka.field` is one hop. + +**Fields are `int`/`uint`/`float` only.** That restriction is the whole feature: a struct of +4-byte scalars is the same bytes in das and in MSL, with no padding rules to keep in sync, so +the host writes its own struct straight through `setBytes` (`metal_set_bytes`, already bound - +Metal caps it at 4KB and a kargs struct is well under). A `float4` field would be 16-byte +aligned in MSL and shift every field after it; a nested struct brings its own alignment. Both +are refused (`tests/msl/_fail_closed/_fc_ustruct_{field,nested}.das`). + +Host side: `run_compute_1d_kargs` for tests, and dasLLAMA's `kn_bytes` / `kn_kargs` on the +capture rail - under graph capture the bytes are COPIED into a pool, since the caller's kargs is +a stack local long gone by replay time. + +**It costs the GPU nothing.** Every field was already a runtime `constant uint&`; the fold trades +N bindings for one argument-buffer read and the kernel branches on exactly the values it did +before. Nothing about the *shape* of a kernel may become a kargs field, though: a block stride, a +lane width, a codec selector belongs in a per-codec overload or a monomorphized generic, where it +stays a literal in the emitted MSL. Passing one as a value and trusting Metal to inline-and-fold +it back is an assumption, not a guarantee - and the assumption is worth nothing in the kernels +that matter. + +Tests: `tests/msl/test_msl_uniform_struct.das` (definition placement, the single `constant&` +parameter, no per-scalar parameter survives) and `tests/metal/test_metal_uniform_struct.das` +(GPU vs CPU-oracle - the only thing that can prove the layout claim; mutation-verified by +inserting one pad field into the emitted struct). + +## 8. Pointer parameters + +A helper takes a raw pointer: `def stage(var p : half4 const?; n : uint)`. This is what lets the +kernels that stream `unsafe(addr(buf[i]))` through a loop - the split-K "D" attention family, the +K-quant mul_mm trio - share a skeleton at all, since the advancing pointer is the shape and +rewriting it as index math changes AGX register allocation. + +**MSL requires an address space in the signature** (`device half4*` vs `threadgroup half4*`) and +the das type `half4?` carries none - the same type can name an @ssbo interior or a @workgroup +interior. The parameter declares it and the call site proves it: + +- **Unmarked is `device`.** Every pointer stream in the zoo is device, so the common case is + plain das with no marker to learn. +- **`@threadgroup p : T?`** opts into threadgroup memory. Parameter annotations already parse + and keep their values, so this needs no grammar. +- **The emitter derives the space from the argument's provenance and cross-checks it** - an + inline `addr(member[i])`, a pointer local, a pointer parameter being forwarded, or `p + n` + over any of those. Disagreement is a das error naming both sides; provenance it cannot trace + (a thread-space local) is refused rather than guessed. The declared space is never trusted on + its own, so a wrong marker cannot reach the runtime MSL compile as a null pipeline. +- **The POINTEE's const is the MSL const.** `T const?` lowers `device const T*` and reads only; + `T?` lowers `device T*` and marks the source member written, exactly as a `var array<T>` + parameter does. (Writing through it in das additionally needs the handle non-const - `var o : + T?` - because das flows the handle's const onto the dereference.) + +Long-term this is a shortcut for **pointer families** as a language-level feature - a real type +axis rather than an annotation. Parked on `modules/dasLLAMA/followup_general.md`. + +Tests: `tests/msl/_msl_common.das` + `test_msl_functions.das` (signatures, forwarding, inline +`addr` arguments, census), `tests/metal/test_metal_functions.das` (GPU advancing dot + threadgroup +slab max vs a directly-computed expectation), `tests/msl/_fail_closed/_fc_ptr_space_{tg,dev}.das` +and `_fc_ptr_untraceable.das` (both mismatch directions + the untraceable case). diff --git a/modules/dasMetal/REVIEW.md b/modules/dasMetal/REVIEW.md index 208e8f7c2a..abbcf2aa5e 100644 --- a/modules/dasMetal/REVIEW.md +++ b/modules/dasMetal/REVIEW.md @@ -1,7 +1,7 @@ # dasMetal Code Review Checklist **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: -`MASTERPLAN.md`. +`ARCHITECTURE.md`. **Every change under this folder applies `modules/REVIEW_SHADER_EMITTERS.md` too.** diff --git a/modules/dasSpirv/ARCHITECTURE.md b/modules/dasSpirv/ARCHITECTURE.md new file mode 100644 index 0000000000..ca5bfcd7d0 --- /dev/null +++ b/modules/dasSpirv/ARCHITECTURE.md @@ -0,0 +1,137 @@ +# dasSpirv - pure-daslang daslang->SPIR-V shader backend + +**Read `ARCHITECTURE_COMMON.md` (repo root) first - its contract binds this document.** The +checklists that bind a diff here are `REVIEW.md` (this folder) and +`modules/REVIEW_SHADER_EMITTERS.md`. The plan sections and the implementation log this +document grew from are archived at `history/dasSpirv/MASTERPLAN_LOG.md`. + +## 1. Why + +The shape dasSpirv exists to replace: shaders authored as hand-written GLSL +`.vert`/`.frag`/`.comp`, compiled offline with glslangValidator into `.spv` blobs, *both* +committed, and loaded at runtime - two languages, an external SDK, committed binaries, no +code-sharing with the host. dasVulkan carries none of that: no GLSL, no committed `.spv`, and +no glslang or SDK dependency. + +dasGlsl already eliminated exactly this for OpenGL: shaders written in daslang, annotated +`[..._program]`, an `AstVisitor` (`GlslExport`) emits GLSL at compile time into a global +captured by `@@fn`, fed to `glShaderSource`. dasSpirv does the same for Vulkan, but emits +**SPIR-V binary directly from the daslang AST** - no GLSL/glslang intermediary, no LLVM. + +SPIR-V is an SSA IR (typed results, basic blocks, structured control flow, a logical +builder) - the same shape `llvm_jit` already lowers daslang to. So dasSpirv is +**dasGlsl's frontend pattern + llvm_jit's SSA-backend pattern, fused into a SPIR-V emitter**. +Just as `[jit]` lowers daslang->LLVM->native, `[compute_shader]` lowers daslang->SPIR-V->GPU. + +What makes it cheap: drivers do all GPU optimization (we emit naive *valid* SPIR-V - so does +glslang); SPIR-V binary is a self-delimiting word stream (trivial to emit and disassemble); +the backend-agnostic AST tooling (`collect_dependencies`, `collect_used_types`, +`make_visitor`) is reusable verbatim; and dasVulkan already has a GPU-verified `out[i]==i*i` +compute test as a ready-made end-to-end gate. + +## 2. Settled decisions + +1. **Lives in the main tree as `modules/dasSpirv`** - sibling to dasGlsl/dasLLVM, in + PR-protected GaijinEntertainment/daScript, sharing main-tree CI + `daslib/coverage` + + `tests/`. SPIR-V is a general daslang capability (also useful for GL4.6/WebGPU), not + Vulkan-specific. +2. **Fresh frontend** - its own shader annotations (sec.3); dasGlsl is the *design map*, not + a code dependency. Reuse only the generic AST tooling (none of which lives in dasGlsl). + **Zero edits to the shipped dasGlsl/dasOpenGL.** +3. **Test-per-instruction is a hard requirement**, enforced by the opcode census; LCOV covers + the runtime-reached files beside it (sec.4). +4. **SPIR-V 1.3 is the default header version; a feature that needs more raises it.** 1.3 + gives StorageBuffer storage class + `Block` (not the deprecated 1.0 BufferBlock+Uniform + path), lavapipe advertises >= 1.2, and at `<= 1.3` the entry-point interface lists only + Input/Output globals (1.4 requires *all* globals, which would churn every stage's + interface). The version is a per-module field, `SpirvModule.version`: a mesh or task stage + raises it to 1.4 because `SPV_EXT_mesh_shader` requires it, and a few subgroup and + cooperative-matrix ops raise it to 1.5. Every other stage stays at 1.3. + +## 3. Files and emission mechanism + +`modules/dasSpirv` is **pure daslang** (mirrors dasGlsl: a `spirv/` subdir of `.das` files + +CMake resolver rows derived from `.das_module`; no `.shared_module`, no C++). dasVulkan +consumes it via `require spirv/...` and feeds the emitted `array<uint>` (SPIR-V words) to `create_shader_module`. + +| File | Gen/Hand | Purpose | +|---|---|---| +| `spirv/spirv_grammar.das` | **generated** | Opcode + GLSL.std.450 enums + StorageClass/Decoration/BuiltIn/ExecutionMode/Capability token constants, generated from the vendored, pinned SPIRV-Headers grammar JSON. Single source of numeric truth (+ opcode->name table for the disassembler). | +| `spirv/spirv_builder.das` | hand | `SpirvModule` (per-section append buffers), monotonic id allocator, type/constant dedup pools, `emit(section, opcode, ...)`, `get_type_id`, `get_const_id`, `finalize() : array<uint>`. AST-agnostic. | +| `spirv/spirv_types.das` | hand | daslang `TypeDecl` -> SPIR-V type-id (scalars, vec/mat, fixed + runtime arrays, structs, pointers) with layout decorations. Highest-detail correctness file. | +| `spirv/spirv_builtins.das` | hand | The builtin surface the emitter recognizes by name: builtin globals (`gl_*`), the opaque resource marker structs (`sampler2D`/`image2D`/`sampler2DShadow`/...), and the intrinsic stubs (`texture`, `imageLoad`, `barrier`, the atomics, the derivatives, ...). Every stub is `[sideeffects]` so no const-fold or DCE pass can reach it before the annotation runs. | +| `spirv/spirv_emit.das` | hand | `SpirvEmit : AstVisitor` codegen visitor + the `[macro_function] generate_spirv(...)` entry point. Every construct the emitter cannot lower has a rejection override or hits the `value_of`/`ptr_of` backstop, so it becomes a clean compile error rather than a bad blob. | +| `spirv/spirv_reflect.das` | hand | `shared public` reflection vocabulary - `SpirvReflection`, `DescriptorBinding`, `PushConstantRange`, `SpirvDescriptorKind`, `SpirvStageFlags` - plus the versioned `array<uint>` encode/decode the host reads. Names no graphics API, so dasSpirv never depends on dasVulkan. | +| `spirv/spirv_shader.das` | hand | The shader annotations, each a `SpirvShader : AstFunctionAnnotation` subclass carrying its stage: `[compute_shader]`, `[spirv_kernel]` (class-method authoring), `[vertex_shader]`, `[fragment_shader]`, `[mesh_shader]`, `[task_shader]`, `[raygen_shader]`, `[miss_shader]`, `[closest_hit_shader]`; plus the `[spirv_decode]` / `[spirv_combine]` / `[spirv_per_element]` callback annotations. `apply` reserves the blob global and its `_reflect` companion; `fixup` runs dependency collection, calls `generate_spirv`, and sets both inits. | +| `spirv/spirv_dis.das` | hand | Minimal disassembler + opcode-census helper (self-delimiting walk: word0 = `(wordCount<<16)\|opcode`). Symbolic via `spirv_grammar`'s opcode->name table. | +| `generator/gen_spirv_grammar.das` | hand | The mini-generator: reads vendored grammar JSON -> emits `spirv/spirv_grammar.das`. | +| `spirv_headers/*.json` | vendored | Pinned `spirv.core.grammar.json` + `extinst.glsl.std.450.grammar.json`; license in `SPIRV_HEADERS.LICENSE`, provenance in `history/dasSpirv/MASTERPLAN_LOG.md`. | +| `CMakeLists.txt` | hand | `ADD_MODULE_DAS_FROM_DESCRIPTOR(spirv spirv)` (rows derived from `.das_module`) + install rule, modeled on `modules/dasGlsl/CMakeLists.txt`. | + +**SSA backend (llvm_jit template).** `SpirvEmit` carries `e2v : table<Expression?;uint>` +(Expression->result-id), `v2v : table<Variable?;uint>` (Variable->pointer-id), +`ite2blocks`/`loop_stack` for control flow, and a hand-maintained `cur_block_terminated` +(SPIR-V has no builder to query - set on every terminator, cleared on `OpLabel`; guards +mirror llvm_jit's `current_block_terminates()`). Section buffers concatenate at `finalize` +in SPIR-V's mandated order: header(5) -> capabilities -> extensions -> ext-imports -> +memory-model -> entry-points -> exec-modes -> debug -> decorations -> types+constants+global-vars +(interleaved, define-before-use, deduplicated) -> functions. + +**Capture mechanism (dasGlsl analog).** dasGlsl's `fixup` sets +`glob.init = new ExprConstString(value := text)`; ours builds an `ExprMakeArray` of +`ExprConstUInt` (one per SPIR-V word) for an `array<uint>` global named by the annotation's +`name=` argument, or `` "{func.name}`spirv" `` when it has none, beside a `{name}_reflect` +companion holding the encoded reflection. `generate_spirv` is a standalone `[macro_function]` +called by **both** `fixup` and the unit tests - so opcode assertions hit the real codegen path +without macro plumbing. + +## 4. Test architecture - "every emitted instruction has a test" + +Three behavioral layers + two enforcement gates (all in main-tree `tests/spirv/` except GPU): + +1. **Opcode-assertion units.** Each test compiles a tiny shader fixture, calls + `generate_spirv`, runs `spirv_dis` to a structured instruction list, and asserts the + expected numeric opcode present/absent/operand-at-offset/decoration present. One test per + emittable instruction kind. A `_golden/` disassembly snapshot of our own validated output + is the forward regression guard - byte-identical where id allocation is stable, + id-isomorphic (`check_iso`) where it is not. +2. **`spirv-val` gate.** Every blob the suite produces is validated with the SPIRV-Tools + `spirv-val` resolved through `VULKAN_SDK`. Soft-skip if absent locally, hard-required in + CI - the real correctness oracle for structured-CFG and define-before-use bugs. +3. **Real-driver behavioral regression.** A one-call framework in dasVulkan, + `run_compute_spirv(words, n) : array<uint>` over `compute_boost`, runs any emitted blob in + ~2 lines; `compute_image_rgba8` + `assert_pixels_exact` are its image-readback twin. The + content is procedural, so the expected pixel is CPU-computable and the gate asserts exact + pixels rather than inequalities. Primary gate = local real GPU; CI = lavapipe software + (GitHub-hosted runners have no GPU, so there is no real-GPU CI lane). + +- **Fail-closed fixtures.** One fixture per rejection path lives in `tests/spirv/_fail_closed/` + and its error text is asserted in `tests/spirv/test_fail_closed.das`. The subdir is not + globbed, so the AOT build never tries to compile a fixture whose whole point is to fail. +- **Gate A - LCOV coverage.** `dastest --cov-path spirv.lcov` (`daslib/coverage.das`) measures + the runtime-reached files (`spirv_builder`, `spirv_types`, `spirv_dis`). It cannot measure + the emitter: `generate_spirv` runs at COMPILE time and LCOV counts runtime line hits, so + `spirv_emit.das` reads 0/N by construction. Shader fixtures carry `[marker(no_coverage)]` - + coverage instrumentation injects calls the emitter cannot lower. +- **Gate B - opcode census (the emitter's coverage proxy).** `spirv_dis` aggregates the opcode + set across *all* fixtures; a meta-test asserts it equals the declared "supported opcodes" + set, both directions. This catches what LCOV cannot - an opcode whose emit line ran but was + never *asserted*. + +## 5. Cross-backend parity - the kernel-model asymmetry ledger + +The mirror rule (REVIEW.md, both emitters) keeps the kernel model symmetric with the MSL +emitter. The ledger of deliberate/pending asymmetries is shared - ONE list, not two - and +lives in `modules/dasMetal/ARCHITECTURE.md` under the same heading. + +## 6. Verification + +- **Standing per-change gate (main tree):** + `daslang dastest/dastest.das -- --test tests/spirv --cov-path spirv.lcov --isolated-mode` + -> all green, opcode census == declared set both directions, every blob spirv-val-clean, no + `GC APP LEAK`. +- **Disassembly check:** an emitted module is dumped via `spirv_dis` (symbolic) and diffed + against external `spirv-dis` as ground truth. +- **Real-driver gate (dasVulkan):** the integration suite under lavapipe and the local real + GPU. +- **Lint/format:** MCP `format_file` on every new `.das`; `lint` clean. Both repos PR-mode. diff --git a/modules/dasSpirv/REVIEW.md b/modules/dasSpirv/REVIEW.md index 85fa264da4..7de7c9b95d 100644 --- a/modules/dasSpirv/REVIEW.md +++ b/modules/dasSpirv/REVIEW.md @@ -2,7 +2,7 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist. Shared emitter rules: `modules/REVIEW_SHADER_EMITTERS.md` - apply that list with this one.** -Architecture doc: `MASTERPLAN.md`. +Architecture doc: `ARCHITECTURE.md`. - **A diff that adds an emitter capability also adds its tests under `tests/spirv/` (repo root), in the same change:** a `_golden/` disassembly or equivalence fixture, and presence diff --git a/modules/dasVulkan/ROADMAP.md b/modules/dasVulkan/ROADMAP.md index 9fa1f7a5dc..1b09c6b495 100644 --- a/modules/dasVulkan/ROADMAP.md +++ b/modules/dasVulkan/ROADMAP.md @@ -241,7 +241,7 @@ are exotic extensions; revisit case-by-case if a consumer needs one. `cmake/DasVulkanCompileShader.cmake` (+ its `include` at `CMakeLists.txt:21`) is the last GLSL-era artifact: a consumer-facing glslangValidator/spirv-opt macro, unused by the repo itself since the dasSpirv migration ("ZERO GLSL, ZERO -committed .spv" - dasSpirv MASTERPLAN). Decision (2026-07-16): shaders are +committed .spv" - `modules/dasSpirv/ARCHITECTURE.md`). Decision (2026-07-16): shaders are always das-authored, the helper goes; deletion deferred ~2 weeks in case a consumer objects. Delete the file + the include line; nothing else references it. diff --git a/plans/msl-flatten.md b/plans/msl-flatten.md index 0dd4f047bb..0bfb0e693c 100644 --- a/plans/msl-flatten.md +++ b/plans/msl-flatten.md @@ -99,6 +99,6 @@ member types keep the hard error (the old field gate, now scoped to what state c production arms on MoeGemvArgs kargs (lab twins keep historical layouts per arm), dead tail duplicates dropped, main end-to-end to the leak assert, 19/19 checks rel 0 - [ ] spirv_emit parity: dasSpirv lowers class kernels (devirtualized method CALLS) but not - the flatten/scope-splice or state-member forms - ledgered in dasMetal/MASTERPLAN.md - sec.Cross-backend parity; if a joined class ever crosses backends, the same flatten design + the flatten/scope-splice or state-member forms - ledgered in modules/dasMetal/ARCHITECTURE.md + sec.5 Cross-backend parity; if a joined class ever crosses backends, the same flatten design applies there (not in this arc's scope) diff --git a/skills/internal/doc_archiving.md b/skills/internal/doc_archiving.md index af44cb04a9..3da8102916 100644 --- a/skills/internal/doc_archiving.md +++ b/skills/internal/doc_archiving.md @@ -12,7 +12,7 @@ sweep turns up stale plan docs. - **STAY:** docs cited as *current* spec / recipe / contract by shipped code, tests, tutorials, skills, or docs - living contracts (e.g. `modules/dasSQLITE/PROVIDER_CONTRACT.md`), durable records that self-describe as - canonical (`MASTERPLAN.md`), README-indexed operative runbooks + canonical, README-indexed operative runbooks (`tune_for_this_box.md`), active-arc plans, and results ledgers (`results.md`). - **The test, per inbound reference:** is it *operative* (a reader needs the doc to act now) or *historical context* (a comment narrating provenance)? Only docs whose @@ -33,7 +33,7 @@ sweep turns up stale plan docs. search plus the ledger. 5. Fix the moved docs' own relative links that point back at staying files (e.g. a moved doc's `results.md` link becomes `../../benchmarks/sql/results.md`). -6. If the area keeps a living index doc (`MASTERPLAN.md`, a module `README.md`, a staying +6. If the area keeps a living index doc (`ARCHITECTURE.md`, a module `README.md`, a staying contract doc), add a short note **in that doc**: what was archived and where. 7. Append one line per doc to the **Archive log** at the bottom of `history/README.md`: `- YYYY-MM-DD \`old/path\` -> \`history/new/path\` - one-line hook`. The ledger is how diff --git a/skills/review_md.md b/skills/review_md.md index 30ddcfe19c..f162a8c53c 100644 --- a/skills/review_md.md +++ b/skills/review_md.md @@ -29,10 +29,9 @@ or a checklist that restates contract text instead of pointing - is a self-revie fixed like any other. An external repo that adopts `REVIEW.md` vendors a copy of `REVIEW_COMMON.md` at its root, so the pointer resolves there too. -`<ARCH-DOC>` is the module's own design document - `ARCHITECTURE.md` (dasLLAMA), -`MASTERPLAN.md` (dasMetal), or the module's `CLAUDE.md` when no separate design doc exists. -Name it concretely; a module whose rules keep wanting a rationale home probably needs to -grow one. +`<ARCH-DOC>` is the module's own design document - `ARCHITECTURE.md` (dasLLAMA, dasMetal, +dasSpirv), or the module's `CLAUDE.md` when no separate design doc exists. Name it +concretely; a module whose rules keep wanting a rationale home probably needs to grow one. ## The executable half - REVIEW.das @@ -61,8 +60,10 @@ whatever sits there today. ## Writing the entries -- Lead each entry with the **bold testable claim**, tail with the consequence ("... is a - defect") where it sharpens the check. +- Lead each entry with the **bold ban or duty binding the diff** ("never X - do Y instead"; + "a diff that changes X also does Y, in the same change"; "a diff that adds X ... is a + defect"). A sentence describing how the system is belongs in the architecture doc, not + here. No spare words: a tail that restates what the head already binds is cut. - The list is flat - no section headers, blank line between entries. A grouping that keeps wanting a header is a separable concern: give it its own file and route to it from the opening (`modules/REVIEW_SHADER_EMITTERS.md` and `modules/dasLLAMA/REVIEW_GPU.md` are the diff --git a/tests/metal/REVIEW.md b/tests/metal/REVIEW.md index 3dfe07a1b1..28cd5a7dd9 100644 --- a/tests/metal/REVIEW.md +++ b/tests/metal/REVIEW.md @@ -1,7 +1,7 @@ # dasMetal kernel tests Code Review Checklist **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: -`modules/dasMetal/MASTERPLAN.md`. +`modules/dasMetal/ARCHITECTURE.md`. **A CPU-oracle test that reuses one kernel-class instance across thread ids is a defect of the test, not of the kernel - build a fresh instance for every thread id.** A CPU-oracle test diff --git a/tests/metal/test_metal_mul_ab.das b/tests/metal/test_metal_mul_ab.das index 447454feed..901f34562d 100644 --- a/tests/metal/test_metal_mul_ab.das +++ b/tests/metal/test_metal_mul_ab.das @@ -1,7 +1,7 @@ options gen2 options indenting = 4 -// Real-GPU behavioral gate vs the CPU-reference oracle (MASTERPLAN test architecture, layer 3). +// Real-GPU behavioral gate vs the CPU-reference oracle (ARCHITECTURE.md sec.4, layer 3). // The kernel body is ordinary daslang, so the same method runs on the CPU: a driver loop sets // gl_GlobalInvocationID and calls the method on an instance whose members are plain arrays — // that run IS the expected output. The GPU result must match exactly (integer-valued floats). diff --git a/tests/msl/REVIEW.md b/tests/msl/REVIEW.md index dbec5b445e..c97ec7d54a 100644 --- a/tests/msl/REVIEW.md +++ b/tests/msl/REVIEW.md @@ -1,7 +1,7 @@ # MSL emitter fixtures Code Review Checklist **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: -`modules/dasMetal/MASTERPLAN.md`. +`modules/dasMetal/ARCHITECTURE.md`. **Weakening the census check is a defect.** The check runs both directions: every kind in `declared_msl_census` is emitted by some fixture, and every kind any fixture emits is declared From 655f61d65e8a9209c5f21e91b6c424358f448861 Mon Sep 17 00:00:00 2001 From: Boris Batkin <bbatkin@gmail.com> Date: Thu, 27 Aug 2026 02:57:16 -0700 Subject: [PATCH 10/15] dragon round: 180 findings judged and applied across 60 documents Five fix clusters applied the dragons' repairs - ambiguities spelled, slogans inverted, duplicates cut to their surviving copy, terms defined in place, arch docs numbered and mood-cleaned. Three wrong dragon premises refuted with code evidence and skipped. Cross-cluster batch: the playground checklist created and routed, the dasGlsl edit ban and metal leak duty landed beside their mechanisms, shipped-exe rules trimmed to the gate's residue, the sample-verify ban grounded in expectations.json. Gates re-run green (utils, src/builtin, nano, dasLLAMA, dasMetal, dasVulkan). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014fKJGuUL8tP58Y6NvoVeSn --- .github/workflows/REVIEW.md | 15 +- ARCHITECTURE_COMMON.md | 28 +- REVIEW_COMMON.md | 159 +++---- daslib/ARCHITECTURE.md | 390 ++++++++++-------- daslib/REVIEW.md | 104 ++--- daslib/REVIEW_LINQ.md | 75 ++-- modules/dasImgui/REVIEW.md | 16 +- modules/dasLLAMA/REVIEW.das | 6 +- modules/dasLLAMA/REVIEW_GPU.md | 100 +++-- modules/dasLLAMA/benchmarks/REVIEW.md | 70 ++-- modules/dasLLAMA/followup_vulkan.md | 28 ++ modules/dasMetal/ARCHITECTURE.md | 218 ++++------ modules/dasMetal/REVIEW.md | 15 +- modules/dasSpirv/ARCHITECTURE.md | 39 +- modules/dasSpirv/REVIEW.md | 3 + modules/dasVulkan/CLAUDE.md | 2 +- modules/dasVulkan/REVIEW.md | 2 +- modules/dasVulkan/tutorials/REVIEW.md | 4 +- nano/REVIEW.md | 16 +- site/REVIEW.md | 104 ++--- site/tests/playground/REVIEW.md | 9 + skills/review_md.md | 162 ++++---- src/builtin/REVIEW.md | 26 +- tests/metal/REVIEW.md | 3 + utils/REVIEW.md | 17 +- utils/das-fmt/REVIEW.md | 6 +- utils/dascov/REVIEW.md | 6 +- utils/dasllama-server/REVIEW.md | 8 +- utils/daspkg/REVIEW.md | 37 +- utils/detect-dupe/REVIEW.md | 6 +- utils/internal/dasllama-ladder/REVIEW.md | 78 ++-- utils/internal/dasweb-buildd/REVIEW.md | 61 +-- .../internal/dasweb-verify/browser/REVIEW.md | 18 +- utils/lsp/REVIEW.md | 4 +- web/examples/ui/REVIEW.md | 6 +- 35 files changed, 962 insertions(+), 879 deletions(-) create mode 100644 site/tests/playground/REVIEW.md diff --git a/.github/workflows/REVIEW.md b/.github/workflows/REVIEW.md index e975aa57b7..086327d3c9 100644 --- a/.github/workflows/REVIEW.md +++ b/.github/workflows/REVIEW.md @@ -3,11 +3,12 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `skills/internal/preflight.md` (repo root). -**A diff that deletes a per-PR gate step in `doc.yml` or `extended_checks.yml`, adds -`continue-on-error` to it, narrows its `if:`, weakens its command, or shrinks the set of -tracked source files it checks is a defect.** Each such step enforces its rule automatically, -with no reviewer involved - `skills/internal/preflight.md` sec."doc.yml - the gates", -sec."extended_checks.yml". +**A diff that deletes a per-PR gate step - a workflow step that runs on `pull_request` and +fails the lane when it finds a defect - adds `continue-on-error` to one, narrows its `if:`, +weakens its command, or shrinks the set of tracked source files it checks is a defect.** Each +such step enforces its rule automatically, with no reviewer involved - +`skills/internal/preflight.md` sec."doc.yml - the gates", sec."extended_checks.yml". -**A diff that adds or changes a per-PR gate step in `doc.yml` or `extended_checks.yml` makes -that step fail the lane on every defect in what it checks, and on nothing else.** +**A diff that adds or changes a per-PR gate step - a workflow step that runs on +`pull_request` and fails the lane when it finds a defect - makes that step fail on every +defect in what it checks, and on nothing else.** diff --git a/ARCHITECTURE_COMMON.md b/ARCHITECTURE_COMMON.md index 7065b15f26..9d4965a4f5 100644 --- a/ARCHITECTURE_COMMON.md +++ b/ARCHITECTURE_COMMON.md @@ -3,29 +3,31 @@ Every folder-scoped `ARCHITECTURE.md` states how that folder's system is built and why. It has two readers: the writer, who reads it BEFORE writing code in the folder, and the reviewer, who follows a rule's citation into it. This file is the part all architecture docs share; its -sibling `REVIEW_COMMON.md` (repo root) is the checklists' contract. The two split one grammar: -a ban or duty on a diff lives in the folder's `REVIEW.md`; a statement of what is true lives -in its `ARCHITECTURE.md`; what happened lives in `/history`. +sibling `REVIEW_COMMON.md` (repo root) is the checklists' contract. **An architecture doc is written in the present tense: what is true of the current tree, and -why that shape wins.** The mood test is mechanical. A sentence that binds a diff with -must/never/"is a defect" is a rule - move it to the folder's `REVIEW.md`. A dated or -past-tense passage is history - archive it to `/history` (`skills/internal/doc_archiving.md`). -A promise, phase, or plan is planned work - move it to the folder's follow-up ledger. +why that shape wins.** The mood test: a sentence that binds a diff with must/never/"is a +defect" is a rule - move it to the folder's `REVIEW.md`. A passage that narrates what +happened - past tense, or dated as a record of an event - is history - archive it to +`/history` (`skills/internal/doc_archiving.md`); a `probe-verified <date>` tag or a version +stamp records no event and stays. A promise, phase, or plan is planned work - move it to the +folder's follow-up ledger. -**What belongs:** +**What belongs in an architecture doc: file charters, mechanisms, definitions, inventories, +and the exception ledger. Nothing else.** - **File charters** - one line per file: what it owns, what never lands in it. - **Mechanisms** - how a subsystem works, and the reason its shape wins over the obvious alternative. - **Definitions** - terms the folder's rules and code cite, each stated once. -- **Inventories** - the folder's override knobs, registered families, sanctioned cases: lists - a rule checks a diff against. +- **Inventories** - lists a rule checks a diff against: override knobs (an env variable or + setting that changes what a run compiles, tunes, or emits), registries a diff adds a row to, + sanctioned cases. - **The exception ledger** - the cases a `REVIEW.md` rule fires on that are ruled acceptable, - each with its reason. A rule stays absolute; this is where its sanctioned violations live. + each with its reason. -**Sections are numbered, and rules cite them by section.** A section number, once cited, is -never reused for different content - append new sections, do not renumber. +**Sections are numbered, and rules cite them by section.** A section number is never reused +for different content: append new sections, never renumber. **A fact that a rule or a code comment cites is load-bearing: it must stay true.** The same-change duty that keeps it true belongs in the folder's `REVIEW.md`, not here. diff --git a/REVIEW_COMMON.md b/REVIEW_COMMON.md index 94337f6e4d..dff8d30096 100644 --- a/REVIEW_COMMON.md +++ b/REVIEW_COMMON.md @@ -7,14 +7,16 @@ folder's criteria, and its opening block points here. An external repo that adop vendors a copy of this file at its root. A statement's home is the folder's architecture doc - `ARCHITECTURE_COMMON.md` (repo root) is that document type's contract. -**What stays in a checklist: rules that can be checked against a diff. Nothing else.** -A reader must be able to apply every rule **without reading the code and without prior -knowledge of the folder.** A term of art is defined in place, in a dozen words, or the rule is -rewritten around plain words. A rule may cite the folder's own architecture doc by section for -the reason behind it; it may not require that section to be read before the rule can be -applied, and it may not quote or cite any other rule document - coupling between rule -documents is done by ROUTING ("a diff touching X applies that checklist too", "a `[tune]` -change is reviewed with `skills/tune.md`"), never by restating. +**What stays in a checklist: rules that can be checked against a diff. Nothing else.** A +reader must be able to apply every rule **without reading the code and without prior knowledge +of the folder.** + +**A term of art is defined in place, in a dozen words, or the rule is rewritten around plain +words.** + +**A checklist never quotes, cites, or restates another rule document** - coupling between rule +documents is ROUTING ("a diff touching X applies that checklist too", "a `[tune]` change is +reviewed with `skills/tune.md`"). **Every checklist reviews itself: a rule a reviewer cannot apply as written is a defect of the checklist.** Mark it like any other finding - a checklist defect blocks nothing, but its fix (a @@ -22,20 +24,20 @@ rewrite or a move, never silent tolerance) lands in the same batch as the round' **New functionality ships with tests - same PR, no follow-up promises.** A new or changed reachable branch ships a test that fails without it; a diff that adds a branch no test -distinguishes is a defect. The audit procedure - including the negative control that settles -"would it fail?" - is `skills/tdd_audit.md`. +distinguishes is a defect. The audit procedure - including how to settle "would this test fail +without the change?" - is `skills/tdd_audit.md`. **A rule that a test, a lint, or the folder's `REVIEW.das` enforces is deleted.** Automation replaces the rule; the checklist keeps at most "weakening that check is a defect." A rule that COULD be automated is a lint or `REVIEW.das` candidate - say so in the review round. -**A `REVIEW.das` beside a checklist is that checklist's mechanical gate** - a standalone -script (`daslang <folder>/REVIEW.das`, run from the repo root; support library -`dastest/review_gate`) that prints `path: finding` lines (`path:line: finding` when a -line is known) and exits nonzero on any. +**A `REVIEW.das` runs standalone and reports in one shape** - `daslang <folder>/REVIEW.das` +from the repo root prints one `path: finding` line per finding (`path:line: finding` when a +line is known) and exits nonzero when any fired. A gate that reports otherwise is a defect. +Support library: `dastest/review_gate`. -**The gate is fail-fix, like a test suite: it runs before every review, and a red gate -stops the round until fixed.** +**Every review round runs the folder's `REVIEW.das` before it reads the diff, and a red gate +stops the round until the gate is green.** **A `REVIEW.das` without its sibling `REVIEW.md` is a defect.** @@ -47,58 +49,73 @@ that handshake is the system working. **A rule lives in the checklist of the narrowest folder that contains its trigger.** Subfolder-scoped rules move to that subfolder's own `REVIEW.md` - creating it is never an -objection; checklists are cheap, thousands can sit in a repository and few fire per PR. The -parent routes strays by KIND ("a `[test]` file, wherever the diff puts it, answers to the -`tests/` subfolder's checklist"). A rule whose trigger lies entirely outside the folder can -never fire and is deleted - better absent than dead. - -**Form, and it is a hard limit:** - -- **One rule is one short paragraph - bold criterion first, blank line between rules.** A - rule that needs more than that is describing how to write code, not how to review it. - Split it or move it. Procedure - a files-to-touch how-to - is not a rule. -- **A rule tells the diff what it must or must not do - it never describes what the system - is.** Two forms exist. A ban says what the system must not be: "never X - do Y instead", - or "a diff that adds X ... is a defect" - the defect spelling binds the reviewer's - verdict, so it is a ban, not a statement. A duty says what a change must also do: "a - diff that changes X also does Y, in the same change". A sentence that describes how the - code is ("state is a stack", "X and Y agree") is a statement, not a rule: the reader has - to work out the check, and the first violating diff turns the sentence false. Rewrite it - as the ban or duty of the diff that could break it, or move it to the architecture doc. -- **A rule is written in plain English - short sentences, common words, no idioms.** A - rule that needs a second read is a defect of the rule. -- **A rule spends no spare words.** A clause that restates what the head already binds - a - defect verdict repeating its own duty, a doubled negation, a filler phrase - is cut. The - shortest wording that keeps the meaning wins. -- **No sections.** A section header is a bucket asking to be filled: a grouping either - decouples into the flat list or is a separable concern that earns its own routed file. -- **Rules are unnumbered and unnicknamed** - numbering invites citation. Anything that needs - a stable reference lives in the checklist's architecture doc. -- **A path in a rule resolves against the checklist's own folder.** A path from the repo - root starts with a root directory the folder does not contain (`modules/...`, `daslib/...`) or - says "`(repo root)`"; when both readings exist - a `tests/` beside the checklist and one at - the root - the rule must say which. -- **Cite files by name; cite the folder's own architecture doc by section.** - No file cites a rule in a checklist by number, position, or nickname - quote the rule's - words instead. A quote states the reason the code is shaped this way and survives any - rewording; a position points at whatever sits there today. -- **Name the API a rule is about; never name an example of it.** A rule governing specific - functions or files must name them or it cannot be checked - that name is the criterion. An - illustrative aside has no such excuse: nothing keeps it in sync, and a stale example is - worse than none. -- **A ban names its replacement.** "Never X" is half a rule; " - do Y instead" completes it - for a few words. -- **One sentence of WHY is allowed where it makes the criterion decidable; anything longer - belongs in the architecture doc.** No history, no PR numbers, no direction of travel; - planned work lives in the folder's follow-up ledger (a checklist that keeps one elsewhere - names it in its opening). -- **The placement block, when a checklist carries one, closes the file**: one bold criterion - ("Placement - one file, one line: a diff keeps each file inside its line, and a new file - adds its line here, with its tests, in the same change.") over `` `file` `` - role lines, - optionally a bare never-phrase ("Zero network"). A clause with its own defect-verdict or - same-change obligation is a rule, and it lives in the flat list above. - -**Adding a rule starts with reading the whole checklist.** A rule appended by an author who -has not just read the file is how checklists rot - duplication, drift, and homeless placement -all start there. And when the rule you just wrote is the longest paragraph in the file, stop: -it is almost certainly several rules, an exception in disguise, or an essay. +objection; checklists are cheap, thousands can sit in a repository and few fire per PR. A rule +whose trigger lies entirely outside the folder can never fire and is deleted. + +**A parent checklist routes strays by KIND, not by location** - "a `[test]` file, wherever the +diff puts it, answers to the `tests/` subfolder's checklist". + +**One rule is one short paragraph - bold criterion first, blank line between rules.** A rule +that needs more than that is describing how to write code, not how to review it. Split it or +move it. + +**A checklist entry that lists files to touch is a defect** - a files-to-touch how-to is +procedure, and it moves to the architecture doc or the skill that owns the task. + +**A rule tells the diff what it must or must not do - it never describes what the system is.** +A ban says what the system must not be: "never X - do Y instead", or "a diff that adds X ... +is a defect" - the defect spelling binds the reviewer's verdict. A duty says what a change +must also do: "a diff that changes X also does Y, in the same change". A sentence that +describes how the code is ("state is a stack", "X and Y agree") is a statement: the first +violating diff turns it false. Rewrite it as the ban or duty of the diff that could break it, +or move it to the architecture doc. + +**A rule is written in plain English - short sentences, common words, no idioms.** A rule that +needs a second read is a defect of the rule. + +**A rule spends no spare words.** A clause that restates what the head already binds - a +defect verdict repeating its own duty, a doubled negation, a filler phrase - is cut. + +**No sections.** A section header is a bucket asking to be filled: a grouping either decouples +into the flat list or is a separable concern that earns its own routed file. + +**Rules are unnumbered and unnicknamed** - numbering invites citation. Anything that needs a +stable reference lives in the checklist's architecture doc. + +**A path in a rule resolves against the checklist's own folder.** A path from the repo root +starts with a root directory the folder does not contain (`modules/...`, `daslib/...`) or says +"`(repo root)`"; when both readings exist - a `tests/` beside the checklist and one at the +root - the rule must say which. + +**Cite files by name; cite the folder's own architecture doc by section, and never require +that section to be read before the rule can be applied.** No file cites a checklist rule by +number, position, or nickname. A file outside the rule documents - source, commit message, +doc - quotes the rule's words; a rule document routes to the checklist instead, and never +quotes or restates its rule. A criterion whose home is another folder's architecture doc is +restated in place and cited nowhere. A quote states the reason the code is shaped this way and +survives any rewording; a position points at whatever sits there today. + +**Name the API a rule is about; never name an example of it.** A rule governing specific +functions or files must name them or it cannot be checked - that name is the criterion. An +illustrative aside has no such excuse: nothing keeps it in sync, and a stale example is worse +than none. + +**A ban names its replacement.** "Never X" is half a rule; " - do Y instead" completes it for +a few words. + +**One sentence of WHY is allowed where it makes the criterion decidable; anything longer +belongs in the architecture doc.** No history, no PR numbers, no direction of travel; planned +work lives in the folder's follow-up ledger (a checklist that keeps one elsewhere names it in +its opening). + +**The placement block, when a checklist carries one, closes the file**: one bold criterion +("Placement - one file, one line: a diff keeps each file inside its line, and a new file adds +its line here, with its tests, in the same change.") over `` `file` `` - role lines, optionally +a bare never-phrase ("Zero network"). A clause with its own defect-verdict or same-change +obligation is a rule, and it lives in the flat list above. + +**Adding a rule starts with reading the whole checklist** - duplication, drift, and homeless +placement all start with a rule appended by an author who had not just read the file. + +**A rule the diff adds that is longer than every rule already in the file is split, its +exception dissolved, or its extra prose moved to the architecture doc.** diff --git a/daslib/ARCHITECTURE.md b/daslib/ARCHITECTURE.md index a6a980e957..ae914280e2 100644 --- a/daslib/ARCHITECTURE.md +++ b/daslib/ARCHITECTURE.md @@ -1,10 +1,9 @@ # daslib architecture notes -Design rationale a maintainer cannot recover from the code alone. One section per -module; entries are anchored to symbols. Grown by the comment-sweep rescue passes - -an entry lands here only when no name, shape, or test can carry it. +Design rationale a maintainer cannot recover from the code alone. One numbered section per +module; entries are anchored to symbols. -## perf_lint +## 1. perf_lint - **Two parallel loop spaces.** `loop_depth`/`in_closure` (+ `VarStackEntry.depth`) is the contract for every loop rule: any closure body is deferred, hence not "in" the loop. @@ -30,8 +29,8 @@ an entry lands here only when no name, shape, or test can carry it. the adjacency and mis-applies the stash. - **`add_ptr_ref` is the qmacro-splice marker.** PERF023 rests on `apply_qrules` wrapping every `$e(...)` tag in `add_ptr_ref(...)`; matching the name suffices because the - templates_boost wrapper is its only producer post-expansion. Its depth tracking must - fire before the `in_closure` gate - the rules block is itself a closure. + templates_boost wrapper is its only producer post-expansion. Its depth tracking fires + before the `in_closure` gate - the rules block is itself a closure. - **Hot-path walk (PERF026-028).** `HotCache` keeps per-function scan results sliced into two flat arrays; `sinks_span_by_fn` doubles as the presence key and the two tables are strictly co-populated (one insert site each, adjacent). The visited set keys on @@ -39,10 +38,10 @@ an entry lands here only when no name, shape, or test can carry it. call sites must report twice, so a function is re-walked once per distinct anchor - a cost taken for per-call-site diagnostics. Generated/generic frames re-anchor to the caller side because their own `at` points into the template. A call to any function - with a `@scratch` parameter prunes that callee's whole subtree from the walk - the - blessing is deliberately TRANSITIVE: a scratch-taking helper is trusted wholesale - (narrowing it to the sink floods real scratch-heavy code - dasLLAMA - with hundreds - of deep-resize findings on buffers the helper family sizes correctly). + with a `@scratch` parameter prunes that callee's whole subtree from the walk - the prune + is deliberately TRANSITIVE, removing the callee's whole subtree and not just the sink, + because narrowing the prune to the sink buries scratch-heavy callers in deep-resize + findings on buffers the helper family sizes correctly. - **PERF030 release credits are consume-once**: a `delete x` licenses exactly one subsequent `x <- ...`, then the warning re-arms; the lowered delete family is matched by name in its four spellings (`_::finalize`, `finalize`, ``builtin`finalize``, @@ -78,7 +77,7 @@ an entry lands here only when no name, shape, or test can carry it. cast, hoist over the ternary) stay valid for every instantiation, so they fire everywhere. -## lint_config +## 2. lint_config - **`.lint_config` is parsed once per process.** `[lint_macro]` apply() fires once per module in the require chain, so an uncached read re-parses the repo file N times per @@ -88,7 +87,7 @@ an entry lands here only when no name, shape, or test can carry it. chosen tradeoff: the in-tree escape hatch is `nolint:PERF006,LINT019` on lines whose code fires only in downstream compiles; per-code vouching would need new syntax. -## lint +## 3. lint - **`lint_error` reports only nodes whose lineinfo belongs to the current function's own file.** An inliner splice or macro-manufactured node from another file gets its @@ -100,9 +99,9 @@ an entry lands here only when no name, shape, or test can carry it. and declarations plus string literals (signatures and name-based lookups leave only a literal). Self-reference is pre-inserted so it cannot keep a private type alive. Functions are checked only when inlining is off - inlining splices small private - callees away and leaves husks indistinguishable from dead code. Public symbols, - lifecycle/macro hooks, virtual methods, and generic bodies are skipped: their callers - are invisible to lint. + callees away and leaves a body whose only callers were inlined away, indistinguishable + from dead code. Public symbols, lifecycle/macro hooks, virtual methods, and generic + bodies are skipped: their callers are invisible to lint. - **Argument rules skip what the author does not own, and converge leaf-first.** LINT012/013/014/023 stay silent on class methods, `[extern]` stubs, `finalize` overloads, and address-taken functions; LINT014 also skips a parameter whose mutability @@ -131,8 +130,12 @@ an entry lands here only when no name, shape, or test can carry it. - **`canVisitWithAliasSubexpression` returns false on purpose** - `assume` keeps the template block at the definition site and clones it per use; the block is never invoked there, so descending would report its arguments unused. +- **C++ mirror pairs**: `lint022_optimized` / `lint022_calls_may_be_inlined` <-> + `Program::getOptimize` / `Program::patchInline`; `stale_scan_line` <-> + `rtti_is_nolint_suppressed`; `is_inline_temp_name` <-> `INLINE_TEMP_PREFIX`. Nothing + fails when one side moves alone. -## style_lint +## 4. style_lint - **Lint runs under policies that skip const-folding and leave some types unset.** Take a variable's type from `Variable._type`, not an `ExprVar`'s; expect `T('c')` to stay an @@ -141,8 +144,10 @@ an entry lands here only when no name, shape, or test can carry it. - **Rules whose forms are identical in the AST read the source span instead** - `defer <|`, generators, STYLE003's `$()`, STYLE028's three invoke spellings. STYLE005 uses LineInfo identity: the parser gives braceless and postfix forms a synthetic block sharing the - terminator's LineInfo, so `blk.at != inner.at` <=> the user wrote braces. Both oracles - break silently when lowering or LineInfo assignment changes. + terminator's LineInfo, so `blk.at != inner.at` <=> the user wrote braces. Both of these + tests - the source-span read and the LineInfo identity check - are the rule's only + evidence for the distinction, and both break silently when lowering or LineInfo + assignment changes. - **STYLE024/025 mirror the compiler's unsafe rules with a deliberate asymmetry: in doubt, mark "needs unsafe".** A false "needs" costs a missed note; a false "redundant" tells the user to delete a wrap the compiler requires. `let v & = unsafe(expr)` is the one @@ -157,7 +162,7 @@ an entry lands here only when no name, shape, or test can carry it. - **Metric rules attribute code to the module that can fix it.** STYLE037/038 walk the `fromGeneric` chain to its root (`[template]` generics instantiate INTO the caller); generated carriers stay unmetered; closures are not length-checked separately - the - host trips first. The 80-line default ~ p97 of this tree's function-length distribution. + host trips first. - **STYLE041 is sound because of a reference count, not the structural scan.** The scan classifies writes, plumb reads, and the single consuming `if`, arming the candidate with that count; `preVisitExprVar` counts every reference in the function, and the warning @@ -188,16 +193,18 @@ an entry lands here only when no name, shape, or test can carry it. an `expect` directive anywhere makes the file a compile-error fixture (its position is semantic to dastest), whereas `lint-skip-file` deeper than the header would let quoted prose unlint a file. -- **The seven `blk.list` index rescans are O(block^2) by choice** - the per-statement - visitor callback carries no index, and a shared index stack would have to push/pop with - nested blocks; block statement lists are short. Do not "fix" one helper by caching - state across callbacks. +- **The `blk.list` index rescans are O(block^2) by choice** - the per-statement visitor + callback carries no index, and a shared index stack would have to push/pop with nested + blocks; block statement lists are short. - **Threshold provenance**: STYLE038's 80 ~ p97 of this tree's function lengths; STYLE041 caps report-arg recursion at 8 and fails closed (deeper is unknown shape); `LINT_SKIP_HEADER_LINES = 16` exists so prose quoting the directive cannot unlint a file; STYLE037's 20 and `note_type`'s depth 24 have no recorded derivation. +- **C++ mirror pairs**: the STYLE024/025 unsafe map <-> infer's `unsafe_*` rules; + `style036_inert_contract` <-> infer's contract clearing. Nothing fails when one side + moves alone. -## aot_cpp +## 5. aot_cpp - **C++ identifier mangling**: `aotSuffixNameEx` prepends `_S`/`_E`/`_V`/`_f_` when a das name is a C++ keyword, holds a non-alnum char, or is `DELETE` (winnt.h). Structs and @@ -206,19 +213,17 @@ an entry lands here only when no name, shape, or test can carry it. finds the pairs per module and forces both suffixes. - **The emitter targets the MSVC/GCC/Clang intersection; the redundant-looking spellings are the portable ones** - `INT64_C`, the explicit `0x8000000000000000` enum arm, - `to_cpp_double`'s named non-finite tokens (mirrors `src/builtin/runtime_string.cpp` and + `to_cpp_double`'s named non-finite tokens (mirrors `src/simulate/runtime_string.cpp` and moves with it), elaborated `struct X`, double parens around `das_iterator` sources (most vexing parse), non-const value loop variables (`first()`/`next()` write into it). - **Sequenced binary operands**: any op2 whose operands are not both side-effect-free is - emitted through `das_ordered2{ L, R }` + an immediately-invoked lambda - braced aggregate - init is the C++ construct that guarantees left-to-right evaluation, matching the - interpreter and JIT. The struct and its member names live in - `include/daScript/simulate/aot.h` and move with the emitter's `__lr.left`/`__lr.right` - spellings. + emitted through `das_ordered2{ L, R }` plus an immediately-invoked lambda. The struct and + its member names live in `include/daScript/simulate/aot.h` and move with the emitter's + `__lr.left`/`__lr.right` spellings. - **Stack-frame `new`/ascend**: per-block storage declared once, the USE site re-initializes per evaluation (memset for `new`, whole-value overwrite for ascend) - dropping the reinit reuses the previous iteration's value. -- **fp16 and the 8/16-bit lattice ride the vec4f policy ABI**: `SimPolicy_HalfVec` is +- **fp16 and the 8/16-bit lattice go through the vec4f policy ABI**: `SimPolicy_HalfVec` is vec4f-shaped even at width 1, so scalar fp16 always casts across the policy boundary, and a lattice-vector policy result must come back typed for `das_equ_val` deduction. Lattice swizzles use element-typed helpers - generic `das_swizzle` caps at 4 fields, @@ -235,31 +240,30 @@ an entry lands here only when no name, shape, or test can carry it. `set_aot_main_module_name_cpp`; a spelling rule changed on one side only produces a TU where the definition and its debug-info references disagree. -## aot_standalone +## 6. aot_standalone - **The generated constructor IS the init protocol** - a standalone context never calls `Context::runInitScript`, so the ctor reproduces its observable semantics inline: `memset(context.globals, 0, context.getGlobalSize())` mirrors runInitScript's globals - memset (`src/runtime/context.cpp`), then the same-TU `__init_script(&context, true)` - (signature written by `aot_cpp.das`'s `preVisitGlobalLet` - a daslib-to-daslib pairing, - nothing checks agreement), then each `[init]` function directly. Deliberate divergences: - `__init_shared` is hardcoded `true` (a fresh standalone context always owns its - shared globals), there is no separate init stack (init locals are C++ locals in AOT; - instead the ctor's base `Context(N)` folds the init headroom in: N = - `options stack + globalInitStackSize` when the script sets `options stack` to a - positive value (honored exactly, below the interpreter's floor included - the floor - protects INTERPRETED init, while standalone init is AOT, so an embedded target's stack - budget wins; zero and negative fall through - `Context(0)` deliberately owns no stack, - so any prologue would assert), else `max(policies.stack, 16384) + globalInitStackSize`, the - no-option arm mirroring the interpreter's init-stack formula in - `src/ast/ast_simulate.cpp` - a pair, deliberately divergent on the explicit-option arm; - `globalInitStackSize` - reaches the emitter through the rtti `Program` binding in - `src/builtin/module_builtin_rtti.cpp` - so an `invoke` during init still - has das stack to push a prologue onto), and there is no `!stopFlags` guard between - `[init]` calls (a panic propagates out of the ctor instead of soft-stopping the - sequence). -- **Global init order is a pact with the allocator**: `StandaloneContextGen`'s + memset (`src/runtime/context.cpp`), then the same-TU `__init_script(&context, true)`, + then each `[init]` function directly. `aot_cpp.das`'s `preVisitGlobalLet` writes the + `__init_script` signature - a daslib-to-daslib pairing nothing checks. +- **`__init_shared` is hardcoded `true`** - a fresh standalone context always owns its + shared globals. +- **The ctor's base `Context(N)` folds the init headroom in, because there is no separate + init stack** - init locals are C++ locals in AOT. A script that sets `options stack` to a + positive value gets `N = options stack + globalInitStackSize`, honored exactly, the + interpreter's floor included: the floor protects INTERPRETED init, while standalone init + is AOT, so an embedded target's stack budget wins. Zero and negative fall through to + `N = max(policies.stack, 16384) + globalInitStackSize`; `Context(0)` owns no stack, so any + prologue would assert. The fall-through arm mirrors the interpreter's init-stack formula + in `src/ast/ast_simulate.cpp` - a pair, deliberately divergent on the explicit-option arm. + `globalInitStackSize` reaches the emitter through the rtti `Program` binding in + `src/builtin/module_builtin_rtti.cpp`, so an `invoke` during init still has das stack for + its prologue. +- **There is no `!stopFlags` guard between `[init]` calls** - a panic propagates out of the + ctor instead of soft-stopping the sequence. +- **Global init order is fixed by `var->index` assignment**: `StandaloneContextGen`'s `preVisitGlobalLet` emits required modules' globals via ordered `for_each_module` before the adapter walks the entry module's own - correct only because `ast_allocate_stack.cpp` assigns `var->index` through the same dependency-first @@ -291,7 +295,7 @@ an entry lands here only when no name, shape, or test can carry it. an embedder TU; two DIFFERENT contexts' headers sharing a das dependency still cannot be included in one TU (the shared types have no per-type guards). -## flatten +## 7. flatten - **Predicated lowering carries one live-mask per exit flavor** - `__flat_live` for return, a per-loop break mask (persists across unrolled copies) and continue mask @@ -304,8 +308,8 @@ an entry lands here only when no name, shape, or test can carry it. are owned by that loop and re-seeded from surviving suffixes (per-call numbering re-mints a live name). - **A CSE/regroup tally counts exactly the regions its rewrite can change** - a duplicate - counted where the rewrite cannot reach never drops below 2 and spins the fixpoint to its - guard. + counted where the rewrite cannot reach never drops below 2 and runs the fixpoint to its + iteration cap. - **`__flat_ret` carries `safeWhenUninitialized` only while every write is a self-referential select** - a lowering change that makes the bare-decl read observable turns the flag into a real uninitialized read. @@ -314,18 +318,20 @@ an entry lands here only when no name, shape, or test can carry it. through index/field/swizzle destabilizes its base; an unrecognized node fails closed as mutable-reading. Uniform duplicates route to the preshader. - **The copy-prop/CSE walks stay O(size)** - one name-to-statement index, one structural - walk; never materialize a `string` per `ExprVar` in a visitor callback (O(n^2) persistent - heap - the heap-overflow amplifier). -- **`MutCollect` is the value-stability oracle, so it counts every store spelling, not the - one the lowering emits.** CSE treats a name outside its set as constant for the whole - block; a missed store is a shared subexpression across a mutation. Copies are only the - visible half - `<-` also zeroes its SOURCE, `:=` lowers to a `builtin`clone`(dst, src)` - CALL rather than an `ExprClone`, `++`/`+=` are their own nodes, and a by-reference + walk. A `string` materialized per `ExprVar` in a visitor callback breaks that: each + `describe()` allocates a string that lives to the end of the pass, so the walk goes + quadratic in heap bytes, not only in time. +- **`MutCollect` is what CSE trusts to say whether a name is stable, so it counts every + store spelling, not the one the lowering emits.** CSE treats a name outside its set as + constant for the whole block; a missed store is a shared subexpression across a mutation. + Copies are only the visible half - `<-` also zeroes its SOURCE, `:=` lowers to a + `builtin`clone`(dst, src)` CALL rather than an `ExprClone`, `++`/`+=` are their own + nodes, and a by-reference argument writes with no assignment node anywhere. Hence the argument arm keys on the callee's parameter type (non-const and `ref` or a ref type), not on a node kind. -- **`delete` on a container of `ExpressionPtr` frees the BUFFER, never the nodes** - the - house rule that `delete array<T?>` frees the pointees is about das-heap `T`, and - `Expression` is a handled C++ type whose instances are not heap chunks at all (the +- **`delete` on a container of `ExpressionPtr` frees the BUFFER, never the nodes** - + `delete array<T?>` frees the pointees only for das-heap `T`, and `Expression` is a + handled C++ type whose instances are not heap chunks at all (the measurement: an `array<S?>` of das structs returns its pointees to `heap_bytes_allocated`, an `array<ExpressionPtr>` returns only the buffer and the nodes surface in the exit GC report). That is why `make_float_ctor`'s const-fold early return may leave its lanes @@ -340,23 +346,22 @@ an entry lands here only when no name, shape, or test can carry it. the drop delete the store. Predicating such a write would need per-out-param temps the lowering does not own. -## ast_verify +## 8. ast_verify - **One node set answers two opposite questions**: a node reached TWICE has two parents (broken unique ownership); a gc-owned node reached ZERO times sits in a slot no visitor enters, so every visitor-driven pass edits a tree it cannot see in full. Slots C++ - `::visit` skips are walked by hand or recorded as whole trees - reading the matching - `::visit` is step one of adding a check. + `::visit` skips are walked by hand or recorded as whole trees. - **Reporting is repair**: diagnostics go through `macro_sticky_error` (infer clears plain errors from a repaired tree); null entries in per-entry-dereferenced lists compact in preVisit; a self-reachable node is CUT, not reported - a cycle kills every later walk. Both passes use the sticky form, post-infer included. `Program::stickyError` records the error AND calls `Program::error`, so it is a strict superset of `macro_error` - the plain - form buys nothing. It is currently unreachable for a loss (every re-entry into - `inferTypes`/`inferTypesDirty` after a post-infer firing is guarded by - `!program->failed()`), but repair is what makes the guarantee necessary rather than - belt-and-suspenders: the guard that found the damage cannot fire twice, so an error - cleared once is gone for good. + form buys nothing. A loss is unreachable while every re-entry into + `inferTypes`/`inferTypesDirty` after a post-infer firing stays guarded by + `!program->failed()`, but repair is what makes the guarantee necessary rather than + redundant: the guard that found the damage cannot fire twice, so an error cleared once is + gone for good. - **`visitExpression` is the only place an ancestor-path entry is dropped.** The C++ adapter's `VISIT_EXPR` macro (`aot_builtin_ast.h`) calls `visitExpression(that)` before every `visitExprXXX`, so a derived visit that erased again would erase nothing - a @@ -372,14 +377,14 @@ an entry lands here only when no name, shape, or test can carry it. (`in_field_init`, set/cleared around the field walk); every other invariant (types, ats) still applies to them. The same-module shape resolves eagerly and never had the hole - the selftest fixture is cross-module for exactly that reason. -- **Each check's licensing C++ site** (the checklist requires one per check; record new ones - here): `ExprOp1.subexpr` - `SimulateVisitor::visit(ExprOp1*)` dereferences; +- **Each check's licensing C++ site**: `ExprOp1.subexpr` - + `SimulateVisitor::visit(ExprOp1*)` dereferences; `TypeDecl.dim` entries - `TypeDecl::dimConst` sentinel (`ast_typedecl.h`); `ExprFor` sources/body - `ExprFor::visit` walks sources and body, never the iterator tags; `ExprBlock.arguments` - `ExprBlock::visit` walks `arguments` only when `isClosure`; `ExprBlock.returnType` - `ExprBlock::visit` walks it only for a closure. -## quote +## 9. quote - **A lowered quote is a generated leaf function, not an inline expression** - one frame per quote instead of inflating every caller's; the result type clones the quote's static @@ -389,23 +394,25 @@ an entry lands here only when no name, shape, or test can carry it. hangs the walk** (`Function.classParent`, `ExprReturn._block`, `ExprVar.pBlock`, `EnumEntry.value`); the rest are post-infer bookkeeping absent from quoted trees. -## templates_boost +## 10. templates_boost - **A `Template`'s substitution tables are freed once, by whichever mechanism the local declares** - `var inscope rules` finalizes at scope exit; a plain `var rules` does not and needs an explicit `delete rules`. Declaring both double-finalizes, and the double is - silent: a plain local leaks 224 B where inscope-only, inscope+delete and plain+delete all - leak 0, so nothing shows up in a heap count. Both `apply_qmacro_template_*` appliers are - inscope-only; the three plain-local `apply_template` overloads pair the declaration with - the delete. Pick one form per local and read the declaration before adding a delete. + silent: inscope-only, inscope+delete and plain+delete all leak nothing, so a heap count + cannot tell a double-finalize from a correct form; only a plain local with no delete + leaks. Both `apply_qmacro_template_*` appliers are inscope-only; the three plain-local + `apply_template` overloads pair the declaration with the delete. - **`apply_template` returns a possibly-NEW root.** `apply_template(rules, at, expr, forceAt)` takes `expr` by value, and the block-form overloads take `Expression?&` only to forward it - none writes the new root back. A substitution that replaces the ROOT node - an identity key body, `_order_by(_)` - is visible only through the return value; non-root - replacements leave the pointer unchanged. Always reassign from the return value, or the - identity case silently keeps the unsubstituted node. + replacements leave the pointer unchanged. +- **`stamp_missing_at` fills only MISSING locations**, unlike the force-at of `$e()`; + `carry_tag_safe_flags` copies the parser's unsafe-wrap flags across `$c` substitution or + the safety is lost where the result lands. -## linq +## 11. linq - **`top_n_by_with_cmp` exists for the fold splice and has no daslib call site.** When an order-by key body is pure and inlineable, `linq_fold_common` emits it with that body @@ -413,7 +420,7 @@ an entry lands here only when no name, shape, or test can carry it. and the per-side `key(v)` dispatch the key-lambda overloads pay. It is not a redundant twin of `top_n_by`. -## linq_boost +## 12. linq_boost - **`BucketLambdaStamper` exists because a bucket-surface lambda cannot infer on its own.** On the `group_by_lazy` element shape `tuple<K; array<E>>`, an untyped lambda in @@ -427,8 +434,14 @@ an entry lands here only when no name, shape, or test can carry it. `_full_outer_join` deliberately do not override: their result parameter is `$Option<T>` on one or both sides, so those result lambdas stay user-spelled. -## linq_das +## 13. linq_das +- **The substituting scanners share one token model.** `substitute_idents`, + `mentions_ident` and `rewrite_group_var` all read it: plain `"..."` content is verbatim, a + `{...}` interpolation body is code that is scanned and substituted, one level of nested + string literal inside an interpolation is verbatim again, and identifier position + classifies the same way everywhere - after-`.` field access and before-single-`=` labels + are not references, with `is_label_position` the shared judge. - **`JOIN_TI` is a spellable identifier on purpose.** `__`-prefixed names are reserved by the compiler, so the transparent-identifier carrier cannot be hidden outside the user's namespace and must stay a single plain token - hence the explicit rejection of a range @@ -455,7 +468,7 @@ an entry lands here only when no name, shape, or test can carry it. rejected with a fix message before those arms: it already carries its row shape, and the typed builders would all mis-fire. -## linq_fold_common +## 14. linq_fold_common - **group_by's per-key update is a dummy plus an address compare, not a `key_exists` probe.** `entry &= tab?[uk] ?? dummy` costs one hash op per element on hits; @@ -476,7 +489,7 @@ an entry lands here only when no name, shape, or test can carry it. tuple's values - each references its iter var by name under an `ExprRef2Value` wrap - and match what the loop binds once a later pass infers it. -## linq_fold_sql +## 15. linq_fold_sql - **`extract_sql_source` walks the call's own `arguments[0]` spine, not linq_fold's flattened `top`** - the SQL DSL ops are not `linqCalls`, so `flatten_linq` stops at the @@ -484,7 +497,7 @@ an entry lands here only when no name, shape, or test can carry it. on `_sql` being in THIS program: a consumer can get `select_from` from a provider boost without `daslib/sql_linq`, and that chain must fall through to the in-memory path. -## linq_fold_decs +## 16. linq_fold_decs - **Column pruning is a four-step pipeline over the `decs_tup` bind, and every step exists to reach bind elision.** Scan the body for `decs_tup.<field>` uses; if the bind is @@ -507,13 +520,13 @@ an entry lands here only when no name, shape, or test can carry it. flatten to a pure copy; renaming the chain's `finalBind` to the iter var skips the binding entirely. Detecting the shape needs TWO `ExprRef2Value` peels: the typer wraps both the projection root and the `ExprVar` inside the `ExprField`. -- **The decs join hook keeps hash-collect and probe inline rather than riding +- **The decs join hook keeps hash-collect and probe inline rather than routing through `wrap_source_loop`** - that is what preserves the count-no-`where` fast path, which answers from bucket lengths instead of walking pairs. Its key lambdas are synthesized upstream by the LinqJoin macro as 1-arg single-return blocks, so the BLOCK's return type is the key type; a change to that synthesized shape breaks the decs join silently. -## sql_linq +## 17. sql_linq - **Equal phase numbers are deliberate.** `PHASE_ORDER_BY == PHASE_SELECT` - ORDER BY commutes with the projection in SQL (it may reference projected aliases or source @@ -556,7 +569,7 @@ an entry lands here only when no name, shape, or test can carry it. lambda's bound variable is renamed to `_`. Gated on a pure single-source chain - join and set-op key lambdas bind distinct sources and are left alone. -## dupe_detect +## 18. dupe_detect - **A run is unextractable when its meaning depends on the scope around it** - the gate counts `unsafe`-authorized operations, `assume` aliases, `defer` (lowered away), and @@ -565,80 +578,123 @@ an entry lands here only when no name, shape, or test can carry it. read `.at` off the node; `ExprConst*` re-enters the base pre-visit hook and the second entry pushes a `-1` marker - lose it and every preorder interval is wrong. -## misc module contracts +## 19. only_nttp -- **C++ mirror pairs (lint family)**: `lint022_optimized` / `lint022_calls_may_be_inlined` - <-> `Program::getOptimize` / `Program::patchInline`; `stale_scan_line` <-> - `rtti_is_nolint_suppressed`; `is_inline_temp_name` <-> `INLINE_TEMP_PREFIX`; the - STYLE024/025 unsafe map <-> infer's `unsafe_*` rules; `style036_inert_contract` <-> infer's - contract clearing; `only_nttp`'s `moreFlags2.nttp` read <-> `ExternalFnInline`'s - `this->nttp = true` (`include/daScript/ast/ast_interop.h`) plus the positional +- **C++ mirror pair**: `only_nttp`'s `moreFlags2.nttp` read <-> `ExternalFnInline`'s + `this->nttp = true` (`include/daScript/ast/ast_interop.h`), plus the positional `MoreFunctionFlags2` argNames list (`src/builtin/module_builtin_ast_flags.cpp`) that - gives the bit its daslang name. Each pair changes in lockstep; nothing fails when one - side moves alone. -- **interfaces**: the implements-marker IS the generated getter field - `is`/`as`/`?as` - key purely on its presence; parent interfaces get their own deduped getter fields. - The const getter's `unsafe(addr<$t(st)? -const>(self))` is the one blessed const-strip - write, and it escapes the DCE trap on two counts, not on the `unsafe`: the constness sits - on the PARAMETER BINDING while the object behind it is an ordinary mutable allocation, and - the store is re-read through the same pointer two lines later while the new proxy escapes - into it - no tier can prove it dead. Emitted AOT C++ keeps the cast and the store verbatim. - Caching into anything the caller owns by const VALUE would not survive this. -- **flat_hash_table**: `hashes[i]` is the slot state - 0 never-used (probe stops), - 1 tombstone (probe continues), above 1 live; a hash function that can return 0 or 1 - loses entries silently. -- **coverage**: instrumentation is `generated` (lint-invisible, inliner-safe) and asserts - become verify so their spliced counters survive release. -- **regex**: zero-width nodes (lookahead, Bos/Eos, word boundaries) leave `subexpr.next` - null on purpose - chaining them would consume the continuation. The literal matchers - compare byte-by-byte and return on the first mismatch - a fixed-`textLen` block compare - (memcmp) reads past the terminator whenever the terminator falls before the literal's - last byte (the tail at least two bytes shorter than the literal). -- **debugger**: `g_installed_agents` is the GC root for every installed agent - the C++ - adapter holds a raw classPtr the das GC cannot see. -- **typemacro_boost**: the parser does not run annotation `apply` for macro-added - functions - the add/erase pair in `tsi_stamp_methods` IS the trigger, not bookkeeping. -- **constant_expression**: the generated specialization is called through `__::` - a - plain name resolves in the call site's DEFINING module, where it does not exist. -- **toml**: the writer is hand-rolled because the builtins do not round-trip (`\v`, - raw 0x7f, float exponent thresholds); a value scan that runs into a bare-key char - rewinds and re-lexes as a key - a new numeric form needs the same rewind. -- **shader_block_layout**: two rails, deliberately separate - the LAYOUT rail admits - int64/uint64 as block members (`compute_block_layout` special-cases them) while the - ARITHMETIC rail rejects 64-bit INT (`arith_width_ok` allows width 64 only for floats); - `cpu_only_lattice_width` keys both emitters' fail-closed diagnostic. -- **shader_lingua_franca**: every symbol is either an exact CPU mirror of its GPU - semantics or a `[sideeffects]` dummy every rail lowers by name - the dummies return - zero on the host, so a CPU replay is an oracle only for the real-bodied set. Unsigned - overloads never fold into signed twins (glslang picks the unsigned opcode). -- **templates_boost**: `stamp_missing_at` fills only MISSING locations (unlike the - force-at of `$e()`); `carry_tag_safe_flags` copies the parser's unsafe-wrap flags - across `$c` substitution or the safety is lost where the result lands. -- **archive**: `MemSerializer.write` grows capacity eagerly because under a - very_safe_context each doubling generation is abandoned, not reused; no alias into - `data` survives a write. -- **json**: `is_json_white_space` is deliberately not the shared `is_white_space` - - RFC 8259 admits exactly space/tab/CR/LF. -- **jsonrpc**: parsing a request is a SCOPE, not a value. `ParsedRequest.params` is a - borrowed view into the parse tree, so the tree has to outlive the handler and cannot - be freed inside the parser - instead the owner is named in the result (`document` on - `ParsedRequest` for `parse_request`, on `ParsedBatch` for `parse_batch`, where the - entries borrow and own nothing) and the scope is closed by `free_request` / - `free_batch`; `dispatch_line` owns the whole scope and frees after the dispatcher has - run over every entry. Freeing is safe for the string fields because daslang never - finalizes a `string` field (`TypeDecl::needDelete` is false for `tString`), so - `method` / `id_str` / `params_json` / `error_envelope` outlive the tree - only the - `params` pointer dangles, which is why the free nulls it. The response side has no - such scope: `parse_one_response` copies everything into strings, so - `parse_response` / `parse_response_batch` free their own tree before returning. -- **sql_migrate**: the audit table is provider-neutral by construction (client-side epoch - seconds, BIGINT); duplicate versions are caught in two layers because neither sees - everything; the `struct_convert_field` overload set is a specificity ladder - deleting - a "duplicate" silently re-routes conversions. -- **fio**: the glob matcher follows POSIX fnmatch on degenerate patterns (unterminated - `[`, `]` as first class member, `**/` slash rules) - conformance, not quirks. -- **decs**: component finalizers are lambdas over the component's own untyped storage - - deleting the lambda IS the finalization event; capacity checks compare the highest - allocated value (`base + count - 1`), not the exclusive end. -- **builtin**: `_table_index_and_init` exists for infer's `default_init_containers` - rewrite of non-store `tab[key]` - it has no daslib call site and is not dead. + gives the bit its daslang name. Nothing fails when one side moves alone. + +## 20. interfaces + +- **The implements-marker IS the generated getter field** - `is`/`as`/`?as` key purely on + its presence; parent interfaces get their own deduped getter fields. +- **The const getter's `unsafe(addr<$t(st)? -const>(self))` is the one sanctioned + const-strip write**, and the dead-store eliminator cannot remove it on two counts, not on + the `unsafe`: the constness sits on the PARAMETER BINDING while the object behind it is an + ordinary mutable allocation, and the store is re-read through the same pointer two lines + later while the new proxy escapes into it - no tier can prove it dead. Emitted AOT C++ + keeps the cast and the store verbatim. Caching into anything the caller owns by const + VALUE would not survive this. + +## 21. flat_hash_table + +- **`hashes[i]` is the slot state** - 0 never-used (probe stops), 1 tombstone (probe + continues), above 1 live; a hash function that can return 0 or 1 loses entries silently. + +## 22. coverage + +- **Instrumentation is `generated`** (lint-invisible, inliner-safe) and asserts become + verify so their spliced counters survive release. + +## 23. regex + +- **Zero-width nodes (lookahead, Bos/Eos, word boundaries) leave `subexpr.next` null on + purpose** - chaining them would consume the continuation. +- **The literal matchers compare byte-by-byte and return on the first mismatch** - a + fixed-`textLen` block compare (memcmp) reads past the terminator whenever the terminator + falls before the literal's last byte (the tail at least two bytes shorter than the + literal). + +## 24. debugger + +- **`g_installed_agents` is the GC root for every installed agent** - the C++ adapter holds + a raw classPtr the das GC cannot see. + +## 25. typemacro_boost + +- **The parser does not run annotation `apply` for macro-added functions** - the add/erase + pair in `tsi_stamp_methods` IS the trigger, not bookkeeping. + +## 26. constant_expression + +- **The generated specialization is called through `__::`** - a plain name resolves in the + call site's DEFINING module, where it does not exist. + +## 27. toml + +- **The writer is hand-rolled because the builtins do not round-trip** (`\v`, raw 0x7f, + float exponent thresholds); a value scan that runs into a bare-key char rewinds and + re-lexes as a key - a new numeric form needs the same rewind. + +## 28. shader_block_layout + +- **Two rails, deliberately separate** - the LAYOUT rail admits int64/uint64 as block + members (`compute_block_layout` special-cases them) while the ARITHMETIC rail rejects + 64-bit INT (`arith_width_ok` allows width 64 only for floats); `cpu_only_lattice_width` + keys both emitters' fail-closed diagnostic. + +## 29. shader_lingua_franca + +- **Every symbol is either an exact CPU mirror of its GPU semantics or a `[sideeffects]` + dummy every rail lowers by name** - the dummies return zero on the host, so a CPU replay + reproduces GPU semantics only for the real-bodied set. Unsigned overloads never fold into + signed twins (glslang picks the unsigned opcode). + +## 30. archive + +- **`MemSerializer.write` grows capacity eagerly** because under a very_safe_context each + doubling generation is abandoned, not reused; no alias into `data` survives a write. + +## 31. json + +- **`is_json_white_space` is deliberately not the shared `is_white_space`** - RFC 8259 + admits exactly space/tab/CR/LF. + +## 32. jsonrpc + +- **Parsing a request is a SCOPE, not a value.** `ParsedRequest.params` is a borrowed view + into the parse tree, so the tree has to outlive the handler and cannot be freed inside the + parser - instead the owner is named in the result (`document` on `ParsedRequest` for + `parse_request`, on `ParsedBatch` for `parse_batch`, where the entries borrow and own + nothing) and the scope is closed by `free_request` / `free_batch`; `dispatch_line` owns + the whole scope and frees after the dispatcher has run over every entry. Freeing is safe + for the string fields because daslang never finalizes a `string` field + (`TypeDecl::needDelete` is false for `tString`), so `method` / `id_str` / `params_json` / + `error_envelope` outlive the tree - only the `params` pointer dangles, which is why the + free nulls it. The response side has no such scope: `parse_one_response` copies everything + into strings, so `parse_response` / `parse_response_batch` free their own tree before + returning. + +## 33. sql_migrate + +- **The audit table is provider-neutral by construction** (client-side epoch seconds, + BIGINT); duplicate versions are caught in two layers because neither sees everything; the + `struct_convert_field` overload set is a specificity ladder - deleting a "duplicate" + silently re-routes conversions. + +## 34. fio + +- **The glob matcher follows POSIX fnmatch on degenerate patterns** (unterminated `[`, `]` + as first class member, `**/` slash rules) - conformance, not quirks. + +## 35. decs + +- **Component finalizers are lambdas over the component's own untyped storage** - deleting + the lambda IS the finalization event; capacity checks compare the highest allocated value + (`base + count - 1`), not the exclusive end. + +## 36. builtin + +- **`_table_index_and_init` exists for infer's `default_init_containers` rewrite of + non-store `tab[key]`** - it has no daslib call site and is not dead. diff --git a/daslib/REVIEW.md b/daslib/REVIEW.md index 7e8eafe848..3e5e85b6a9 100644 --- a/daslib/REVIEW.md +++ b/daslib/REVIEW.md @@ -15,23 +15,24 @@ collides across rules. `// nolint` at any frame of the reported chain.** The sink often bottoms out in daslib while the actionable line is an intermediate call site. -**A diff that moves an `in_closure` / `in_deferred` guard inside a lint visitor method as -cleanup is a defect.** The guard's position decides which rules apply inside a lambda; the -per-rule policy is in `ARCHITECTURE.md` sec. perf_lint. +**A diff that moves an `in_closure` / `in_deferred` guard to a different statement position +inside a lint visitor method is a defect.** The guard's position decides which rules apply +inside a lambda; the per-rule policy is in `ARCHITECTURE.md` sec. 1. + +**Never move `add_ptr_ref`'s depth tracking behind the `in_closure` gate.** The rules block +is itself a closure, so a gated tracker never sees the splice. **A lint warning that anchors anywhere but the code its fix rewrites is a defect.** A remedy that deletes a statement reports the statement; a remedy that edits one variable's initializer reports the variable. **Never exit a construct's visitor on a path that skips resetting or restoring the visitor -state scoped to that construct.** +state scoped to that construct.** State an early return leaves set poisons the next +construct's visit. **Never keep per-loop visitor state in a bare scalar - keep it in a stack.** A scalar survives into the sibling loop's exit path and unbalances its counter. -**Never let `preVisitFunction` return before it resets the per-function visitor state.** -State an early return leaves set poisons the next function. - **A diff that makes a daslib predicate or emitted identifier depend on a C++-side definition records the pair in `ARCHITECTURE.md`, in its module's section, naming both sides.** Nothing catches it when one side later moves alone. @@ -40,10 +41,15 @@ sides.** Nothing catches it when one side later moves alone. it changes the other side and updates the pair's `ARCHITECTURE.md` entry in the same diff.** -**Weakening the nolint-window tests is a defect** - `tests/lint/test_nolint_suppression.das` -pins that a string literal, a URL, and a mid-comment `nolint:` do not suppress while a -first-token directive after `//` or `//!` does, and `tests/lint/test_stale_nolint.das` pins -that a `lint-skip-file` past the header window is prose. +**A comment-sweep diff adds an `ARCHITECTURE.md` entry only where no name, shape, or test +can carry the fact.** + +**Weakening `tests/lint/test_nolint_suppression.das` is a defect** - it pins that a string +literal, a URL, and a mid-comment `nolint:` do not suppress while a first-token directive +after `//` or `//!` does. + +**Weakening `tests/lint/test_stale_nolint.das` is a defect** - it pins that a +`lint-skip-file` past the header window is prose. **A diff that changes `build_lint_macro_disabled` keeps its four sources layered in this order: defaults, repo `off`, repo `on`, environment.** Env last lets a one-run @@ -71,14 +77,15 @@ an early record lets the RHS's own read clear the store it just recorded. **Never match a lint rule's callee name against the instance - match the root generic.** Instantiation mangles names and strips witness arguments. -**A lint rule that fires inside the module whose idiom it advertises is a defect.** +**A lint rule that fires on the source of the module that implements the idiom it suggests +is a defect.** **A diff that adds or changes a collapse suggestion gates it on receiver type, arity, cloneability, and duplicate constant keys, so the suggestion compiles and keeps the reported shape's semantics.** -**A diff that makes two sibling collapse rules fire on one shape is a defect.** Overlaps -stay partitioned: STYLE021 owns the const-key run over STYLE031, STYLE032 owns fresh-empty +**A diff that makes two collapse checks fire on one shape is a defect.** Overlaps stay +partitioned: STYLE021 owns the const-key run over STYLE031, STYLE032 owns fresh-empty declarations over STYLE033, and STYLE033's chain-receiver check leaves a plain-variable receiver to STYLE033's variable-form check. @@ -86,10 +93,14 @@ receiver to STYLE033's variable-form check. colon.** Suppression, disable lists, and dedup all parse the code from the message, so a message that leads with anything else makes the rule unsuppressable and self-colliding. -**A diff that adds or changes a `canVisit*` override in style_lint teaches the unsafe-frame -push/pop about the subtree the override skips.** The walk pushes one frame per expression -and pops it, so a skipped subtree unbalances the count; the balance panic in each entry -point is the tripwire. +**A diff that adds or changes a `canVisit*` override in style_lint keeps the unsafe-frame +count balanced across the subtree the override skips.** The walk pushes one frame per +expression and pops it, so a skipped subtree unbalances the count; the balance panic in each +entry point is the tripwire. + +**Never cache `blk.list` index state across style_lint visitor callbacks.** The +per-statement callback carries no index and a shared stack would have to push/pop with +nested blocks; block statement lists are short, so the rescan is the cheap form. **A diff that changes an arity cap or the overload set it mirrors - `MAX_CONCAT_ARITY` <-> linq's `concat` variadics, `MAX_VARIADIC_PUSH_ARITY` <-> builtin @@ -128,7 +139,7 @@ that skips the seeding spells structs differently from the run that seeded it. node's location.** `match_error` stores the `LineInfo` pointer BORROWED, and access nodes are cloned per field inside a bare scope and die with it. -**Never report from a macro that lowers lambdas through `macro_error` - report through +**Never report an error with `macro_error` from a macro that lowers lambdas - report with `macro_sticky_error`.** A later pass lowers the lambda to a plain function and clears non-sticky errors. @@ -137,10 +148,10 @@ non-sticky errors. every byte >= 0x80. **A diff that adds or changes a parser result that borrows a view into a tree the parser -allocated names the owning field and ships the scope-ender that frees it.** The tree must -outlive the caller's read, and daslang finalizes neither a raw pointer field nor a local -container at scope exit, so a borrowed view with no named owner leaks the whole document per -call. +allocated names the field that owns the tree and ships the function the caller runs to free +it.** The tree must outlive the caller's read, and daslang finalizes neither a raw pointer +field nor a local container at scope exit, so a borrowed view with no named owner leaks the +whole document per call. **Never convert a string with `int64` / `uint64` / `double` in a lexer or decoder reachable from a file, a socket, or a model - use the non-throwing `to_*` twin and report through the @@ -154,21 +165,23 @@ input-controlled branch. **A diff that adds or changes a flatten_opt rewrite arm ships a read-only residual predicate with that arm** - the oracle the residual visitors call to prove the pass complete. -**Never let a residual oracle's gate differ from its arm's gate, and never call the -transform from an oracle.** A gate the transform takes as a parameter is threaded into the -oracle too, and the tests that call the oracle pass the same value. A narrower oracle is a -false pass, a wider one a false miss, a re-spelled parameter default a false pass on every -non-default run; calling the transform aliases the live tree. +**Never let a residual oracle's gate differ from its arm's gate.** A gate the transform +takes as a parameter is threaded into the oracle too, and the tests that call the oracle +pass the same value; a narrower oracle is a false pass, a wider one a false miss, a +re-spelled parameter default a false pass on every non-default run. + +**Never call the transform from an oracle** - calling it aliases the live tree. -**A diff that adds a fold/fuse arm declares that arm's float class.** -Inf/NaN/rounding/association changes are fast-math-only; bit-exact per-lane rewrites are -never gated. An arm added without that decision silently changes output under -`_flatten_no_fast_math`. +**A diff that adds a fold/fuse arm gates it on `no_fast_math` when the rewrite changes +Inf/NaN, rounding, or association, and leaves it ungated when the rewrite is bit-exact per +lane.** An ungated value-changing arm silently changes output under `_flatten_no_fast_math`. **Never emit a fused call before proving the target module can resolve it - fall back to the unfused shape on a miss.** Otherwise the pass turns a shader that compiled into an unresolvable call on a narrower backend. +**Never materialize a `string` per `ExprVar` in a flatten visitor callback.** + **A new store spelling joins `MutCollect` in the same change.** CSE reads a name outside the mutable set as constant for the whole block, so an uncollected store is a silently shared subexpression. @@ -189,31 +202,30 @@ visitor machinery. and splice that.** Re-splicing re-evaluates a call once per splice, so a lock/unlock pair releases a different temporary than it took. -**A diff that adds or changes a `[sql_table]` helper pair registers its 2-arg form BEFORE -the 1-arg form.** `find_struct_helper_fn` keeps the LAST match and the finish pass rewrites -the helper it returns, so a swapped pair attaches index DDL to the wrong overload. +**A diff that adds a `delete` for a `Template`'s substitution tables reads the declaration +first.** An `inscope` local already finalizes, so the added delete is a silent +double-finalize. -**A generated SQL statement's column list and its bind function change in one edit.** Bind -indices are placeholder positions over bindable columns, never struct field positions; -drift is silently wrong data, not an error. +**A call to `apply_template` assigns the result back into the expression it passed.** A +root-node substitution is visible only through the return value, so a discarded result +silently keeps the unsubstituted node. **A new daslib spelling that reads the environment joins the env_registry marker lists in the same change.** The scanners match daslang source text; a missing spelling makes every enforcement test pass vacuously. -**A diff that adds or changes an `ast_verify` check names, in `ARCHITECTURE.md` sec. -ast_verify, the C++ `::visit` site that dereferences the slot unguarded.** A check with no -such site invents an invariant the compiler does not hold, and the two passes disagree - a -slot infer fills in is checkable only post-infer. +**A diff that adds or changes an `ast_verify` check names, in `ARCHITECTURE.md` sec. 8, the +C++ `::visit` site that dereferences the slot unguarded.** A check with no such site invents +an invariant the compiler does not hold, and the two passes disagree - a slot infer fills in +is checkable only post-infer. **A diff that adds or changes an `ast_verify` check repairs the slot it reports, not only reports it.** A broken slot left in place trips every later pass that walks the same node. -**Never sum a 64-bit range as `uint64` before proving both operands non-negative** -(`erase(at, count)`): a signed sum wraps, the bounds test passes, and the walk leaves the -array silently. +**Never sum a range's start and count as `uint64` before proving both non-negative** - a +signed sum wraps, the bounds test passes, and the walk leaves the array silently. -**Never emit a branch a macro-time value can decide from macro-built AST - branch in daslang +**Never put a branch a macro-time value can decide into macro-built AST - branch in daslang and emit only the taken arm.** Nothing folds at macro-application time, so a generated `if ($v(flag))` keeps its dead arm and type-checks it. diff --git a/daslib/REVIEW_LINQ.md b/daslib/REVIEW_LINQ.md index 89bdaea5ea..3ccabd89f8 100644 --- a/daslib/REVIEW_LINQ.md +++ b/daslib/REVIEW_LINQ.md @@ -1,20 +1,18 @@ # daslib linq family Code Review Checklist -**A diff touching the linq family - `linq*.das`, `sql_*.das` (the SQL lowering and its -bind-side helpers, `sql_boost.das` and `sql_provider.das` included) - applies this checklist +**A diff touching the linq family - `linq*.das`, `sql_*.das` - applies this checklist together with `REVIEW.md`.** `REVIEW_COMMON.md` (repo root) binds this file too. Architecture doc: `ARCHITECTURE.md`. **A fused emit that binds a terminator's default or compare argument lazily, more than once, or -below the top of the generated invoke, is a defect** - bind it once at the top, even on paths -that never use it, because the tier-2 `linq.das` overload evaluates it that way. This covers -every `*_or_default` decs lane, including an empty-tail fast path that would evaluate the -default lazily. +below the top of the generated invoke, every `*_or_default` decs lane included, is a defect** - +bind it once at the top, even on paths that never use it, because the unfused `linq.das` +overload evaluates it that way. -**A fused emit whose empty-source behavior differs from the tier-2 `linq.das` overload's - -panicking where tier-2 yields the default, or yielding a default where tier-2 panics - is a -defect.** A fused `first` over a prefilter buffer panics before it reaches `min`/`max`, which -return an uninitialized reference on an empty array. +**A fused emit whose empty-source behavior differs from the unfused `linq.das` overload's - +panicking where the unfused overload yields the default, or yielding a default where it +panics - is a defect.** A fused `first` over a prefilter buffer panics before it reaches +`min`/`max`, which return an uninitialized reference on an empty array. **Never add `count` or `first` to the 2-arg reducer set (`is_bucket_reducer_call`) - it admits only a reducer whose second argument is a selector.** `sum` / `min` / `max` / `average` take a @@ -50,11 +48,11 @@ in the same change** - `key_less` / `less_masked` (`linq.das`), which sort eager `try_make_inline_cmp_keys` (`linq_fold_common.das`), which emits an inline `_::less` if-chain for the same chain. All spell bit `i` of `mask` as "key `i` descending" (LSB = first key), flip operand order for descending, and break ties in key order; one side changed alone makes a -spliced chain and its tier-2 fallback sort differently, with no error. +spliced chain and its unfused fallback sort differently, with no error. **A diff that changes the inline `_::less` if-chain's key cap or `less_masked`'s tuple overloads changes both, in the same change.** A key tuple longer than the cap declines the splice and sorts -eagerly, so a cap raised without the matching overload leaves a spliced chain with no tier-2 twin +eagerly, so a cap raised without the matching overload leaves a spliced chain with no unfused twin to agree with. **Never reserve `n` in a `top_n*` lane over an iterator - let the bounded-heap fill phase grow @@ -65,19 +63,18 @@ the whole `n` upfront for no win. is a defect.** Chains nest, and the inner `_` has to shadow the outer one. **Never move the `_sql` hand-off below `normalize_order_reverse` / `collapse_chained_*` or below -`if (empty(calls))` - it runs ahead of chain normalization and ahead of the empty-chain bail.** -`_sql` re-expands the pristine expression on the next infer pass, and a `[sql_table]` source with -no chain ops at all is still SQL. +`if (empty(calls))`.** `_sql` re-expands the pristine expression on the next infer pass, and a +`[sql_table]` source with no chain ops at all is still SQL. **A pattern row that emits when its captured `select` (the `srcsel` slot) cannot be wrapped into a projected adapter is a defect - skip the row instead.** The row's emit assumes the projected element, so running it on the un-projected adapter orders or dedups raw rows - a wrong result, not a missed splice. -**A diff that reorders the `build_*_rows()` calls in `register_all_linq_fold_rows` changes -which arm claims a chain - review it as a pattern-priority change, never as a cleanup.** Rows -land in `splice_patterns` in call order, the walker takes the first match, and that registrar -is the only place the order exists. +**A diff that reorders the `build_*_rows()` calls in `register_all_linq_fold_rows` without +naming the chains whose matching arm changes is a defect.** Rows land in `splice_patterns` in +call order, the walker takes the first match, and that registrar is the only place the order +exists. **A sql_linq column-ref arm that accepts a `$e(recv).$f(field)` qmatch without first proving `recv is ExprVar` is a defect.** A nested receiver otherwise matches under a foreign name and @@ -88,9 +85,9 @@ of it is a defect.** That type gate routes a whole-row carry var to the clean ro instead of to a macro-time crash in `pred_to_sql`. **A call to a SQL-returning helper that checks only one of `q.hadError` and `empty(frag)` is a -defect - check both.** Operator arms wrap an empty child into non-empty junk and several helpers -return `""` without setting `hadError`, so one check alone emits malformed SQL in place of a -diagnostic. +defect - check both.** Operator arms wrap an empty child into a non-empty fragment and several +helpers return `""` without setting `hadError`, so one check alone emits malformed SQL in place +of a diagnostic. **A path from `analyze_chain` to `build_sql_string` / `collect_query_binds` that skips `maybe_finalize_distinct_by_passthrough` or `maybe_wrap_take_before_aggregate` is a defect.** @@ -101,8 +98,8 @@ LIMIT/OFFSET on the aggregate. exactly one producer per query.** The wrap drops the earlier subquery and orphans its binds - a bind/placeholder mismatch, with no error. -**A diff that adds a projection slot pushes one entry into each of the five index-parallel -projection arrays - `selectCols`, `selectColAliases`, `selectColSqlFragments`, `selectColTypes`, +**A diff that adds a projection slot pushes one entry into every index-parallel projection +array - `selectCols`, `selectColAliases`, `selectColSqlFragments`, `selectColTypes`, `projRecordNames` - in the same change.** `push_source_column` and `push_computed_proj_slot` cover only part of the set and leave the rest to their caller; a partial push desyncs the SELECT list from the row builder, with no error. @@ -115,19 +112,25 @@ position in `collect_query_binds`, in the same change.** `sql_to_frags_ex` re-sc SQL text and pairs markers with that list in occurrence order, so a clause added to the emitter alone binds every later placeholder to the wrong value, with no error. -**Never narrow or widen the linq_das clause-keyword test - a clause keyword is a depth-0 whole -word not preceded by `.` and not preceded by a `>` that tails `|>`, `=>` or `->`.** Narrowing +**A generated SQL statement's column list and its bind function change in one edit.** Bind +indices are placeholder positions over bindable columns, never struct field positions; drift is +silently wrong data, not an error. + +**A diff that adds or changes a `[sql_table]` helper pair registers its 2-arg form BEFORE the +1-arg form.** `find_struct_helper_fn` keeps the LAST match and the finish pass rewrites the +helper it returns, so a swapped pair attaches index DDL to the wrong overload. + +**Never narrow or widen the linq_das clause-keyword test - a clause keyword is a whole word at +bracket nesting depth zero, not preceded by `.` and not preceded by a `>` that tails `|>`, `=>` +or `->`.** Narrowing lets an in-body aggregate parse as a `select` clause; widening the exclusion to any `>` hides a clause keyword that legitimately follows a generic bracket or a comparison. **A diff that changes the token model in one substituting linq_das scanner - `substitute_idents`, -`mentions_ident`, `rewrite_group_var` - changes it in the others, in the same change.** The -shared model: plain `"..."` content is verbatim, a `{...}` interpolation body is code that is -scanned and substituted, one level of nested string literal inside an interpolation is verbatim -again, and identifier position classifies the same way everywhere - after-`.` field access and -before-single-`=` labels are not references, with `is_label_position` the shared judge. A model -change in one scanner desyncs `mentions_ident` from the rewrite it gates, and the emitter then -renames a parameter the spliced projection still references. +`mentions_ident`, `rewrite_group_var` - changes it in the others, in the same change** (the +shared model: `ARCHITECTURE.md` sec. 13). A model change in one scanner desyncs +`mentions_ident` from the rewrite it gates, and the emitter then renames a parameter the +spliced projection still references. **Giving `find_kw_depth0` the substituting scanners' interp-as-code model is a defect - it skips a whole string literal, interpolation bodies included.** A clause keyword inside `"{...}"` must @@ -136,7 +139,7 @@ never claim a stage terminal. **A `parse_one_stage` that reads a `where` or `orderby` position before it resolves the stage terminal - the earliest `select`/`group` at or after `start` - is a defect.** The terminal bounds the stage, so scanning clauses first lets a later stage's `where`, past this stage's `into`, -hijack this stage's terminal lookup. +be taken as this stage's terminal. **Never widen the `group ... into g` rewrite past its two spellings - bare `g` -> `g._1` and `g.key` -> `g._0` - leave every other `g.<member>` verbatim.** Raw tuple access then keeps working @@ -151,8 +154,8 @@ take, all after any `where_` - is a defect.** **Emitting the decs range guards in any order other than take-cap -> skip counter -> skip_while flag -> take_while break -> take bump is a defect** - that order mirrors the array side's -`wrap_with_ranges`. The bump is last so an element the while-guards rejected does not eat the -`take(N)` budget. +`wrap_with_ranges`. The bump is last so an element the while-guards rejected does not count +against `take(N)`. **A predicate-driven decs range accepted with a select in its prefix is a defect** - its predicate peels against the source tuple, so a select ahead of it changes the element the diff --git a/modules/dasImgui/REVIEW.md b/modules/dasImgui/REVIEW.md index c3fcb79242..543ee7259f 100644 --- a/modules/dasImgui/REVIEW.md +++ b/modules/dasImgui/REVIEW.md @@ -3,12 +3,12 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `CLAUDE.md`. -**Never add or change a dasImgui test file outside `modules/dasImgui/tests` - put it under that -folder instead.** A dasImgui test file, wherever the diff puts it, also answers to the `tests/` -subfolder's checklist (`modules/dasImgui/tests/REVIEW.md`). +**Never add or change a dasImgui test file - a `.das` file that declares a `[test]` function or +is named `test_*.das`, `failed_imgui_*.das`, or `record_*.das` - outside `modules/dasImgui/tests` +- put it under that folder instead.** A dasImgui test file, wherever the diff puts it, also +answers to the `tests/` subfolder's checklist (`modules/dasImgui/tests/REVIEW.md`). -**A diff that changes executable content in any `.das` file under this folder, or in any file -under `bind/` or `src/`, runs the test suite on the author's host OS before the PR: -`preflight --only imgui`** (the per-OS exclude split: module `CLAUDE.md` sec. Tests). No per-PR CI -lane runs that suite (`.github/workflows/nightly_imgui.yml` is nightly-only), so the local run is -the only pre-merge gate. +**A diff that changes anything but comments in a `.das` file under this folder, or any file +under `modules/dasImgui/bind/` or `modules/dasImgui/src/`, runs the test suite on the author's +host OS before the PR: `preflight --only imgui`** (the per-OS exclude split: module `CLAUDE.md` +sec. Tests). diff --git a/modules/dasLLAMA/REVIEW.das b/modules/dasLLAMA/REVIEW.das index b1a751bd97..161f5fc1cc 100644 --- a/modules/dasLLAMA/REVIEW.das +++ b/modules/dasLLAMA/REVIEW.das @@ -187,8 +187,8 @@ def private check_family_seams(carrier, kind_enum : string; exempt : table<strin } } -// REVIEW_GPU.md's backend-role partition, mechanically: every dasllama_<gpu>_*.das file's -// suffix is a role the checklist's list carries — a new role file adds its role to BOTH. +// The backend-role partition, mechanically: every dasllama_<gpu>_*.das file's suffix is a role +// ARCHITECTURE.md sec.1.5's role table carries — a new role file adds its role to BOTH. var private GPU_ROLES_METAL <- { "kernels", "common", "decode", "prefill", "gemm", "shapes", "tower", "asr_dec", "lens" } @@ -233,7 +233,7 @@ def private check_gpu_role_partition { let known = backend == "metal" ? key_exists(GPU_ROLES_METAL, role) : key_exists(GPU_ROLES_VULKAN, role) if (!known) { gate_finding("modules/dasLLAMA/dasllama/{name}", - "backend file with role '{role}' that REVIEW_GPU.md's partition does not carry - add the role to the checklist's list and to this gate's set in the same change") + "backend file with role '{role}' that the partition does not carry - add the role to ARCHITECTURE.md sec.1.5's role table and this gate's set in the same change") } } } diff --git a/modules/dasLLAMA/REVIEW_GPU.md b/modules/dasLLAMA/REVIEW_GPU.md index e173d7af9f..fd43b0f27d 100644 --- a/modules/dasLLAMA/REVIEW_GPU.md +++ b/modules/dasLLAMA/REVIEW_GPU.md @@ -10,23 +10,23 @@ mirrors applies this list together with `REVIEW.md`.** the choice at compile time instead.** A `class template` / `def abstract` / `def override` splice is compile-time and conforms - check the emission, not the das spelling. -**Never give a `*_decline_caps` predicate a parameter beyond the model and the call shape - -window-setup state is asked by `prefill_decline` / `decode_decline` instead.** A caps -parameter that reports the session's setup progress - rather than the CALL, its row count or -its span shape - is a defect however it is derived. +**Never give a `*_decline_caps` predicate a parameter beyond the model and the call shape - its +row count and its span shape - however that parameter is derived; window-setup state is asked +by `prefill_decline` / `decode_decline` instead.** **A bounds or tail guard that branches per iteration in a kernel's main loop, where the host already knows its answer as it picks the pipeline, is a defect - stamp the guard instead.** -Stamped means the guard rides a `@template_constant` - a `static_if` block, or a value select -on the constant. The guard-free instance is the one stamped without the guard, and the guard -is absent from that instance's generated `*_msl` global. A diff that edits an existing -kernel's loop answers to this exactly as a new kernel class does. +Stamped means the guard is carried by a `@template_constant` - a `static_if` block, or a value +select on the constant. The guard-free instance is the one stamped without the guard, and the +guard is absent from that instance's generated `*_msl` global. **Never let a `matmul2d` left or right operand reach the op as `float` outside a kernel class stamped `[metal_kernel(float_a_ok=true)]` - convert it in the pass that writes the operand's -buffer, or in the staging loop that reads it.** The stamped set is ledgered in -`ARCHITECTURE.md` sec.2.2b, and a stamp that lands without its ledger line is a defect. A -float operand keeps the op off its native fast path. +buffer, or in the staging loop that reads it.** A float operand keeps the op off its native +fast path. + +**A diff that stamps a kernel class `[metal_kernel(float_a_ok=true)]` lands its +`ARCHITECTURE.md` sec.2.2b ledger line in the same change.** **Never threadgroup-stage a `matmul2d` operand whose staged form matches its stored form - stream it from device instead.** A dequant, a transpose, or a layout or element-type change @@ -38,7 +38,7 @@ and is already coalesced. Per-element strided staging with div/mod addressing pa of what the contiguous form costs. **Never decide a kernel row's validity or owner by scanning the per-bucket base and count -arrays - read the one per-row entry instead.** The bucket-building kernel stamps that per-row +arrays - read the one per-row entry instead.** The bucket-building kernel writes that per-row entry. The scan repeats on every thread of every row's threadgroup, and it grows with the bucket count. @@ -49,8 +49,8 @@ padded chunk's walk can run past the live extent, and one poisoned read in a sha corrupts real rows. **Never leave a pipeline of dispatches with fewer scratch buffers than it has dispatches in -flight - add buffers until it has at least that many, or give each dispatch site its own.** -One shared scratch serializes the whole chain through its write-after-read hazards. +flight - give each dispatch site its own instead.** One shared scratch serializes the whole +chain through its write-after-read hazards. **A diff that adds dispatches to an encoder path to save bandwidth also gates that path on work size, in the same change.** The gate's threshold is measured at both ends of the size @@ -60,19 +60,21 @@ ladder. The small-work regression hides behind the big-work win. constant's authoritative site, in the same change.** An in-body tile constant is confirmed literal in the generated `*_msl` global or the SPIR-V dump. A grid or threadgroup constant is read off the class's `[metal_dispatch]` / `[vk_dispatch]` `grid=`/`tg=` spec, whose `"n/c"` -form is a CEIL-divide; the spec alone decides, and no builder read is needed. A uniform's -value is read at the single writer that fills its buffer, and nowhere upstream of it. +form is a CEIL-divide; the spec alone decides. A uniform's value is read at the single writer +that fills its buffer. **A kernel twin that binds a different kargs (kernel-argument struct) type than its sibling twin, or shifts a shared field to a different binding number, is a defect - even where one twin ignores that field.** Kernel twins are kernel classes whose bodies differ on one stamp -axis. +axis - one compile-time choice, such as single/batch, format, or single-pass/chunked. + +**A copy-pasted kernel twin, or a kernel split into hand instances where a `static_if` on a +`@template_constant` serves, is a defect - kernel twins stamp one `class template`, whatever +the stamp axis is (single/batch, format, single-pass/chunked).** Body divergence is carried by +a `@template_constant`, or by an overridden method spliced flat at emission. -**A copy-pasted kernel twin, a kernel split into hand instances where a `static_if` on a -`@template_constant` serves, and a dummy-bound field where a gate serves are defects - kernel -twins stamp one `class template`, whatever the stamp axis is (single/batch, format, -single-pass/chunked).** Body divergence rides the stamp axis - a `@template_constant`, or an -overridden method spliced flat at emission. A stamp-varying binding rides `@template_gate`. +**A dummy-bound field where a gate serves is a defect - a stamp-varying binding is carried by +`@template_gate` instead.** **A diff that forks a kernel class out of a shared template shows that the bodies no longer differ on a single stamp axis.** The same diff names the axis that is gone, in the surviving @@ -80,15 +82,18 @@ template's comment. **A `[metal_dispatch]` / `[vk_dispatch]` field whose memory is load-once - a model plane, or an `upload_region` upload never written after arming - is a defect unless it carries -`@role = "weight"`, even when the kernel compiles and passes parity.** `weight` on per-encode -data - a pooled buffer the host refills each encode - is a defect too: it drops the hazard -staging. A per-encode field either omits `@role` or names the access its body performs. +`@role = "weight"`, even when the kernel compiles and passes parity.** + +**`@role = "weight"` on per-encode data - a pooled buffer the host refills each encode - is a +defect; a per-encode field either omits `@role` or names the access its body performs.** +`weight` drops the hazard staging. **A new kernel class carries `[metal_dispatch]` / `[vk_dispatch]` with every annotation the generated builder reads - per-field `@binding` / `@role` / `@off` / `@span` / `@default`, -`@workgroup` state with its `tgmem=` dispatch key.** A NEW hand-written `enc_*` body is a -defect unless it is a wrapper - a format or twin pick, a default-filling wrapper, or a -composite over generated builders. +`@workgroup` state with its `tgmem=` dispatch key.** + +**A NEW hand-written `enc_*` body is a defect unless it is a wrapper - a format or twin pick, a +default-filling wrapper, or a composite over generated builders.** **A hand-rolled bind list that dispatches a kernel in `dasllama/` or `performance/` is a defect - dispatch through the kernel's `enc_*` builder instead.** @@ -100,29 +105,20 @@ host-side never reaches the device, so it does not count. **Never bind a scalar that the other bound scalars already determine - derive it in the builder instead.** Binding it separately adds a second place to get it wrong. -**Never key a cache on a host address alone - carry the span and the form in the key too.** A -hit must cover the request, and different upload forms live in separate tables. +**Never key a cache on a host address alone - carry the span and the form, the element type and +layout the upload produces, in the key too.** A hit must cover the request. **Never put a backend-only capability outside its backend's file for the matching role - a -capability with no matching role gets its own role file.** The `dasllama_metal_*` and -`dasllama_vulkan_*` files partition into these roles: the kernel home (`_kernels` on Metal, -`_classes` on Vulkan), `_common` (device state and plumbing), `_decode`, `_prefill`, `_gemm` -(the Metal batch-GEMM donor backend), `_shapes` (portable servability gates), `_tower` (the -encoder-tower driver), `_asr_dec` (the ASR-decoder driver), `_seams` (the Vulkan single-op -resident-driver seams), and the kernel-access lens (`_lens` on Metal, `_dispatch` on Vulkan). -Vulkan's backend entry is `dasllama/dasllama_math_vulkan.das`, which carries the capability -probe, the arm, the `.dlim` identity source, and the `[init]` that installs every hook. Never -add a role's file to a backend that does not have the capability. +capability with no matching role gets its own role file.** `ARCHITECTURE.md` sec.1.5 carries +the role table. + +**Never add a role's file to a backend that does not have the capability.** **A module that creates its own GPU device or queue is a defect - a GPU family shares the one device and queue from `dasllama/dasllama_<gpu>_common.das`'s init.** -**Never compile or release a Metal PSO serving the engine outside the file that owns its -kernel class** - it goes through that file's init/release pair: `metal_decode_init` / -`metal_kernels_release` in `dasllama/dasllama_metal_kernels.das`, `metal_prefill_init` / -`metal_prefill_shutdown` in `dasllama/dasllama_metal_prefill.das`. A pipeline that a -per-kernel unit test under `tests/` builds for its own run is that test's to compile and -release. +**Never compile or release a Metal PSO from an engine file (`dasllama/`) other than the one +that owns its kernel class** - it goes through that file's own init/release pair. **Never put race code outside the file that owns the kernel family - the shared scaffolding (`race_buf`, `race_envelope_ok`, `race_pair_ms`) belongs to @@ -151,7 +147,7 @@ where it is NEGOTIATED, not where it binds.** The bind site cannot shrink a buff sized wrong. **A change to code that a served GPU decode or prefill path executes ships GPU-vs-CPU parity -on one q8 and one kq model with the armed mirror codec.** That code is a driver +on one q8 and one kq (K-quant) model with the armed mirror codec.** That code is a driver (`dasllama/dasllama_metal_decode.das`, `dasllama/dasllama_metal_prefill.das`, `dasllama/dasllama_vulkan_decode.das`, `dasllama/dasllama_vulkan_prefill.das`), a kernel class one of them dispatches, that class's builder, the servability gates @@ -180,7 +176,7 @@ fix in the same change. **A change to the bake-trim path in `dasllama/dasllama_gpu_resident.das` (`trim_model_planes`) ships a `dasllama-convert --trim` bake plus a serve of the trimmed image, on one q8 and one kq -model.** Parity runs never reach it. +(K-quant) model.** Parity runs never reach it. **A change to `dasllama/dasllama_metal_asr_dec.das`, to `dasllama/dasllama_metal_common.das`, or to any kernel class the ASR decoder dispatches or builder the ASR decoder borrows, ships @@ -200,11 +196,13 @@ serves both codecs, so a codec no kernel covers silently drops that codec's GPU **A resident override that touches the mirror before gating the session on the armed mirror codec and on the flat (non-paged) cache is a defect** - a resident override is a decode/prefill hook the whole-model residency rail registers in -`dasllama/dasllama_common.das`'s override registries. An override that byte-copies mirror -bytes across codecs is a defect too: bytes move only between same-codec session rows and -mirror rows, and a cross-codec copy corrupts the host authority. +`dasllama/dasllama_common.das`'s override registries. + +**An override that byte-copies mirror bytes across codecs is a defect - bytes move only between +same-codec session rows and mirror rows.** A cross-codec copy corrupts the host's authoritative +cache. -**Never cache a descriptor set across dispatches in state `vk_drop_model_state` does not +**Never cache a descriptor set across dispatches in state that `vk_drop_model_state` does not clear** - put it in a `*_ready` latch, or in a field inside `g_gpu` or the weight arena in `dasllama/dasllama_vulkan_common.das`. diff --git a/modules/dasLLAMA/benchmarks/REVIEW.md b/modules/dasLLAMA/benchmarks/REVIEW.md index 8df581ccc5..e0beedb964 100644 --- a/modules/dasLLAMA/benchmarks/REVIEW.md +++ b/modules/dasLLAMA/benchmarks/REVIEW.md @@ -1,39 +1,37 @@ # dasLLAMA benchmarks Code Review Checklist **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture -doc: `../PROFILE.md`. Planned work: `../PERF_LEDGER.md` for a performance followup, -`../followup_general.md` for everything else. - -**A diff that adds or changes an instrument whose timed body dispatches a pipeline the model -runtime selected also calls `tune_gate()` (`../performance/profile_common.das`) before that -instrument's first timed rep.** A pipeline the instrument compiled itself is not one the -runtime selected. Tokenizing and detokenizing run no forward pass, so nothing there is gated: -`encode`, `encode_`, `decode`, and `decode_`, on a `Model` or a `Tokenizer`. Without the gate -the instrument measures fallback kernels silently. - -**A diff that adds or changes a race times both arms interleaved in one process.** A race is -an instrument that compares two implementations. A Metal race interleaves through -`race_pair_ms`. Two separate runs measure the box's drift between them as much as they measure -the arms. - -**A diff that adds or changes an A/B arm that produces no comparable output carries the -literal token `timing-only` in the arm's report line.** The rule fires on an arm reported as -adoptable evidence. An arm with no comparable output exists only to time work. - -**A diff that adds or changes an A/B arm that computes the baseline's result in a different -precision prints a bounded-difference compare.** The rule fires on an arm reported as -adoptable evidence. The compare runs against the baseline arm or against the CPU reference. -The report line also prints the bound the arm passed. - -**A diff that adds or changes an A/B arm that computes the baseline's result in the same -precision prints the bit-exact compare over the sampled region.** The rule fires on an arm -reported as adoptable evidence. The sampled region is the set of output elements the run -compares. The compare prints on the report's "bit-exact vs ..." line. - -**A diff that adds or changes a race that picks between two implementations also checks the -race's baseline arm against a CPU reference.** The baseline arm is the arm the race already -trusts. The reference check runs in the same process, on the same output elements the arms are -judged on. Two arms can agree and both be wrong; only the reference makes the winner right. +doc: `../ARCHITECTURE.md` (the benchmark rig is sec.2.5). Planned work: `../PERF_LEDGER.md` for +a performance followup, `../followup_general.md` for everything else. + +**A diff that adds or changes an instrument whose timed body runs a forward pass through a +pipeline the model runtime selected also calls `tune_gate()` +(`../performance/profile_common.das`) before that instrument's first timed rep.** A pipeline +the instrument compiled itself is not one the runtime selected. Without the gate the instrument +measures fallback kernels silently. + +**A diff that adds or changes a race times both arms interleaved in one process - a Metal race +through `race_pair_ms`.** A race is an instrument that compares two implementations. Two +separate runs measure the box's drift between them as much as they measure the arms. + +**A diff that adds or changes a race arm offered as the reason to adopt a change, where the arm +produces no comparable output, carries the literal token `timing-only` in the arm's report +line.** + +**A diff that adds or changes a race arm offered as the reason to adopt a change, where the arm +computes the baseline's result in a different precision, prints a bounded-difference compare.** +The compare runs against the baseline arm or against the CPU reference. The report line also +prints the bound the arm passed. + +**A diff that adds or changes a race arm offered as the reason to adopt a change, where the arm +computes the baseline's result in the same precision, prints the bit-exact compare over the +sampled region.** The sampled region is the set of output elements the run compares. The +compare prints on the report's "bit-exact vs ..." line. + +**A diff that adds or changes a race also checks the race's baseline arm against a CPU +reference.** The baseline arm is the arm running the implementation already in use. The +reference check runs in the same process, on the same output elements the arms are judged on. +Two arms can agree and both be wrong; only the reference makes the winner right. **A diff that adds or changes a knockout or sweep instrument carries the literal text `ATTRIBUTION SWEEP` in the instrument's file header comment.** A knockout or sweep instrument @@ -57,6 +55,6 @@ measured wall from another also prints both raw walls on that report line.** **A diff that changes what a board cell times ships before/after rows for each affected cell and corpus.** A board cell is a timed cell of the published results board: one `../performance/gen_bench_records.das` spawns, or a manual `lcpp_bench.das` cell with its own -`../PROFILE.md` section. What a cell times changes when its code changes, when its input -corpus changes, or when the pinned reference build changes (`DEFAULT_REF_SHA` in -`setup_lcpp_ref.das`). The diff may instead state that the measured quantity is unchanged. +`../PROFILE.md` section. What a cell times changes when a change to its code, to its input +corpus, or to the pinned reference build (`DEFAULT_REF_SHA` in `setup_lcpp_ref.das`) moves the +measured quantity. diff --git a/modules/dasLLAMA/followup_vulkan.md b/modules/dasLLAMA/followup_vulkan.md index 31b789919d..0e2240c7e3 100644 --- a/modules/dasLLAMA/followup_vulkan.md +++ b/modules/dasLLAMA/followup_vulkan.md @@ -213,3 +213,31 @@ module) is independent and can land any time - it is pure structure. `ensure_*` runs dry). The live path declines residency in `vk_rdec_prepare`; the plan side needs `max_wg_bytes` in the probed config + the dlim identity, mirroring `max_storage_range`. + +20. **A coopmat2-class analogue for the tmm2d family.** dasMetal exposes a tensor-op family + the kernel classes call directly - `tmm2d_tg_begin` / `tmm2d_tg_step` / `tmm2d_tg_store` + plus the per-format decode arms (`tmm2d_q8_f32`, `tmm2d_q8u_f32`, `tmm2d_f16w_f32`, + `tmm2d_f32_bf16_f32`, ...), all access-classified in `dasllama_kernel_access.das`. Vulkan + has no twin: the cm2 work (item 11) is a prefill GEMM arm on + `VK_NV_cooperative_matrix2`, not a builtin family a kernel class composes. Decide whether + the tmm2d shape maps onto coopmat2 tiles, and if it does, expose it as the same builtin + family on the SPIR-V side so one class body serves both backends - rather than a second, + Vulkan-only GEMM dialect beside cm2. + +21. **Metal-side grid-literal validation.** `[vk_dispatch]` rejects a divide-form `grid` + whose referenced name is not a declared `int64` param, and says why in the message (the + generated ceil-div math is int64, and daslang has no promotion). `[metal_dispatch]` + checks only that a grid dim carries at most one `/`, so the same mistake on Metal + surfaces as an infer error inside generated code instead of a lens diagnostic. Both + lenses already call the shared `mk_grid_dim` in `dasllama_kernel_access.das` - move the + check there too, so it fires once for both. + +22. **Metal kernel-corpus inheritance dedup.** The Vulkan classes factor their families into + base + leaves - `KqGemvBase` with `KqGemvK4/Q40/K5/K6`, `KqBatchBase` with its batch + leaves, `MoeCmBase` with the cm/mm leaves, `DnScanBase` with `DnScanP1/P2`. Metal does + this in places (`MetalMoeMulMmK6` and its siblings sit on `MetalMoeMulMmBase`), but its + GEMV families do not: `MetalKqGemvK4/K5/K5C/K6` and + `MetalMoeGemvQ8/K4/K5/K6/Mx4/Q51` are flat classes repeating the identical + `x`/`y`/`ndim`/`ddim` binding block, differing only in the weight-plane views and the + decode. Give those two families a base the way `MetalMoeMulMmBase` already does, so a + binding or epilogue fix lands once per family instead of once per variant. diff --git a/modules/dasMetal/ARCHITECTURE.md b/modules/dasMetal/ARCHITECTURE.md index 08bec0370d..1db3f2ce64 100644 --- a/modules/dasMetal/ARCHITECTURE.md +++ b/modules/dasMetal/ARCHITECTURE.md @@ -17,11 +17,12 @@ stays dasVulkan's job. The shader-language facts that shape the design: **LLVM cannot emit MSL** (no backend; AIR, Metal's binary form, is a private version-unstable LLVM-IR dialect) and **Metal does not ingest SPIR-V** - its sanctioned inputs are MSL source and DXIL. Translation libraries -exist (SPIRV-Cross is MoltenVK's own shader stage), but vendoring one is a third-party C++ -dependency that contradicts this module's thesis: *Metal.framework is just there*. +exist (SPIRV-Cross is the shader translator MoltenVK itself uses), but vendoring one is a +third-party C++ dependency that contradicts this module's premise - Metal.framework ships with +the OS, so there is nothing to vendor. -So dasMetal uses the house pattern, already proven three times: dasGlsl emits **GLSL text** -from the daslang AST (`GlslExport`), dasSpirv emits **SPIR-V words**, and `daslib/aot_cpp.das` +So dasMetal uses the house pattern: dasGlsl emits **GLSL text** from the daslang AST +(`GlslExport`), dasSpirv emits **SPIR-V words**, and `daslib/aot_cpp.das` emits **C++ for the entire language**. MSL is a C++14 dialect; a compute-only MSL text emitter is squarely inside the established pattern. Just as `[compute_shader]` lowers daslang->SPIR-V->Vulkan, `[metal_kernel]` lowers daslang->MSL->Metal. @@ -34,14 +35,14 @@ reused as-is. ## 2. Settled decisions 1. **Lives in the main tree as `modules/dasMetal`.** The C++ binding is APPLE-gated; the - `metal/` daslang files (emitter + annotation) are **pure das and registered on ALL - platforms** - emitter/text tests run on every CI lane; only GPU execution is Apple-only. + `metal/` daslang files (emitter + annotation) are **pure das and registered on every + platform** - emitter/text tests run on every CI lane; only GPU execution is Apple-only. 2. **Compute only.** No render pipelines, no drawables, no CAMetalLayer. Graphics = dasVulkan. 3. **Zero third-party code.** No metal-cpp, no SPIRV-Cross, no MoltenVK, no committed `.msl`/`.metallib`, no external SDK. The binding is a hand-written Obj-C++ shim (`src/dasMetal.mm`) over the system Metal.framework - the compute subset only (sec.3). (metal-cpp + dasClangBind rejected: vendored headers and binder churn for a surface this - small; revisit only if it outgrows hand maintenance.) + small.) 4. **Class-based authoring: kernels are class methods, resources are class members.** A compute pass is a `class` whose `@ssbo @binding = N` members declare the buffers and whose `[metal_kernel]` methods are the kernels - no module-scope resource globals (they @@ -50,26 +51,22 @@ reused as-is. one buffer set = multiple methods in one class. Builtin globals (`gl_GlobalInvocationID`, ...) come from the shared `daslib/shader_lingua_franca`, which `metal/metal_builtins.das` re-exports. Because the body is ordinary daslang, the same - method also executes on the CPU - the primary correctness oracle (sec.4). **Zero edits to - shipped dasSpirv/dasGlsl.** + method also executes on the CPU - the primary correctness oracle (sec.4). 5. **Naming.** C++ module `das_metal` (`Module("das_metal")`, class `Module_DasMetal`). das files under `metal/`: `require metal/msl_shader`, `require metal/das_metal_boost`. Consumers guard: `require ?das_metal metal/das_metal_boost` + `static_if (typeinfo builtin_module_exists(das_metal))` (the sql_boost provider pattern). -6. **Test-per-construct is a hard requirement**, enforced by a construct census (gate B) + - the real-frontend compile gate. LCOV covers runtime files only - the emitter runs at - compile time, invisible to line coverage (dasSpirv finding; census is the proxy). -7. **No `.msl` file is committed and no MSL is hand-written.** MSL exists only as text the - emitter produces at compile time. +6. **Coverage of the emitter is the construct census (gate B), not LCOV.** LCOV covers runtime + files only - the emitter runs at compile time, invisible to line coverage. +7. **MSL exists only as text the emitter produces at compile time.** 8. **`fastmath` is a `[metal_kernel]` property, ON by default.** daslang's own posture is fastmath-on, so we are not chasing bit-exact float parity; Metal's runtime compiler also defaults fast-math ON and we keep it. Float oracles compare with tolerance (ints bit-exact). `[metal_kernel(fastmath=false)]` per kernel when isolating a divergence needs strict IEEE. -9. **Every `matmul2d_descriptor` sets `relaxed_precision = true`.** RP = false keeps the - op off the M5 tensor unit's fast path - measured 2-3x across the tmm2d families; the - cooperative-tensor register layout the fast path uses is what RP licenses. `REVIEW.das` - enforces the emitter's descriptor sites. +9. **`relaxed_precision = false` keeps a `matmul2d` off the M5 tensor unit's fast path** - + 2-3x across the tmm2d families. The cooperative-tensor register layout the fast path uses + is what relaxed precision licenses. ## 3. Files and lowering mechanism @@ -77,40 +74,20 @@ reused as-is. | File | Gen/Hand | Purpose | |---|---|---| +| `src/dasMetal.h` | hand | The C++ header `src/dasMetal.mm` and any embedder share: the opaque handle struct declarations and the `DAS_MOD_API` prototype of every extern below. Declarations only - no implementation, no Obj-C, no binding registration. | | `src/dasMetal.mm` | hand | `Module("das_metal")` - Obj-C++ shim over Metal.framework. Opaque annotated handles (device, queue, command buffer, compute encoder, pipeline state, library, function, buffer) + the extern surface below. Compiled with ARC; handles cross to das as `__bridge_retained void*`; `metal_release` = `__bridge_transfer`. Shim-side live-object counter for the leak gate. APPLE-only; links `-framework Metal -framework Foundation`. | -| `metal/msl_types.das` | hand | daslang `TypeDecl` -> MSL type name (32-bit scalars/bool + the 16/8-bit lattice - MSL has native `half`/`short`/`char` - and their 2/3/4 vectors, classified via the shared `daslib/shader_block_layout` rails). `msl_buffer_elem_name` gives the layout-bearing spelling: 3-lane elements take MSL's `packed_T3` (das packs tightly; unified memory means the das array IS the buffer). | -| `metal/metal_builtins.das` | hand | Metal-only builtin surface over the shared lingua franca (the spirv_builtins pattern): re-exports `daslib/shader_lingua_franca` (whence `gl_WorkGroupSize` and the four `gl_Subgroup*` IDs, shared with dasSpirv) and adds the `simd_sum`/`simd_shuffle*` intrinsics (Metal spellings, float/int/uint). Identity stub bodies = width-1 simdgroup CPU semantics. | +| `metal/msl_types.das` | hand | daslang `TypeDecl` -> MSL type name: 32-bit scalars/bool plus the 16- and 8-bit scalars (`half`/`short`/`char` in MSL) and their 2/3/4 vectors, classified via the shared `daslib/shader_block_layout` rails. `msl_buffer_elem_name` gives the layout-bearing spelling: 3-lane elements take MSL's `packed_T3` (das packs tightly; unified memory means the das array IS the buffer). | +| `metal/metal_builtins.das` | hand | Metal-only builtin surface over the shared lingua franca (the spirv_builtins pattern): re-exports `daslib/shader_lingua_franca` (whence `gl_WorkGroupSize` and the four `gl_Subgroup*` IDs, shared with dasSpirv) and adds the `simd_sum`/`simd_shuffle*` intrinsics (Metal spellings, float/int/uint). The CPU bodies return their argument unchanged - on the CPU a simdgroup is one lane. | | `metal/msl_emit.das` | hand | The text emitter: `generate_msl(fn, var errors, cfg, var census, var tgmem) : string`. Manual recursion (`emit_value`/`emit_stmt`, mirroring `spirv_emit`). Kernel-signature synthesis from `@ssbo` globals (the one structural novelty - below). Records the construct census at every emit site. | -| `metal/msl_shader.das` | hand | `[metal_kernel]` function-macro (`MetalKernel : AstFunctionAnnotation`, modeled on `SpirvShader`), applied to a **class method**; args: `name`, `fastmath` (default **true**). `apply` declares the public MSL-text global - the `name=` argument, or `<Class>_<method>_msl` derived from the method - plus a `<name>_fastmath : bool` companion feeding the pipeline-compile options; **`fixup` fills `glob.init = new ExprConstString(...)`** - string capture is call-free, so fixup suffices (dasGlsl precedent). Does `require msl_emit public` + `require metal_builtins public`. | +| `metal/msl_shader.das` | hand | `[metal_kernel]` function-macro (`MetalKernel : AstFunctionAnnotation`, modeled on `SpirvShader`), applied to a **class method**; args: `name`, `fastmath` (default **true**). `apply` declares the public MSL-text global - the `name=` argument, or `<Class>_<method>_msl` derived from the method - plus a `<name>_fastmath : bool` companion feeding the pipeline-compile options; **`fixup` fills `glob.init = new ExprConstString(...)`** - the initializer is a constant string, so nothing has to be called to build it and the late `fixup` pass is enough. Does `require msl_emit public` + `require metal_builtins public`. | | `metal/das_metal_boost.das` | hand | Host sugar over `das_metal`: `with_metal_device`, `pipeline_from_kernel` (compile + error surfacing), unified-memory buffer helpers, `run_compute_1d` one-liner, live-object leak assert. `require das_metal` -> usable only where the C++ module exists. | | `CMakeLists.txt` | hand | `ADD_MODULE_DAS_FROM_DESCRIPTOR(metal metal)` unconditional (rows derived from `.das_module`, emitter everywhere); `IF(APPLE)`: `ADD_MODULE_CPP(DasMetal)` + `ADD_MODULE_LIB` + frameworks. Install rule mirrors dasSpirv's. | -**Extern surface.** `src/dasMetal.mm` binds the compute subset of Metal.framework, in -families: - -- **Device + queue:** `metal_create_system_default_device`, `metal_device_name`, - `metal_device_has_unified_memory`, `metal_max_buffer_length`, `metal_new_command_queue`. -- **Library + pipeline:** `metal_new_library_from_source(dev, src, fastmath; var error)`, - `metal_new_function`, `metal_new_compute_pipeline`, `metal_pipeline_max_total_threads`, - `metal_pipeline_thread_execution_width`. -- **Buffers:** `metal_new_buffer` (storageModeShared) and its `_untracked` / `_no_copy` / - `_no_copy_untracked` variants; `metal_buffer_contents(buf) : void?` - the unified-memory - pointer the host reads and writes directly, with no map/unmap. -- **Encoding + dispatch:** `metal_new_command_buffer` (+ `_unretained`), - `metal_new_compute_encoder` (+ `_concurrent`), `metal_set_pipeline`, `metal_set_buffer`, - `metal_set_bytes`, `metal_set_threadgroup_memory_length`, `metal_dispatch_threadgroups`, - `metal_dispatch_threads` (the Apple-silicon exact-grid fast path), `metal_memory_barrier`, - `metal_end_encoding`, `metal_commit`, `metal_wait_until_completed`, - `metal_dispatch_call_count`. -- **Errors + timing:** `metal_command_buffer_error`, `metal_command_buffer_failed`, - `metal_command_buffer_gpu_start_time` / `_end_time`, `metal_command_buffer_kernel_start_time` - / `_end_time`. -- **Events:** `metal_new_shared_event`, `metal_shared_event_signal`, `metal_cb_wait_for_event`. -- **Residency:** `metal_new_residency_set`, `metal_residency_set_add_buffer`, - `metal_residency_set_commit`, `metal_residency_set_request`, `metal_residency_set_heartbeat`, - `metal_residency_heartbeat_live` / `_sets` / `_ticks`. -- **Lifetime:** `metal_release` (one overload per handle type), `metal_live_object_count`, - `metal_live_object_report`. +**Extern surface.** `src/dasMetal.mm` binds the compute subset of Metal.framework in eight +families: device+queue, library+pipeline, buffers, encoding+dispatch, errors+timing, events, +residency, lifetime. Two carry non-obvious contracts: `metal_buffer_contents(buf) : void?` is +the unified-memory pointer the host reads and writes directly, with no map/unmap; +`metal_dispatch_threads` is the Apple-silicon exact-grid fast path. **Kernel-signature lowering (the one structural novelty).** MSL has no module-scope device globals - and the authoring class's members map onto its model exactly: each `@ssbo` @@ -126,25 +103,25 @@ error. Identifiers colliding with MSL keywords (`kernel`, `device`, `constant`, (sec.7) and a `@workgroup` member to a threadgroup pointer derived from the entry's `_tgmem` companion. -## 4. Test architecture - "every emitted construct has a test" +## 4. Test architecture Three behavioral layers + enforcement gates: -1. **Text-assertion units** (`tests/msl/`, ALL platforms). Each test compiles a tiny +1. **Text-assertion units** (`tests/msl/`, repo root, every platform). Each test compiles a tiny `[metal_kernel]` fixture, calls `generate_msl`, asserts structural facts (signature shape, attribute per binding, const-ness, statement forms), plus a golden snapshot of - *our own* emitted text as a forward regression guard (dasSpirv's byte-snapshot amendment). + *our own* emitted text as a forward regression guard. 2. **Compile gate** (the spirv-val analog). Every emitted MSL must compile through the **real Metal frontend**: in-process `metal_new_library_from_source` where `das_metal` exists (no Xcode required - the OS MTLCompilerService), soft-skip elsewhere, hard-required on the macOS CI lane. (`xcrun metal` offline compile is a secondary local oracle only - it needs full Xcode, not CLT.) -3. **Real-GPU behavioral gate vs the CPU-reference oracle** (`tests/metal/`, Apple-only). - The kernel body is ordinary daslang, so the **same method runs on the CPU**: a driver loop - sets `gl_GlobalInvocationID` and calls the method on a class instance whose members are - plain arrays - that CPU run (interp/JIT) produces the expected buffer contents with zero - second-source effort. GPU results compare against it: ints bit-exact, floats with tolerance - (fastmath on both sides - sec.2 decision 8). Files are `require ?das_metal` + +3. **Real-GPU behavioral gate vs the CPU-reference oracle** (`tests/metal/`, repo root, + Apple-only). The kernel body is ordinary daslang, so the **same method runs on the CPU**: + a driver loop sets `gl_GlobalInvocationID` and calls the method on a class instance whose + members are plain arrays - that CPU run (interp/JIT) produces the expected buffer contents + with zero second-source effort. GPU results compare against it: ints bit-exact, floats with + tolerance (fastmath on both sides - sec.2 decision 8). Files are `require ?das_metal` + `static_if builtin_module_exists` guarded, so they compile and no-op cleanly on non-Apple lanes. Primary gate = **local M-boxes**; the macOS CI lane runs the compile gate, because its Apple Paravirtual device is unproven for execution. @@ -157,100 +134,72 @@ Three behavioral layers + enforcement gates: census across all fixtures and asserts equality with the declared supported set, both directions. Census-record and emit are fused in one helper so they cannot drift; the golden-snapshot layer catches emissions that bypass it. -- **Leak gate.** Every `tests/metal/` file asserts `metal_live_object_count() == 0` at - exit - Metal objects are invisible to all six das leak detectors, so the shim counts. +- **Leak gate.** Metal objects are invisible to the das leak detectors, so the shim counts live + objects - `metal_live_object_count()`. ## 5. Cross-backend parity - the kernel-model asymmetry ledger -The mirror rule (REVIEW.md, both emitters) records deliberate or pending kernel-model -asymmetries HERE - one ledger for both backends (dasSpirv's `ARCHITECTURE.md` points at this -section). The primary correctness oracle remains the CPU-reference run of the same das body, -cheaper and stricter than a second GPU; cross-GPU parity of one source is a nice-to-have now -that both emitters read class members. Current entries: - -- **Class-member authoring is symmetric now.** dasSpirv gained `[spirv_kernel]` classes (this - emitter's model, ported wholesale in the vulkan class-kernel arc) - the original blocker on - stacking both shader annotations on one body is gone. Free-function callees and - devirtualized overrides are symmetric too (msl_emit's devirtualization table was the donor). -- **Multi-kernel-per-class is symmetric now; `family=` surface sharing stays Vulkan-only.** - Both lenses accept N kernel methods per class with `kernel=` picking per instance - (`_mtl_toy.das` / `_vkd_toy.das` are the sibling fixtures). `family=` remains Vulkan's - alone - Metal's `enc_*` builder is the whole generated surface, so there is nothing for a - family to share. -- **Method flattening, scope splicing, and plain-member thread-local STATE are Metal-only.** +`modules/REVIEW_SHADER_EMITTERS.md` requires a diff that adds a kernel-model capability to one +emitter to add it to the other or record the asymmetry here - one ledger for both backends. The +primary correctness oracle is the CPU-reference run of the same das body - cheaper and stricter +than a second GPU. Cross-GPU parity of one source is secondary. Current entries: + +- **`family=` surface sharing is Vulkan-only.** Both emitters accept N kernel methods per class + with `kernel=` picking per instance (the sibling fixtures are + `modules/dasLLAMA/tests/_mtl_toy.das` and `modules/dasLLAMA/tests/_vkd_toy.das`); `family=` + has no Metal counterpart because Metal's `enc_*` builder is the whole generated surface, so a + family has nothing to share. +- **Method flattening, scope splicing, and plain-member thread-local state are Metal-only.** `msl_emit` splices method bodies inline (statically devirtualized, value and statement position) and lowers plain class members to kernel-entry thread-locals (scalar-with-init, - fixed array, carried pointer walk - the state contract is per-thread); `spirv_emit` still - rejects an unannotated member and lowers method calls as calls. If a joined or - state-carrying class ever crosses backends, the same design ports - deliberate asymmetry - until then. -- **The tmm2d tensor-GEMM builtin family is Metal-only (pending, not deliberate).** The - whole-GEMM helpers (`tmm2d_f32_bf16_f32`/`_f16w_f32`/`_q8*_f32` with the bk staging - depths) and the staged tg protocol (`begin/step` in `deva`/`nt`/`devant` flavors, - `tile_tg`, `rowscale`, the `store`/`store_half`/`store_bias` tails, `sgmat_to_half`) - lower MPP `matmul2d` cooperative tensors; the Vulkan analogue would sit on the coopmat2 - classes when a consumer needs it. Until then the CPU-replay bodies are the only - cross-checked twin. -- **Inheritance in the kernel corpus: Vulkan leans on it, Metal does not yet.** The vulkan - classes are base+leaf families (kq GEMV/batch, flash attention, deltanet); the metal - classes are flat, with at least one base duplicated inline (MetalMoeMulMmK6). Dedup - opportunity, not a correctness gap. -- **Grid-literal validation: Vulkan rejects non-int64 ceil-div grid params at macro apply; - the metal lens still carries that silent infer trap.** Pending metal-side fix. -- **cm2 decode-in-load (and its class-method decode form) is Vulkan-only by hardware.** - `[spirv_decode]` - including the method form, where a das-level self is erased from the - rigid 3-param signature so the decode body reads class members (a separate scale plane, - push constants, @workgroup staging) - lowers SPV_NV_cooperative_matrix2 tensor loads. - Metal-4 tensors have no decode-callback analogue; the Metal quant GEMMs stage dequant - through threadgroup memory instead. Deliberate, target-specific - not a pending port. - -## 6. Verification - -- **Standing per-change gate (main tree):** - `daslang dastest/dastest.das -- --test tests/msl --isolated-mode` green on every platform; - `tests/metal` green on Apple; census == declared set both directions; every emitted kernel - compile-gate clean; no `GC APP LEAK`; `metal_live_object_count()` 0. -- **M-box run book (behavioral + parity):** `bin/daslang tests/metal/...` on the M-boxes; the - zen2 side runs the same parity fixtures through dasVulkan. -- **Lint/format:** MCP `format_file` + `lint` on every new `.das`; `cpp_format_file` on the - `.mm`. PR-mode throughout. + fixed array, carried pointer walk - the state contract is per-thread); `spirv_emit` rejects + an unannotated member and lowers method calls as calls. The asymmetry is deliberate. +- **The tmm2d tensor-GEMM builtin family is Metal-only; the asymmetry is pending, not + deliberate.** The family - whole-GEMM helpers plus the staged threadgroup protocol - lowers + MPP `matmul2d` cooperative tensors, and the CPU-replay bodies are its only cross-checked + twin. +- **Grid-literal validation is asymmetric; pending, not deliberate.** Vulkan rejects a + non-int64 ceil-div grid parameter at macro apply; the Metal emitter accepts it and infers a + type silently. +- **cm2 decode-in-load is Vulkan-only by hardware.** `[spirv_decode]` lowers + SPV_NV_cooperative_matrix2 tensor loads. Metal-4 tensors have no decode-callback analogue, so + the Metal quant GEMMs stage dequant through threadgroup memory instead. Deliberate, + target-specific - not a pending port. ## 7. `@uniform` structs - the kargs form -A kernel with a dozen scalars used to cost a dozen binds. `@uniform @binding = N ka : KArgs`, -where `KArgs` is a plain das struct, costs one: the struct definition emits into the MSL -preamble, the member becomes `constant KArgs& ka [[buffer(N)]]`, and `ka.field` is one hop. +A kernel with a dozen scalars would otherwise cost a dozen binds. +`@uniform @binding = N ka : KArgs`, where `KArgs` is a plain das struct, costs one: the struct +definition emits into the MSL preamble, the member becomes `constant KArgs& ka [[buffer(N)]]`, +and `ka.field` is one hop. **Fields are `int`/`uint`/`float` only.** That restriction is the whole feature: a struct of 4-byte scalars is the same bytes in das and in MSL, with no padding rules to keep in sync, so the host writes its own struct straight through `setBytes` (`metal_set_bytes`, already bound - Metal caps it at 4KB and a kargs struct is well under). A `float4` field would be 16-byte aligned in MSL and shift every field after it; a nested struct brings its own alignment. Both -are refused (`tests/msl/_fail_closed/_fc_ustruct_{field,nested}.das`). +are refused (`tests/msl/_fail_closed/_fc_ustruct_{field,nested}.das`, repo root). -Host side: `run_compute_1d_kargs` for tests, and dasLLAMA's `kn_bytes` / `kn_kargs` on the -capture rail - under graph capture the bytes are COPIED into a pool, since the caller's kargs is -a stack local long gone by replay time. +Host side: `run_compute_1d_kargs` for tests, and dasLLAMA's `kn_bytes` / `kn_kargs`, which copy +the bytes into a pool under graph capture - the caller's kargs is a stack local, long gone by +replay time. **It costs the GPU nothing.** Every field was already a runtime `constant uint&`; the fold trades N bindings for one argument-buffer read and the kernel branches on exactly the values it did -before. Nothing about the *shape* of a kernel may become a kargs field, though: a block stride, a -lane width, a codec selector belongs in a per-codec overload or a monomorphized generic, where it -stays a literal in the emitted MSL. Passing one as a value and trusting Metal to inline-and-fold -it back is an assumption, not a guarantee - and the assumption is worth nothing in the kernels -that matter. +before. Metal inlining a passed shape value back into a constant is an assumption, not a +guarantee. -Tests: `tests/msl/test_msl_uniform_struct.das` (definition placement, the single `constant&` -parameter, no per-scalar parameter survives) and `tests/metal/test_metal_uniform_struct.das` -(GPU vs CPU-oracle - the only thing that can prove the layout claim; mutation-verified by -inserting one pad field into the emitted struct). +Tests: `tests/msl/test_msl_uniform_struct.das` (repo root - definition placement, the single +`constant&` parameter, no per-scalar parameter survives) and +`tests/metal/test_metal_uniform_struct.das` (repo root - GPU vs CPU-oracle, the only thing that +can prove the layout claim; mutation-verified by inserting one pad field into the emitted +struct). ## 8. Pointer parameters A helper takes a raw pointer: `def stage(var p : half4 const?; n : uint)`. This is what lets the -kernels that stream `unsafe(addr(buf[i]))` through a loop - the split-K "D" attention family, the -K-quant mul_mm trio - share a skeleton at all, since the advancing pointer is the shape and -rewriting it as index math changes AGX register allocation. +kernels that stream `unsafe(addr(buf[i]))` through a loop share one skeleton: the advancing +pointer is the shape, and rewriting it as index math changes AGX register allocation. **MSL requires an address space in the signature** (`device half4*` vs `threadgroup half4*`) and the das type `half4?` carries none - the same type can name an @ssbo interior or a @workgroup @@ -265,15 +214,14 @@ interior. The parameter declares it and the call site proves it: over any of those. Disagreement is a das error naming both sides; provenance it cannot trace (a thread-space local) is refused rather than guessed. The declared space is never trusted on its own, so a wrong marker cannot reach the runtime MSL compile as a null pipeline. -- **The POINTEE's const is the MSL const.** `T const?` lowers `device const T*` and reads only; +- **The pointee's const is the MSL const.** `T const?` lowers `device const T*` and reads only; `T?` lowers `device T*` and marks the source member written, exactly as a `var array<T>` parameter does. (Writing through it in das additionally needs the handle non-const - `var o : T?` - because das flows the handle's const onto the dereference.) -Long-term this is a shortcut for **pointer families** as a language-level feature - a real type -axis rather than an annotation. Parked on `modules/dasLLAMA/followup_general.md`. - -Tests: `tests/msl/_msl_common.das` + `test_msl_functions.das` (signatures, forwarding, inline -`addr` arguments, census), `tests/metal/test_metal_functions.das` (GPU advancing dot + threadgroup -slab max vs a directly-computed expectation), `tests/msl/_fail_closed/_fc_ptr_space_{tg,dev}.das` -and `_fc_ptr_untraceable.das` (both mismatch directions + the untraceable case). +Tests (all repo root): `tests/msl/_msl_common.das` + `tests/msl/test_msl_functions.das` +(signatures, forwarding, inline `addr` arguments, census), `tests/metal/test_metal_functions.das` +(GPU advancing dot + threadgroup slab max vs a directly-computed expectation), +`tests/msl/_fail_closed/_fc_ptr_space_{tg,dev}.das` and +`tests/msl/_fail_closed/_fc_ptr_untraceable.das` (both mismatch directions + the untraceable +case). diff --git a/modules/dasMetal/REVIEW.md b/modules/dasMetal/REVIEW.md index abbcf2aa5e..fff1c1214e 100644 --- a/modules/dasMetal/REVIEW.md +++ b/modules/dasMetal/REVIEW.md @@ -16,8 +16,9 @@ the diff puts it.** An emitted-text fixture answers to `tests/msl/REVIEW.md` (re kind lets either one go untested. - **A new or changed `[metal_kernel]` annotation argument ships a `tests/msl/` (repo root) - fixture in the same change.** The fixture asserts what the argument changes: the published - global it names, or the difference it makes to the emitted text. + fixture in the same change.** The fixture asserts what the argument changes: the global the + macro declares for it - a module-level global holding the kernel's MSL text or a compile + option - or the difference it makes to the emitted text. - **A new construct the MSL emitter rejects at compile time ships a `tests/msl/_fail_closed/` (repo root) fixture in the same change.** The same change asserts that construct's error @@ -25,12 +26,14 @@ the diff puts it.** An emitted-text fixture answers to `tests/msl/REVIEW.md` (re compile error that names the rejected construct. - **A kernel behavioral change ships a CPU-oracle test under `tests/metal/` (repo root).** A - CPU-oracle test compares the GPU result against a CPU-computed expectation. A regression in - dasMetal alone must fail this module's own tests. + CPU-oracle test compares the GPU result against a CPU-computed expectation. - **A change visible only in the emitted text ships a `tests/msl/` (repo root) fixture.** The fixture asserts the emitted text that the change alters. +- **Never commit a `.msl` file and never hand-write MSL - emit it from a `[metal_kernel]` body + instead.** + - **A new or changed host extern under `modules/dasMetal/src/` ships a host-side test under `tests/metal/` (repo root) in the same change.** A changed public function in `metal/das_metal_boost.das` fires this rule too. The test `feint`s when no Metal device is @@ -43,5 +46,5 @@ the diff puts it.** An emitted-text fixture answers to `tests/msl/REVIEW.md` (re - **Never zero a cooperative tensor element by element before a `matmul2d` `run` accumulates into it - `get_destination_cooperative_tensor` already hands it back zeroed.** That walk - forces every element into real storage before the K loop, and that costs the op its fast - path for the whole loop. + forces every element into real storage before the accumulation loop, and that costs the op + its fast path for the whole loop. diff --git a/modules/dasSpirv/ARCHITECTURE.md b/modules/dasSpirv/ARCHITECTURE.md index ca5bfcd7d0..86e597fffd 100644 --- a/modules/dasSpirv/ARCHITECTURE.md +++ b/modules/dasSpirv/ARCHITECTURE.md @@ -13,20 +13,20 @@ committed, and loaded at runtime - two languages, an external SDK, committed bin code-sharing with the host. dasVulkan carries none of that: no GLSL, no committed `.spv`, and no glslang or SDK dependency. -dasGlsl already eliminated exactly this for OpenGL: shaders written in daslang, annotated +dasGlsl does this for OpenGL: shaders written in daslang, annotated `[..._program]`, an `AstVisitor` (`GlslExport`) emits GLSL at compile time into a global captured by `@@fn`, fed to `glShaderSource`. dasSpirv does the same for Vulkan, but emits **SPIR-V binary directly from the daslang AST** - no GLSL/glslang intermediary, no LLVM. SPIR-V is an SSA IR (typed results, basic blocks, structured control flow, a logical -builder) - the same shape `llvm_jit` already lowers daslang to. So dasSpirv is +builder) - the same shape `llvm_jit` lowers daslang to. So dasSpirv is **dasGlsl's frontend pattern + llvm_jit's SSA-backend pattern, fused into a SPIR-V emitter**. Just as `[jit]` lowers daslang->LLVM->native, `[compute_shader]` lowers daslang->SPIR-V->GPU. What makes it cheap: drivers do all GPU optimization (we emit naive *valid* SPIR-V - so does glslang); SPIR-V binary is a self-delimiting word stream (trivial to emit and disassemble); the backend-agnostic AST tooling (`collect_dependencies`, `collect_used_types`, -`make_visitor`) is reusable verbatim; and dasVulkan already has a GPU-verified `out[i]==i*i` +`make_visitor`) is reusable verbatim; and dasVulkan has a GPU-verified `out[i]==i*i` compute test as a ready-made end-to-end gate. ## 2. Settled decisions @@ -35,11 +35,11 @@ compute test as a ready-made end-to-end gate. PR-protected GaijinEntertainment/daScript, sharing main-tree CI + `daslib/coverage` + `tests/`. SPIR-V is a general daslang capability (also useful for GL4.6/WebGPU), not Vulkan-specific. -2. **Fresh frontend** - its own shader annotations (sec.3); dasGlsl is the *design map*, not - a code dependency. Reuse only the generic AST tooling (none of which lives in dasGlsl). - **Zero edits to the shipped dasGlsl/dasOpenGL.** -3. **Test-per-instruction is a hard requirement**, enforced by the opcode census; LCOV covers - the runtime-reached files beside it (sec.4). +2. **Fresh frontend** - its own shader annotations (sec.3). dasSpirv has no code dependency on + dasGlsl or dasOpenGL; it reuses only the generic AST tooling, none of which lives in + dasGlsl. +3. The opcode census declares the supported opcode set and is checked against the fixtures in + both directions; LCOV covers the runtime-reached files beside it (sec.4). 4. **SPIR-V 1.3 is the default header version; a feature that needs more raises it.** 1.3 gives StorageBuffer storage class + `Block` (not the deprecated 1.0 BufferBlock+Uniform path), lavapipe advertises >= 1.2, and at `<= 1.3` the entry-point interface lists only @@ -52,7 +52,8 @@ compute test as a ready-made end-to-end gate. `modules/dasSpirv` is **pure daslang** (mirrors dasGlsl: a `spirv/` subdir of `.das` files + CMake resolver rows derived from `.das_module`; no `.shared_module`, no C++). dasVulkan -consumes it via `require spirv/...` and feeds the emitted `array<uint>` (SPIR-V words) to `create_shader_module`. +consumes it via `require spirv/...` and feeds the emitted `array<uint>` (SPIR-V words) to +`create_shader_module`. | File | Gen/Hand | Purpose | |---|---|---| @@ -66,7 +67,7 @@ consumes it via `require spirv/...` and feeds the emitted `array<uint>` (SPIR-V | `spirv/spirv_dis.das` | hand | Minimal disassembler + opcode-census helper (self-delimiting walk: word0 = `(wordCount<<16)\|opcode`). Symbolic via `spirv_grammar`'s opcode->name table. | | `generator/gen_spirv_grammar.das` | hand | The mini-generator: reads vendored grammar JSON -> emits `spirv/spirv_grammar.das`. | | `spirv_headers/*.json` | vendored | Pinned `spirv.core.grammar.json` + `extinst.glsl.std.450.grammar.json`; license in `SPIRV_HEADERS.LICENSE`, provenance in `history/dasSpirv/MASTERPLAN_LOG.md`. | -| `CMakeLists.txt` | hand | `ADD_MODULE_DAS_FROM_DESCRIPTOR(spirv spirv)` (rows derived from `.das_module`) + install rule, modeled on `modules/dasGlsl/CMakeLists.txt`. | +| `CMakeLists.txt` | hand | `ADD_MODULE_DAS_FROM_DESCRIPTOR(spirv spirv)` + install rule, modeled on `modules/dasGlsl/CMakeLists.txt`. | **SSA backend (llvm_jit template).** `SpirvEmit` carries `e2v : table<Expression?;uint>` (Expression->result-id), `v2v : table<Variable?;uint>` (Variable->pointer-id), @@ -85,9 +86,14 @@ companion holding the encoded reflection. `generate_spirv` is a standalone `[mac called by **both** `fixup` and the unit tests - so opcode assertions hit the real codegen path without macro plumbing. +**`[spirv_decode]` method form.** The decode callback's SPIR-V signature is a rigid three +parameters. The method form erases the das-level `self` from it, so the decode body still reads +its class members - a separate scale plane, push constants, `@workgroup` staging. + ## 4. Test architecture - "every emitted instruction has a test" -Three behavioral layers + two enforcement gates (all in main-tree `tests/spirv/` except GPU): +The behavioral layers, then the enforcement gates (all in main-tree `tests/spirv/` except the +real-driver layer, which lives in dasVulkan): 1. **Opcode-assertion units.** Each test compiles a tiny shader fixture, calls `generate_spirv`, runs `spirv_dis` to a structured instruction list, and asserts the @@ -99,8 +105,8 @@ Three behavioral layers + two enforcement gates (all in main-tree `tests/spirv/` `spirv-val` resolved through `VULKAN_SDK`. Soft-skip if absent locally, hard-required in CI - the real correctness oracle for structured-CFG and define-before-use bugs. 3. **Real-driver behavioral regression.** A one-call framework in dasVulkan, - `run_compute_spirv(words, n) : array<uint>` over `compute_boost`, runs any emitted blob in - ~2 lines; `compute_image_rgba8` + `assert_pixels_exact` are its image-readback twin. The + `run_compute_spirv(words, n) : array<uint>` over `compute_boost`, runs any emitted blob; + `compute_image_rgba8` + `assert_pixels_exact` are its image-readback twin. The content is procedural, so the expected pixel is CPU-computable and the gate asserts exact pixels rather than inequalities. Primary gate = local real GPU; CI = lavapipe software (GitHub-hosted runners have no GPU, so there is no real-GPU CI lane). @@ -120,9 +126,9 @@ Three behavioral layers + two enforcement gates (all in main-tree `tests/spirv/` ## 5. Cross-backend parity - the kernel-model asymmetry ledger -The mirror rule (REVIEW.md, both emitters) keeps the kernel model symmetric with the MSL -emitter. The ledger of deliberate/pending asymmetries is shared - ONE list, not two - and -lives in `modules/dasMetal/ARCHITECTURE.md` under the same heading. +`modules/REVIEW_SHADER_EMITTERS.md` requires a kernel-model capability added to one emitter to +be added to the other or recorded as an asymmetry. That ledger is shared - one list for both +backends, not one per backend - and lives in `modules/dasMetal/ARCHITECTURE.md` sec.5. ## 6. Verification @@ -134,4 +140,3 @@ lives in `modules/dasMetal/ARCHITECTURE.md` under the same heading. against external `spirv-dis` as ground truth. - **Real-driver gate (dasVulkan):** the integration suite under lavapipe and the local real GPU. -- **Lint/format:** MCP `format_file` on every new `.das`; `lint` clean. Both repos PR-mode. diff --git a/modules/dasSpirv/REVIEW.md b/modules/dasSpirv/REVIEW.md index 7de7c9b95d..3b97084670 100644 --- a/modules/dasSpirv/REVIEW.md +++ b/modules/dasSpirv/REVIEW.md @@ -20,3 +20,6 @@ Architecture doc: `ARCHITECTURE.md`. - **A diff that leaves a fixture emitting an opcode the opcode census does not declare, or leaves the census declaring an opcode no fixture emits, is a defect.** + +- **A diff under `modules/dasSpirv` that edits a file under `modules/dasGlsl` or + `modules/dasOpenGL` is a defect** - dasSpirv copies dasGlsl's design, not its code. diff --git a/modules/dasVulkan/CLAUDE.md b/modules/dasVulkan/CLAUDE.md index 4b55b8dfb4..4735b8c8bc 100644 --- a/modules/dasVulkan/CLAUDE.md +++ b/modules/dasVulkan/CLAUDE.md @@ -87,7 +87,7 @@ Run from the repo root so the cwd-relative shader paths resolve. Test bodies mus Each tutorial is a self-contained unit under `tutorials/<NN_name>/`: the offscreen module + its `[compute_shader]`/`[shader]` blob + a pixel-oracle `[test]` (the CI gate) + a `recording/` driver. -**Every rendering tutorial MUST also ship a windowed mode** at `tutorials/<NN_name>/window/show_<name>.das` - a real GLFW window presenting the live animation (compute->blit->present, or a swapchain render pass). The offscreen `[test]` proves correctness; the windowed driver is how a human actually *sees* it. The windowed driver: +The windowed driver at `tutorials/<NN_name>/window/show_<name>.das`: - calls `glfwInitVulkanLoader(vk_get_instance_proc_addr())` **before** `glfwInit` so GLFW finds the same loader on every platform (see the macOS gotcha below); - lives in `window/` so the tutorials `.das_test` skips it in CI (the lavapipe CI daslang build is `-DDAS_GLFW_DISABLED=ON` - no display, no GLFW). diff --git a/modules/dasVulkan/REVIEW.md b/modules/dasVulkan/REVIEW.md index 6da7269d11..36e90130b3 100644 --- a/modules/dasVulkan/REVIEW.md +++ b/modules/dasVulkan/REVIEW.md @@ -3,7 +3,7 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `CLAUDE.md`. Planned work: `ROADMAP.md`. A tutorial - a `tutorials/<NN_name>/` unit, and any `record_*.das` recording driver wherever the diff puts it - answers to the `tutorials/` subfolder's -checklist. A generator change - any `generator/*.das` emitter or `generator/skip_report.txt` - +checklist. A generator change - any `generator/*.das` file or `generator/skip_report.txt` - answers to the `generator/` subfolder's checklist. **Weakening `REVIEW.das` (beside this file) is a defect:** dropping a check, adding a name to diff --git a/modules/dasVulkan/tutorials/REVIEW.md b/modules/dasVulkan/tutorials/REVIEW.md index 69014576f0..565f8bb376 100644 --- a/modules/dasVulkan/tutorials/REVIEW.md +++ b/modules/dasVulkan/tutorials/REVIEW.md @@ -7,5 +7,5 @@ it, or the shared `recording/` harness - is reviewed with `skills/internal/vulka **A diff that adds a tutorial folder also adds its `<NN_name>/window/show_<name>.das` viewer, in the same change** (`<name>` is the folder name without the numeric prefix) - a GLFW window that -shows the animation as it runs. The offscreen test proves correctness; only the window lets a -human see the result. +presents the tutorial's rendered output live. The offscreen test proves correctness; only the +window lets a human see the result. diff --git a/nano/REVIEW.md b/nano/REVIEW.md index d55d68c8c0..3283549ec5 100644 --- a/nano/REVIEW.md +++ b/nano/REVIEW.md @@ -7,10 +7,10 @@ runtime's tree instead, moving its runtime half into a file of its own.** An edited copy is a fork that drifts silently. -**Never use a name from `daScript/ast/ast.h` beyond `Annotation`, `TypeAnnotation` and -`Module::name` in a source nano compiles - its own under `src/` here and the reused ones under -`src/` (repo root).** Any other name compiles here and means something different than it does -in the full runtime. +**Never use a name from `include/daScript/ast/ast.h` (this folder) beyond `Annotation`, +`TypeAnnotation` and `Module::name` in a source nano compiles - its own under `src/` here and +the reused ones under `src/` (repo root).** Any other name compiles here and means something +different than it does in the full runtime. **Never add a stub to `src/nano_stubs.cpp` that returns a default and continues - implement the behaviour or stop the program instead.** On a target with no debugger attached, a default @@ -18,9 +18,9 @@ return turns a missing feature into a wrong answer. **Weakening the include-clear check in `REVIEW.das` (beside this file) is a defect.** -**Never add a name to `include/daScript/simulate/simulate.h` unless a source nano compiles -unchanged from the full runtime, or generated code, already refers to it.** This header is a -subset, not a copy: a name added for later use is one nobody will know to remove. +**Never add a name to `include/daScript/simulate/simulate.h` (this folder) unless a source +nano compiles unchanged from the full runtime, or generated code, already refers to it.** This +header is a subset, not a copy: a name added for later use is one nobody will know to remove. **Never rename a member of this folder's `Context` or change its type - it keeps the name and the type it has in `include/daScript/simulate/simulate.h` at the repo root.** Generated code @@ -35,7 +35,7 @@ embedders deciding whether their script fits. fail-closed seam (a place where nano stops the program instead of returning a default) also adds an example under `examples/standalone/` and a case in `tests-cpp/big/nano_ctx/test_nano_ctx.cpp`, in the same change.** A tier with no program -linking it stops working with nothing turning red. +linking it stops working and no test fails. **Never put an estimated number in `README.md` - measure it, and name in the table the toolchain and the targets that produced it.** A reader reproduces the number from those two diff --git a/site/REVIEW.md b/site/REVIEW.md index 8fe1134854..b73264438d 100644 --- a/site/REVIEW.md +++ b/site/REVIEW.md @@ -1,19 +1,23 @@ # site (daslang.io) Code Review Checklist **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: -`README.md`. +`README.md`. **A Playwright spec (`*.spec.js`), wherever the diff puts it, answers to the +`tests/playground/` checklist.** **Never show on a page a hand-written shell command, flag, or output line invented for illustration - show only a command that runs verbatim and produces the result the page -shows.** Never show a command whose implementation does not exist yet. +shows.** **A diff that writes a `cmd` field in `files/dasllama/bench_records.json` that is not the exact argv `modules/dasLLAMA/performance/gen_bench_records.das` (or the released bench exe it -spawns) ran is a defect.** Never put a `cmd` + `date` pair on a rendered row - the pair sits -on the run object that produced the rows. +spawns) ran is a defect.** -**A row `files/dasllama.js` renders without its run's receipt line is a defect - render that -line on every row the run produced.** One run's `cmd` + `date` covers several rendered rows. +**A row `files/dasllama.js` renders without its run's `cmd` + `date` line is a defect - render +that line on every row the run produced.** One run's `cmd` + `date` covers several rendered +rows. + +**A `cmd` + `date` pair on a rendered row in `files/dasllama/bench_records.json` is a defect - +put the pair on the run object that produced the rows.** **A diff that leaves `files/dasllama/bench_records.json` differing from what re-running `modules/dasLLAMA/performance/gen_site_records.das` writes is a hand edit and a defect - @@ -21,59 +25,61 @@ change the generator inputs and re-run it instead.** The generator merges every `modules/dasLLAMA/performance/records/<box>.json` and applies `modules/dasLLAMA/performance/records/annotations.json`. -**A diff that changes `files/performance_bench.json` without changing `benchmarks/sql/results.md` -in the same change, or that leaves any cell in the record differing from the same -family-and-lane cell in the `benchmarks/sql/results.md` tables, is a defect.** -`benchmarks/sql/_update_results.das --site-json` writes the record from the same sweep output -that regenerates those tables. - -**A diff that changes `files/performance_engines.json` without changing -`examples/benchmarks/sql/results.md` in the same change, or that leaves any cell in the record -differing from the same family-and-lane cell in the `examples/benchmarks/sql/results.md` -tables, is a defect.** `examples/benchmarks/sql/_update_results.das --site-json` writes the -record from the same sweep output that regenerates those tables. - -**A benchmark number hand-carried into `benchmarks.html`, or a second record file for data a -dasProfile record already holds, is a defect - render the page only from the vendored -dasProfile records, the `files/profile_results_<platform>.json` files pages.yml fetches from -borisbat/dasProfile at deploy time and `files/forge.js`'s front-page cycler reads.** -Provenance - the `das_capture` and `ext_capture` stamps - travels in the dasProfile files -themselves. - -**A partial snippet shown as a code card, a code sample without its "try it on playground" -link, or pseudo-code presented as code is a defect - show every sample as a full program that -compiles and runs with the current toolchain, linked to "try it on playground".** A daslang -sample not written in gen2 is a defect. - -**A diff that adds a test under `tests/playground/` (this folder) needing the daslang runtime, -or that makes an existing test there need it, puts `@wasm` in that test's title, in the same -change.** The per-PR lane stages the site without WASM artifacts and runs the suite with -`--grep-invert '@wasm'` (`playground-e2e.yml`), so an untagged runtime-dependent test fails -every PR. +**A diff that changes `files/performance_bench.json` also changes `benchmarks/sql/results.md`, +in the same change.** `benchmarks/sql/_update_results.das --site-json` writes the record from +the same sweep output that regenerates those tables. + +**A cell in `files/performance_bench.json` differing from the same family-and-lane cell in the +`benchmarks/sql/results.md` tables is a defect.** + +**A diff that changes `files/performance_engines.json` also changes +`examples/benchmarks/sql/results.md` (repo root), in the same change.** +`examples/benchmarks/sql/_update_results.das --site-json` writes the record from the same sweep +output that regenerates those tables. + +**A cell in `files/performance_engines.json` differing from the same family-and-lane cell in +the `examples/benchmarks/sql/results.md` (repo root) tables is a defect.** + +**A benchmark number hand-carried into `benchmarks.html` is a defect - render the page only +from the vendored dasProfile records, the `files/profile_results_<platform>.json` files +`.github/workflows/pages.yml` (repo root) fetches from borisbat/dasProfile at deploy time.** + +**A second record file for data a dasProfile record already holds is a defect - read the +dasProfile record.** Provenance - the `das_capture` and `ext_capture` stamps - travels in the +dasProfile files themselves. + +**A code sample shown as a partial snippet or as pseudo-code is a defect - show every sample as +a full program that compiles and runs with the current toolchain.** + +**A code sample without a "try it on playground" link is a defect - link every sample.** + +**A daslang sample not written in gen2 is a defect.** **A diff that changes the deployed playground sources - `playground/` (this folder), `web/examples/ui/src/`, or `web/examples/ui/samples/` - states a run of the WASM-staged Playwright suite (`tests/playground/`, this folder) in its PR body or commit message, naming -the passes and any failures, in the same change.** The stated run also names the runtime -artifacts it used: built from this change when the change feeds `web/output/daslang_static.*`, -the deployed ones otherwise. The no-WASM lane cannot see a broken runtime path, and every -sample on the page runs through that path. +the passes and any failures, in the same change.** The no-WASM lane cannot see a broken runtime +path, and every sample on the page runs through that path. + +**A stated Playwright run names the runtime artifacts it used: built from this change when the +change feeds `web/output/daslang_static.*`, the deployed ones otherwise.** + +**A diff that puts a measurement number - a rate, a duration, a size, a score some run +produced, never a fixture or run parameter such as an image's dimensions or a token budget - on +a page without rendering it from live data, copying it from a checked-in measurement record, or +marking it a placeholder in a source comment is a defect.** -**A number on a page that reports a measurement - a rate, a duration, a size, a score some run -produced - is a defect when it is not rendered from live data, not copied from a checked-in -measurement record, and not named a placeholder by a source comment.** A placeholder that -could be mistaken for a fact is a defect. A number stating a fixture or a run parameter (an -image's dimensions, a token budget) reports no measurement and is outside this rule. +**A placeholder number a page reader could take for a fact is a defect - mark it as a +placeholder on the page itself.** **A diff that changes one copy of a `dl-*` selector defined in BOTH `files/dasllama-table.css` (the file dasllama.io loads) and `dasllama.html`'s inline `<style>` copy, and not the other, is a defect - give both copies the same body.** -**A diff that adds or moves a `dl-*` selector defines it where its markup lives - the markup a -page renders, counting what `files/dasllama.js` injects: in `dasllama.html` when -`dasllama.html` renders it, in `files/dasllama-table.css` as well when `site-dasllama/`'s -pages (repo root - the dasllama.io site) render it too, and in `files/dasllama-table.css` -alone when only `site-dasllama/`'s pages render it.** +**A diff that adds or moves a `dl-*` selector defines it in `dasllama.html` when +`dasllama.html`'s markup uses it, and in `files/dasllama-table.css` when a page under +`site-dasllama/` (repo root - the dasllama.io site) uses it.** Markup counts what +`files/dasllama.js` injects. **An entry in `_news/*.md` for something not yet true at publish time is a defect - a news entry states a real, shipped event.** diff --git a/site/tests/playground/REVIEW.md b/site/tests/playground/REVIEW.md new file mode 100644 index 0000000000..07980e7c65 --- /dev/null +++ b/site/tests/playground/REVIEW.md @@ -0,0 +1,9 @@ +# Playground Playwright suite Code Review Checklist + +**Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: +`site/README.md`. + +**A diff that adds a test in this folder needing the daslang runtime, or that makes an existing +test here need it, puts `@wasm` in that test's title, in the same change.** The per-PR lane +stages the site without WASM artifacts and runs the suite with `--grep-invert '@wasm'` +(`playground-e2e.yml`), so an untagged runtime-dependent test fails every PR. diff --git a/skills/review_md.md b/skills/review_md.md index f162a8c53c..44505944a7 100644 --- a/skills/review_md.md +++ b/skills/review_md.md @@ -8,8 +8,8 @@ placing the file IS the registration). ## The contract - one file, shared The rules every checklist lives under - the whole shared contract - live ONCE, in -`REVIEW_COMMON.md` at the repo root. A checklist does not restate them; -its opening is this block, verbatim except for the name and the two facts it declares: +`REVIEW_COMMON.md` at the repo root. A checklist does not restate them; its opening is this +block, verbatim except for the module name and the architecture-doc path: ```markdown # <Module> Code Review Checklist @@ -18,65 +18,46 @@ its opening is this block, verbatim except for the name and the two facts it dec `<ARCH-DOC>`. ``` -A checklist that keeps its follow-up ledger somewhere non-obvious appends -`` Planned work: `<ledger>`. `` to the same line. A checklist with routed companions - a -shared concern file, subfolder checklists, a skill reviewing a construct family - carries -their routing lines in or right after the opening, each keyed by KIND ("a `[test]` file, -wherever the diff puts it, answers to the `tests/` subfolder's checklist"), because the -folder walk finds only files named `REVIEW.md` - a companion is reachable solely through its -routing line. An opening that deviates from this block - -or a checklist that restates contract text instead of pointing - is a self-review finding, -fixed like any other. An external repo that adopts `REVIEW.md` vendors a copy of -`REVIEW_COMMON.md` at its root, so the pointer resolves there too. - -`<ARCH-DOC>` is the module's own design document - `ARCHITECTURE.md` (dasLLAMA, dasMetal, -dasSpirv), or the module's `CLAUDE.md` when no separate design doc exists. Name it -concretely; a module whose rules keep wanting a rationale home probably needs to grow one. +A checklist whose folder has a follow-up ledger appends `` Planned work: `<ledger>`. `` to +the same line. A checklist with routed companions - a shared concern file, subfolder +checklists, a skill reviewing a construct family - carries their routing lines in or right +after the opening, each keyed by KIND ("a `[test]` file, wherever the diff puts it, answers to +the `tests/` subfolder's checklist"), because the folder walk finds only files named +`REVIEW.md` - a companion is reachable solely through its routing line. An opening that +deviates from this block - or a checklist that restates contract text instead of pointing - is +a self-review finding, fixed like any other. + +`<ARCH-DOC>` is the module's own design document - its `ARCHITECTURE.md` when it has one, +otherwise its `CLAUDE.md`. Name it concretely; a module with no rationale home needs one +before its rules can cite a reason. ## The executable half - REVIEW.das -The gate contract - what a `REVIEW.das` is, its output protocol, fail-fix, the orphan -rule, and when a checklist rule is deleted in its favor - lives in `REVIEW_COMMON.md`; -this section carries only the authoring craft. A rule is gate material when it is -mechanically decidable from tree state alone: a file exists, a name matches a pattern, -list A equals list B, nothing under X references Y. The support library's API is -documented in `dastest/README.md`. A gate self-reports through `gate_verdict`, so a -hand-run is just `bin/daslang <folder>/REVIEW.das` from the repo root. +The gate contract - what a `REVIEW.das` is, its output protocol, fail-fix, the orphan rule, +and when a checklist rule is deleted in its favor - lives in `REVIEW_COMMON.md`; this section +carries only the authoring craft. A rule is gate material when it is mechanically decidable +from tree state alone: a file exists, a name matches a pattern, list A equals list B, nothing +under X references Y. The support library's API is documented in `dastest/README.md`. A gate +ends by returning `gate_verdict(<name>)` - the support call that prints the accumulated +findings and sets the exit code - so no runner is needed to hand-run one. A worked example is `utils/REVIEW.das`. (repo-only) The walkers that execute gates are repo tooling: the per-diff walk (gates before agent -review) is `skills/internal/make_pr.md` step 0a; the tree-wide walk is -`utils/internal/review-md/all.das`, the `extended_checks` "Run REVIEW.das gates" CI step. (repo-only) - -## Why "no numbers" is load-bearing - -A numbered entry gets cited - from other modules, from commit messages, from other rule -files - and the citation outlives the numbering. Applying another checklist is done by -routing ("a diff touching X applies that checklist too"). Borrowing a criterion whose home -is another folder's document is done by restating the criterion in place and citing -nothing - a restated criterion survives any rewording of its home; a citation points at -whatever sits there today. +review) is the REVIEW audit row of `skills/internal/make_pr.md`; the tree-wide walk is +`utils/internal/review-md/all.das`, the `extended_checks` "Run REVIEW.das gates" CI step. +(repo-only) ## Writing the entries -- Lead each entry with the **bold ban or duty binding the diff** ("never X - do Y instead"; - "a diff that changes X also does Y, in the same change"; "a diff that adds X ... is a - defect"). A sentence describing how the system is belongs in the architecture doc, not - here. No spare words: a tail that restates what the head already binds is cut. -- The list is flat - no section headers, blank line between entries. A grouping that keeps - wanting a header is a separable concern: give it its own file and route to it from the - opening (`modules/REVIEW_SHADER_EMITTERS.md` and `modules/dasLLAMA/REVIEW_GPU.md` are the - pattern) - or its rules are subfolder-scoped and move to that subfolder's own `REVIEW.md`. -- The placement block, when the module carries one, closes the file in the fixed form - `REVIEW_COMMON.md` states: one bold criterion over `` `file` `` - role lines with at most - a bare never-phrase. A smuggled rule moves to the flat list; the `placement-auditor` agent - checks the map against the territory. -- Test rules add the specifics the constitutional branch-test clause can't carry - where - tests live, which suite runs for which change, platform gates. They may sharpen the - clause; they never weaken it, and never restate it. -- The rule for rule-changes is the same as everywhere else in this repo: replace stale text - outright, no dated entries, no "as of" markers. +- A separable concern gets its own file, routed to from the opening - + `modules/REVIEW_SHADER_EMITTERS.md` and `modules/dasLLAMA/REVIEW_GPU.md` are the pattern. +- The `placement-auditor` agent checks a placement block's file list against the folder's + actual contents. +- Test rules add the specifics `REVIEW_COMMON.md`'s "New functionality ships with tests" + clause can't carry - where tests live, which suite runs for which change, platform gates. + They may sharpen it; they never weaken it, and never restate it. +- A rule change replaces stale text outright - no dated entries, no "as of" markers. ## Followability - the slimming dimension @@ -92,11 +73,9 @@ which of them the round edits: one property unites them. State the property; names go stale, the property doesn't. - **Mechanism prose exceeding the rule** - more words on how the code works than on what to check. The mechanism moves to `<ARCH-DOC>`; the rule keeps its one sentence of WHY. -- **Structurally homeless rule** - a rule whose trigger is a change *outside* this folder. - Checklist discovery walks the parent directories of the changed files, so a diff - elsewhere can never surface the rule - it will never fire. Move it to the checklist - of the narrowest folder containing its trigger (creating that subfolder's `REVIEW.md` is - never an objection), or to the skill/CLAUDE.md that governs that code. +- **Structurally homeless rule** - a rule whose trigger is a change outside this folder, so + the folder walk can never surface it. Move it to the narrowest folder containing its + trigger, or to the skill or `CLAUDE.md` that governs that code. - **Overlapping rules** - two entries whose criteria make the reviewer check the same thing twice. Merge into the sharper one. @@ -112,39 +91,44 @@ correctness endlessly, so stop when nothing serious remains - the marginal edit dragon round and buys polish, not verdicts. A round opens a checklist for edit only when it holds a finding of one of these classes: -1. **Verdict-blocking on this diff** - the defect prevented or flipped a verdict while - auditing the change under review; the auditor cites the code site where rule application - failed, or the finding is not accepted. -2. **A defect in a rule the diff itself adds or edits.** -3. **Serious latent** - a factually stale claim (names, lists, APIs no longer matching the - tree), a contradiction between two rules, or a fused rule hiding a second obligation. - The test for serious: acting on it changes what a reviewer CHECKS or CONCLUDES; a change - that only improves the reading is minor. - -Minor findings ride along only in a file already open for the reasons above - never the -sole reason to touch a checklist - and are otherwise dropped, not ledgered: a latent defect -that matters returns as class 1 with a citation. Disposition: FIX NOW when the repair is -forced (one reasonable edit - staleness is the canonical case); ASK the user when the -repair is a semantic choice (which side of a contradiction wins, a rule deleted because a -test enforces it, a scope reassignment). Auditors tag each self-review finding -`blocking | stale | contradiction | structural | formatting`; the tag locates the defect, -and the forced-vs-semantic test on its REPAIR decides the disposition. One checklist-edit -batch per round, one dragon pass over the batch; a FRESH dragon then re-reads cold. -Serious findings it returns open one more batch on the same terms; the round exits when a -fresh cold read returns none. Wording a dragon itself authored -and the round applied verbatim is exempt from the next pass - never run dragon on dragon; -re-judging its own prose is oscillation by construction. A document a round has ruled on is -not re-dragoned in later rounds unless a serious rule change touches it - a ruling stands. +- **Verdict-blocking on this diff** - the defect prevented or flipped a verdict while + auditing the change under review; the auditor cites the code site where rule application + failed, or the finding is not accepted. +- **In-diff rule defect** - a defect in a rule the diff itself adds or edits. +- **Serious latent** - a factually stale claim (names, lists, APIs no longer matching the + tree), a contradiction between two rules, or a fused rule hiding a second obligation. + The test for serious: acting on it changes what a reviewer CHECKS or CONCLUDES; a change + that only improves the reading is minor. + +**Minor findings ride along only in a file already open for one of the classes above** - +never the sole reason to touch a checklist, and otherwise dropped, not ledgered: a latent +defect that matters returns verdict-blocking, with a citation. + +**FIX NOW when the repair is forced** - one reasonable edit exists; staleness is the +canonical case. **ASK the user when the repair is a semantic choice** - which side of a +contradiction wins, a rule deleted because a test enforces it, a scope reassignment. + +**Auditors tag each self-review finding `blocking | stale | contradiction | structural | +formatting`** - the tag locates the defect; the forced-vs-semantic test on the repair +decides the disposition. + +One checklist-edit batch per round, one dragon pass over the batch; a FRESH dragon then +re-reads cold. Serious findings it returns open one more batch on the same terms; the round +exits when a fresh cold read returns none. + +A dragon pass reads only the wording the round itself authored; wording a dragon authored and +the round applied verbatim is not part of the batch - re-judging its own prose oscillates. A +document a round has ruled on gets no further dragon pass in later rounds unless a serious +rule change touches it - a ruling stands. ## Reviewing a REVIEW.md diff -Check the change against `REVIEW_COMMON.md`: is every new entry diff-checkable in -isolation? one paragraph? unnumbered? exception-free? example-free? at most one sentence of -WHY, and only where it makes the criterion decidable? Does a moved rationale actually land -in the architecture doc, or did it just get deleted? - and every other limit -`REVIEW_COMMON.md` states; that file is the law, these are its most-missed questions. -Then apply the self-review rule literally - an entry a reviewer cannot apply as written -is itself a finding - and run the followability classes above over the touched rules. The `dragon` agent runs this audit on -any modified rule document; the `placement-auditor` agent audits a placement block against -its folder. The canonical conforming set is `modules/dasLLAMA/REVIEW.md` and its routed -companions. +Check the change against `REVIEW_COMMON.md`: is every new entry diff-checkable in isolation? +one paragraph? unnumbered? exception-free? example-free? at most one sentence of WHY, and +only where it makes the criterion decidable? Does a moved rationale actually land in the +architecture doc, or did it just get deleted? Those are `REVIEW_COMMON.md`'s most-missed +questions, not its limits - that file is the law. Then apply the self-review rule literally, +and run the followability classes above over the touched rules. The `dragon` agent runs this +audit on any modified rule document; the `placement-auditor` agent audits a placement block +against its folder. The canonical conforming set is `modules/dasLLAMA/REVIEW.md` and its +routed companions. diff --git a/src/builtin/REVIEW.md b/src/builtin/REVIEW.md index 272a9928ea..d5ea84145e 100644 --- a/src/builtin/REVIEW.md +++ b/src/builtin/REVIEW.md @@ -3,18 +3,16 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `ARCHITECTURE.md`. -- **A bind added or changed under this folder uses `addExternInline` or - `addExternInlineEx` when its module is `$` (builtin), `math`, `strings` or `jit`, its - result is a plain value - not a reference, and not a result the callee writes into the - caller's result slot - and it is not an interop bind (`addInterop`); a bind in any other - module uses an `addExtern...` entry point whose name does not contain `Inline`.** This - rule does not decide the flavor of binds of the generic container and equality helpers - declared in `include/daScript/ast/ast_handle.h` (repo root) - the `das_vector_*`, - `das_equ*`, `das_nequ*`, `das_handle_equ*` and `das_handle_nequ*` families. Binds in - `$`, `math`, `strings` and `jit` run inside interpreted inner loops, so each gets its own - node the callee inlines into, while binds elsewhere share one node per signature to keep - binaries small. +- **A plain-value bind added or changed under this folder uses `addExternInline` or + `addExternInlineEx` when its module is an Inline module, and an `addExtern...` entry point + whose name does not contain `Inline` in every other module.** The Inline modules are `$` + (builtin), `math`, `strings` and `jit`. A plain-value bind returns neither a reference nor a + result the callee writes into the caller's result slot, is not an interop bind + (`addInterop`), and is not a generic container or equality helper (`das_vector_*`, + `das_equ*`, `das_nequ*`, `das_handle_equ*`, `das_handle_nequ*`). Inline-module binds run + inside interpreted inner loops, so each gets its own node the callee inlines into, while + binds elsewhere share one node per signature to keep binaries small. -- **Weakening `src/builtin/REVIEW.das`'s bind-flavor scan is a defect** - fix a bind the - gate reports by switching the bind, never by editing the gate and never by dropping a - module from `review_nttp.das`'s `require` list, which sets the modules the scan covers. +- **Weakening `review_nttp.das`'s bind-flavor scan, which `REVIEW.das` runs, is a defect** - + fix a bind the scan reports by switching the bind, and never drop a module from + `review_nttp.das`'s `require` list, which sets the modules the scan covers. diff --git a/tests/metal/REVIEW.md b/tests/metal/REVIEW.md index 28cd5a7dd9..a23d3e3326 100644 --- a/tests/metal/REVIEW.md +++ b/tests/metal/REVIEW.md @@ -8,3 +8,6 @@ test, not of the kernel - build a fresh instance for every thread id.** A CPU-or produces its expected values by running the kernel method on the CPU, in a driver loop that sets `gl_GlobalInvocationID` once per thread id. Kernel members are per-thread state, so a reused instance carries the previous thread's values into the next. + +**A file in this folder that creates any Metal object asserts `metal_live_object_count() == 0` +before it returns.** diff --git a/utils/REVIEW.md b/utils/REVIEW.md index c74f9d013c..d9db23509f 100644 --- a/utils/REVIEW.md +++ b/utils/REVIEW.md @@ -4,15 +4,18 @@ doc: `CLAUDE.md` (repo root). **A file that belongs to a tool is reviewed with that tool's own `REVIEW.md`, where one exists, -as well as with this checklist - the tool it belongs to decides, not the directory it is in.** +as well as with this checklist - not the checklist of the directory the file sits in.** **Weakening `REVIEW.das` (beside this file) is a defect.** What the gate checks is read from the script itself, and each check's finding text states its rule. -**A tool removed from `DAS_UTILS_SHIPPED_EXES` (`CMakeLists.txt`, beside this file) records -the decision in that tool's own `REVIEW.md`, wherever that tool's directory is, in the same -change** - the gate cannot see a removed entry. A tool deleted outright records the decision -beside the list in `CMakeLists.txt` instead. +**A diff that drops a tool from `DAS_UTILS_SHIPPED_EXES` (`CMakeLists.txt`, beside this file) +while keeping that tool's directory records the decision to stop shipping it in that tool's +own `REVIEW.md`, wherever that directory is, in the same change** - the gate cannot see a +removed entry. + +**A diff that deletes a tool outright records the decision beside `DAS_UTILS_SHIPPED_EXES` in +`CMakeLists.txt` (beside this file), in the same change.** **A new or changed test for a `utils/` tool whose load-bearing assertions a CI lane can run - the assertions that prove the change, not a skip-path assertion - ships with a CI row that @@ -20,8 +23,8 @@ executes those assertions, wherever the diff puts the test, added in the same ch row already covers it.** A row that only compile-checks the test (`dastest --compile-only`) does not execute them. A test whose assertions no row executes never runs again. -**A test for a `utils/` tool whose load-bearing assertions no CI lane can run - they need -something no CI machine has - ships with a CI row that compile-checks it.** +**A new or changed test for a `utils/` tool whose load-bearing assertions no CI lane can run +ships with a CI row that compile-checks it.** **A new or changed test for a `utils/` tool that gets a compile-only row records its executed run in the PR description**: the machine the assertions ran on, what that machine had that CI diff --git a/utils/das-fmt/REVIEW.md b/utils/das-fmt/REVIEW.md index bbe95f4637..419f07abf9 100644 --- a/utils/das-fmt/REVIEW.md +++ b/utils/das-fmt/REVIEW.md @@ -3,6 +3,6 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `README.md`. -**A diff that drops `das-fmt` from `DAS_UTILS` or `DAS_UTILS_SHIPPED_EXES` in -`utils/CMakeLists.txt` (repo root) is a defect** - das-fmt ships as a prebuilt exe, and dropping -either entry breaks the release bundle. +**A diff that removes `das-fmt` from `DAS_UTILS_SHIPPED_EXES` in `utils/CMakeLists.txt` is a +defect.** The removal drops the prebuilt das-fmt exe from the release bundle, and the gate +cannot see a removed entry. diff --git a/utils/dascov/REVIEW.md b/utils/dascov/REVIEW.md index f32a236bd1..83896d3a59 100644 --- a/utils/dascov/REVIEW.md +++ b/utils/dascov/REVIEW.md @@ -3,6 +3,6 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `README.md`. -**A diff that removes `dascov` from `DAS_UTILS` or `DAS_UTILS_SHIPPED_EXES` in -`utils/CMakeLists.txt` is a defect.** Either entry missing drops the prebuilt dascov exe from -the release bundle. +**A diff that removes `dascov` from `DAS_UTILS_SHIPPED_EXES` in `utils/CMakeLists.txt` is a +defect.** The removal drops the prebuilt dascov exe from the release bundle, and the gate +cannot see a removed entry. diff --git a/utils/dasllama-server/REVIEW.md b/utils/dasllama-server/REVIEW.md index 4136e9cbd8..969de12544 100644 --- a/utils/dasllama-server/REVIEW.md +++ b/utils/dasllama-server/REVIEW.md @@ -4,8 +4,7 @@ doc: `README.md`. **A diff to a file that requires a `dasllama/*` module applies `modules/dasLLAMA/REVIEW.md` -(repo root) too** - the server reaches only public dasLLAMA facade verbs, and a diff here can -break that facade. +(repo root) too.** **Never write or edit a fixture under `tests/fixtures/` by hand - capture the bytes of a real server run and normalize the machine-local paths instead.** `tests/fixtures/README.md` gives @@ -22,7 +21,8 @@ the `README.md` route row, the endpoint table in `doc/source/reference/utils/dasllama_server.rst` (repo root), and `control.html` where the page uses it. -**A diff that adds, removes, or renames a key in a response body re-captures its fixtures -under `tests/fixtures/` and mentions the key in its `README.md` route row, in the same +**A diff that adds, removes, or renames a key in a response body re-captures that route's +fixtures under `tests/fixtures/` and updates that route's row in `README.md` (beside this +file) - a new or renamed key is listed there, a removed key is struck from it - in the same change** - the fixtures are the recorded response shape, and the row is where a consumer learns the key exists. diff --git a/utils/daspkg/REVIEW.md b/utils/daspkg/REVIEW.md index 859a3fa2d2..b9f04facd1 100644 --- a/utils/daspkg/REVIEW.md +++ b/utils/daspkg/REVIEW.md @@ -9,35 +9,38 @@ network, interpreted. **A diff whose changed lines sit inside a function that runs a git command - in `commands.das`, `index.das`, or `utils.das` - also runs the integration suite, in the same -change** - a behavior-preserving refactor still counts. The integration suite is `bin/daslang dastest/dastest.das -- --test +change.** The integration suite is `bin/daslang dastest/dastest.das -- --test utils/daspkg/test_daspkg_git.das`, and it needs network (the `borisbat/daspkg-test-*` fixture repos). -**A change to `cmd_release`, `cmd_release_wasm`, or a `release_*` helper is verified on macOS, -or the review says it was not.** The release layout differs per platform (`.app` bundle vs flat -directory), and no per-PR CI runs the daspkg test suites. +**A change to `cmd_release`, `cmd_release_wasm`, or a `release_*` helper states in the review +whether the release was run on macOS.** The release layout differs per platform (`.app` bundle +vs flat directory). -**A diff that adds a command or a flag also adds its test, its `print_usage` line, and its -`README.md` table row, in the same change.** +**A diff that adds a command or a flag also adds its `print_usage` line and its `README.md` +table row, in the same change.** -**A diff that drops `daspkg` from `DAS_UTILS` or `DAS_UTILS_SHIPPED_EXES` in -`utils/CMakeLists.txt` (repo root) is a defect** - daspkg ships as a prebuilt exe, and dropping -either entry breaks the release bundle. +**A diff that removes `daspkg` from `DAS_UTILS_SHIPPED_EXES` in `utils/CMakeLists.txt` is a +defect.** The removal drops the prebuilt daspkg exe from the release bundle, and the gate +cannot see a removed entry. **A `cmd_release` bundle that ships an exe without a tune sidecar beside it is a defect** - the tune sidecar is the `<bundle>.tune.json` file of measured kernel choices the exe reads at run time. -**A diff that lets a release path other than `--quick` inherit a prior sidecar, or lets -`--quick` accept an incomplete one, is a defect** - incomplete means missing any scope key - a -`tune_scopes` entry the exe's deps JSON reports - the same completeness the release build -itself checks. +**A diff that lets a release path other than `--quick` reuse a sidecar from an earlier run is a +defect.** -**A release path that overwrites or deletes a `release_include_if_missing` file, on any -platform, is a defect.** +**A diff that lets `--quick` accept an incomplete sidecar is a defect** - incomplete means +missing a scope key, that is, an entry of the `tune_scopes` list in the deps JSON that +`daslang -exe --list-shared-modules` writes. -**A release path that finishes a bundle without writing `.daspkg_release.manifest`, on any -platform, is a defect.** +**A release path that overwrites or deletes a `release_include_if_missing` file is a defect** - +one the package's `release()` declares that way: a starter file deployed once, then owned by +the user. + +**A release path that finishes a bundle without writing `.daspkg_release.manifest` is a +defect.** **A test in `test_daspkg.das` that reaches the network is a defect** - network coverage belongs in `test_daspkg_git.das`. diff --git a/utils/detect-dupe/REVIEW.md b/utils/detect-dupe/REVIEW.md index 16310d2e7f..6e08104ff4 100644 --- a/utils/detect-dupe/REVIEW.md +++ b/utils/detect-dupe/REVIEW.md @@ -3,6 +3,6 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `README.md`. -**A diff removing detect-dupe from `DAS_UTILS_SHIPPED_EXES` records the removal decision in -this file.** detect-dupe ships as a prebuilt exe, and `utils/REVIEW.das` machine-checks the -membership itself. +**A diff removing detect-dupe from `DAS_UTILS_SHIPPED_EXES` in `utils/CMakeLists.txt` records +the removal decision in this file.** detect-dupe ships as a prebuilt exe, and no gate can see +an entry that is gone. diff --git a/utils/internal/dasllama-ladder/REVIEW.md b/utils/internal/dasllama-ladder/REVIEW.md index 9f132ddef7..b1ecb308f7 100644 --- a/utils/internal/dasllama-ladder/REVIEW.md +++ b/utils/internal/dasllama-ladder/REVIEW.md @@ -3,22 +3,19 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `README.md`. Planned work: `plans/dasllama_io_site.md`. -**A diff that adds or changes a route, a store operation, or a config or limit behavior also -adds a dastest test for it in this directory, in the same change.** `main.das` and `admin.das` -stay argv/dispatch glue over tested modules, so they need no test of their own. - **Never put a `[test]` file for this folder under the repo-root `tests/` tree, and never register one in any `CMakeLists.txt` - a `[test]` file lives in this directory and requires its siblings by bare name.** -**Never reach the service from a test except through the local `with_ladder_server` harness -(`test_ladder_server.das`) on this directory's reserved test port 19015, and never start a -server in a store test - a store test calls `ladder_store` directly.** A store behavior proven -only through HTTP, or an HTTP behavior proven only against the store, is a defect. +**A test reaches the service only through the local `with_ladder_server` harness +(`test_ladder_server.das`) on this directory's reserved test port 19015, and a store test +starts no server - it calls `ladder_store` directly.** + +**A store behavior proven only through HTTP, or an HTTP behavior proven only against the store, +is a defect.** -**Never let a test write into the repo tree or anywhere outside `temp_directory` - root every -path the test creates under `temp_directory` and delete it before the test ends; a store test -opens `:memory:`, never a file.** +**Never let a test write outside `temp_directory` - root every path the test creates there and +delete it before the test ends; a store test opens `:memory:`, never a file.** **Operator routes (`/admin/*`, `/shutdown`) never appear in `caddy.snippet`, or in any Caddyfile route that reaches this service, a catch-all included - operators reach them on the @@ -26,29 +23,27 @@ service port itself, over the ssh tunnel.** Caddy proxies from the same box, so `is_loopback_peer` sees `127.0.0.1` for every proxied request, which leaves the Caddyfile as the only real boundary. -**Never drop one of the three checks in `is_operator_caller` - a loopback transport peer, a -loopback `Host` authority, and same-origin-or-headerless (the request carries no `Origin` -header, or its `Origin` names the same authority as its `Host`) - and never gate an operator -route on the peer alone: every operator route gates through `is_operator_caller`.** The `Host` -and `Origin` checks together are the CSRF guard for the `/admin/` page over an ssh tunnel - -same-origin refuses a cross-origin fetch, and the loopback `Host` refuses DNS rebinding. +**Never drop a check from `is_operator_caller` - a loopback transport peer, a loopback `Host` +authority, and same-origin-or-headerless (the request carries no `Origin` header, or its +`Origin` names the same authority as its `Host`).** The `Host` and `Origin` checks together are +the CSRF guard for the `/admin/` page over an ssh tunnel - same-origin refuses a cross-origin +fetch, and the loopback `Host` refuses DNS rebinding. + +**Every operator route gates through `is_operator_caller`.** **A diff that adds a route a public caller needs also adds it to `caddy.snippet`, in the same change as its handler; never edit `caddy.snippet` to match the deployed Caddyfile - edit the deployed Caddyfile to match `caddy.snippet` instead.** **On a route that serves board data, mutates the store, or refuses a caller, a response path -that does not log one `ladder.req` line through `log_request` is a defect - refusals log too.** -The bare liveness probe `GET /healthz` is none of these and logs nothing. +that does not log one `ladder.req` line through `log_request` is a defect.** **A handler that reads a request body without first checking `body_is_byte_faithful` is a defect.** -**A store-backed handler that does anything beyond four things - check transport shape, gate -the request (operator gate, submit-open, attempt-limit), make one store call, format the -response - is a defect.** SQL, hashing, and store policy go in `ladder_store.das`, which never -requires `dashv`. The `/admin/` page and `/healthz` reach no store and are not store-backed -handlers. +**A handler that calls `ladder_store` and does anything beyond these - check transport shape, +gate the request (operator gate, submit-open, attempt-limit), make one store call, format the +response - is a defect.** **A diff that defaults `submit_open` to true in `LadderArgs` or `LadderPolicy`, lets `/api/submit/sidecar` or `/api/submit/records` answer anything but 403 while `submit_open` is @@ -56,32 +51,29 @@ false, or adds an opener reachable from a non-loopback path, is a defect** - onl `/admin/submit` route flips the gate. **In `caddy.snippet` every proxied route other than the `/api/submit/records` and -`/api/submit/sidecar` matcher carries the small read cap; a large-body allowance on any other -route, or a read cap on either submit matcher, is a defect.** +`/api/submit/sidecar` matcher carries the `request_body { max_size 64KB }` cap, and that +matcher carries `max_size 8MB` and no smaller cap.** -**Never require the dasLLAMA engine, dasLLVM, or any model machinery from a file in this -directory - the one dasLLAMA module allowed is the engine-free -`dasllama/dasllama_exchange_schema` (a public entry of that module's facade lint), required -from `ladder_store.das` and nowhere else** (`README.md` sec.3). Its `dasllama_lint` carrier is -a compile-time macro, not engine code. +**A file in this directory requires no dasLLAMA, dasLLVM, or model machinery other than the +engine-free `dasllama/dasllama_exchange_schema`, and only `ladder_store.das` requires that** +(`README.md` sec.3). **A write path that stores a document without validating it first is a defect: every sidecar - community or planted - passes `validate_sidecar_submission`, every community record store passes `validate_record_submission`, and `import_official_store` passes `validate_record_store` -(shape only - official record history predates the release counter).** A sidecar without the -version stamp can never be served, so storing one is always an error. +(shape only - official records predate the `DASLLAMA_VERSION` stamp).** -**Never write `Source` or `Verified` outside store code, and never let a value from a public -(proxied) request reach either column - the loopback operator surface sets `Verified` through -store calls, never through SQL in a handler.** +**Never write `Source` or `Verified` outside `ladder_store.das`, and never let a value from a +public (proxied) request reach either column - the loopback operator surface sets `Verified` +through a `ladder_store` call.** **Never validate, hash, or store sidecar text - community or planted - that has not been through `exchange_strip_private`, and never store a community record store that has not been through `redact_record_paths`.** Hashing the cleaned text on both sidecar paths is what makes -re-plant promote instead of duplicate. `import_official_store` stores its document as-is. +re-plant promote instead of duplicate. -**Never update a document after its insert - derived columns may be recomputed, and a document -leaves only by deleting its submission.** +**Never change a submission's stored `Doc` after its insert - a document leaves only by +deleting its submission.** **Never edit a shipped `[sql_migration]` body - a schema change adds a new, higher version in the same stream.** @@ -98,9 +90,11 @@ a shell. **A diff that changes where the service or watchdog writes at runtime - log path, working directory, database location - also adds a matching `ReadWritePaths` entry to the systemd unit -`provision` writes, in the same change.** Relaxing that unit's sandbox is a defect: -`ProtectSystem=strict`, an emptied `CapabilityBoundingSet`, and `ReadWritePaths` no wider than -the data dir and the release tree. +`provision` writes, in the same change.** + +**The systemd unit `provision` writes keeps `ProtectSystem=strict`, an empty +`CapabilityBoundingSet`, and `ReadWritePaths` no wider than the data dir and the release tree; +a diff that widens any of the three is a defect.** **Placement - one file, one line: a diff keeps each file inside its line, and a new file adds its line here, with its tests, in the same change.** diff --git a/utils/internal/dasweb-buildd/REVIEW.md b/utils/internal/dasweb-buildd/REVIEW.md index 57784b20f5..a40dead1aa 100644 --- a/utils/internal/dasweb-buildd/REVIEW.md +++ b/utils/internal/dasweb-buildd/REVIEW.md @@ -1,28 +1,22 @@ # dasweb-buildd Code Review Checklist **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: -`README.md`. Planned work: `plans/dasweb_wasm_pipeline.md`. - -**A diff that adds or changes behavior in `buildd_config.das`, `buildd_core.das`, or -`buildd_client.das` ships a dastest test in this directory, in the same change.** A diff that -puts behavior beyond glue and orchestration over those modules into `main.das` or -`buildd_service.das` is a defect. +`README.md`. Planned work: `plans/dasweb_wasm_pipeline.md` (repo root). **Never put a `[test]` file under the global `tests/` tree, and never register one in a `CMakeLists.txt` - a `[test]` file for this directory lives here and requires its siblings by bare name.** **A test that reaches any host or port other than the stub playground in -`test_buildd_client.das` on this directory's reserved test port 19014 is a defect, and so is a -test of `buildd_core.das` that reaches the network at all.** The playground's real endpoints are -proven in its own directory. +`test_buildd_client.das`, on this directory's reserved test port 19014, is a defect.** + +**A test of `buildd_core.das` that reaches the network at all is a defect.** -**A diff that adds or changes a test that touches the filesystem roots that test's paths in -`temp_directory` and deletes what the test creates.** +**A test that touches the filesystem creates its paths under the directory `temp_directory()` +returns and deletes what it creates before it ends.** -**Never let the health server bind an address other than loopback - it calls -`set_bind_host("127.0.0.1")` between `init` and `start`.** A diff that removes that call, moves -it past `start`, or makes it conditional is a defect. +**Never let the health server bind an address other than loopback - it must call +`set_bind_host("127.0.0.1")` unconditionally, between `init` and `start`.** **Never let `POST /shutdown` act on a request without checking that the transport peer address is loopback.** No header can carry that proof. @@ -34,14 +28,21 @@ banner reports set or unset and the provenance instead.** source or a build output alike - without passing that name through `is_valid_source_filename`, `is_valid_asset_path`, or `validate_asset_paths` (`buildd_core.das`) first is a defect.** -**A diff that lets a build publish a file its mode does not declare by name - through a suffix -filter, a glob, or any rule that widens the build's own output set - is a defect.** The build -runs the user's own compile-time code and can write anything into the output directory. +**A diff that lets a build publish a file the job's build mode - the build kind, which lists +its output files by name - does not declare by name, through a suffix filter, a glob, or any +rule that widens the build's own output set, is a defect.** The build runs the user's own +compile-time code and can write anything into the output directory. + +**A diff that adds a mount to `run_build.sh` also gives that mount's reason in `README.md`'s +The sandbox section, in the same change.** + +**A diff that gives a build a host path which is not an explicit mount in `run_build.sh` is a +defect.** -**A diff that adds a mount to `run_build.sh` without a stated reason, that gives a build a host -path which is not an explicit mount in that script, or that adds a way to run a build outside -the container sandbox defined by `Containerfile` and `run_build.sh`, is a defect.** Mounting -anything that holds a secret, a key, or another service's data is a defect. +**A diff that adds a way to run a build outside the container sandbox defined by +`Containerfile` and `run_build.sh` is a defect.** + +**Never mount anything that holds a secret, a key, or another service's data.** **A change justified by `no_unsafe`, `no_init`, or any compile-time policy flag standing in for the sandbox's isolation is a defect.** Compile-time code reads files with no `unsafe` at all. @@ -54,14 +55,17 @@ the sandbox's isolation is a defect.** Compile-time code reads files with no `un **A diff that leaves a claimed job able to end without structured log lines for its start and its outcome (done or failed, duration, exit code) is a defect.** -**A diff that leaves a config key out of the startup log, drops that key's provenance, or moves -that log after the first poll is a defect.** +**A diff that leaves a config key out of the startup banner, drops that key's provenance, or +moves that banner after the first poll is a defect.** + +**Never put state that outlives one `update` call anywhere but a module-global.** -**Never put lifecycle-owned state anywhere but a module-global, and never hold a collectable -value - one the GC can free - in a `main`-loop local across a call to `maybe_collect_gc()`.** +**Never hold a collectable value - one the GC can free - in a `main`-loop local across a call +to `maybe_collect_gc()`.** **A diff that adds a code path where a claimed job ends without an upload - success or failure - -is a defect.** Leaving the claim for the stale-requeue sweep is not a resolution. +is a defect.** Leaving the claim for the stale-requeue sweep - the server-side sweep that +re-queues jobs from builders that died - is not a resolution. **Never let a job resolve without removing its per-job scratch directory.** @@ -79,8 +83,9 @@ the same change.** **A diff that changes the wasm-archive step's build command or archive list in `roll_toolchain.sh`, or `modules/dasImgui/.das_package`, also changes the other to match, in -the same change - the two differ only in `liblibDasModuleClipboard.a`, which the wasm-archive -list leaves out.** +the same change.** + +**The wasm-archive list in `roll_toolchain.sh` never carries `liblibDasModuleClipboard.a`.** **Never ship a file an operator edits on the box with plain `release_include` - use `release_include_if_missing`, so an upgrade keeps the operator's edits.** diff --git a/utils/internal/dasweb-verify/browser/REVIEW.md b/utils/internal/dasweb-verify/browser/REVIEW.md index ae5cc72686..3282865a58 100644 --- a/utils/internal/dasweb-verify/browser/REVIEW.md +++ b/utils/internal/dasweb-verify/browser/REVIEW.md @@ -5,15 +5,15 @@ doc: `../README.md`. **A diff that leaves a pure helper in this folder - data in, data out, no playwright, no DOM, no network - without a `node:test` case in `protocol.test.mjs` is a defect, wherever the diff -puts the helper.** `runner.mjs` and `probe.mjs` hold only browser- and playwright-bound code, -which the nightly CI run proves. +puts the helper.** -**Never put a budget or a pattern in a sample source - put it in `expectations.json` -instead.** A `// verify:` line in a sample changes the sample's content hash and throws away -the build-cache entry the nightly depends on. +**Never make the verifier take a time budget or an expected-output pattern from a sample +source - read both from `expectations.json`.** A `// verify:` line in a sample changes the +sample's content hash and throws away the build-cache entry the nightly depends on. -**Weakening the fail-closed checks in `protocol.test.mjs` is a defect** - an unknown sample -stays a FAIL, manifest drift stays a WARN, and the manifest-coverage case keeps reading the +**Weakening the fail-closed checks in `protocol.test.mjs` is a defect** - a deployed sample +with no `expectations.json` row stays a FAIL, a name in the deployed manifest but not the repo +manifest (or the reverse) stays a WARN, and the manifest-coverage case keeps reading the shipped manifest `web/examples/ui/samples/data.json` (repo root). **Never make the verifier change a build's hash to force a rebuild - report the cached result @@ -24,8 +24,8 @@ a fresh hash only hides the bug. returns - through the frame the playground embeds it in; open the artifact URL as a top-level page instead.** That frame is cross-origin by design, so nothing inside it can be read. -**Never make `probe.mjs` poll `glGetError` on a page that has its own GL-error watcher - poll -only where the page has none (artifact pages, `pollGl: true`), never on the playground, where +**`probe.mjs` polls `glGetError` only on pages with no GL-error watcher of their own - artifact +pages, installed with `pollGl: true` - and never on the playground, where `site/playground/run-frame.html` (repo root) already polls.** `getError` clears the flag, so only one poller per context can see an error. diff --git a/utils/lsp/REVIEW.md b/utils/lsp/REVIEW.md index 965f9d0bf5..96773db62a 100644 --- a/utils/lsp/REVIEW.md +++ b/utils/lsp/REVIEW.md @@ -4,5 +4,5 @@ `README.md`. **Never add `lsp` to `DAS_UTILS` or `DAS_UTILS_SHIPPED_EXES` in `utils/CMakeLists.txt` - clients -start the server through `lsp_supervisor.py` instead.** Development always starts it through that -python script, so an exe form would ship without anyone having used it. +start the server through `lsp_supervisor.py` instead.** An exe form would ship without anyone +having used it. diff --git a/web/examples/ui/REVIEW.md b/web/examples/ui/REVIEW.md index a2ff4b9bd0..70dfd395bb 100644 --- a/web/examples/ui/REVIEW.md +++ b/web/examples/ui/REVIEW.md @@ -5,7 +5,11 @@ **Every rule of `site/REVIEW.md` binds a change under `src/` and `samples/` as if the files lived under `site/playground/`.** The deploy copies them into the site (`pages.yml`), so -`site/playground/` never shows the change and only this line makes that checklist apply. +`site/playground/` never shows the change. + +**Never write a `// verify:` line into a sample source - put the budget or the expected pattern +in `utils/internal/dasweb-verify/browser/expectations.json` instead.** A sample-source change +voids that sample's nightly build-cache entry. **A file of the boulder-dash sample - its `.das` sources, its dastest suites, its `ARCHITECTURE.md` - answers to `samples/examples/boulder-dash/REVIEW.md` wherever the From ad1b725d509a83259db2b10c0f71e97740b11fa5 Mon Sep 17 00:00:00 2001 From: Boris Batkin <bbatkin@gmail.com> Date: Thu, 27 Aug 2026 05:10:37 -0700 Subject: [PATCH 11/15] no stray .md beside shipped code - LINT025 + install gates Implementation folders hold only REVIEW*.md, ARCHITECTURE*.md, LAWS.md (.lint_config [docs] rule_docs_only, armed for daslib and all module implementation folders; utils/lint folder pass, tested with negative controls). Neither those documents nor any ledger installs: the root manifest gate and the bundle smoke test ban the rule/record classes, with REVIEW_COMMON.md and ARCHITECTURE_COMMON.md at the install root as the two vendoring exceptions - ARCHITECTURE_COMMON.md now ships beside its sibling. daslib offenders resolved: the comment-sweep ledger moved to plans/, the linq_fold masterplan distilled into ARCHITECTURE.md sec.37 (claims verified against shipped code) with its phase log archived to history/. The stale comment-clean test now pins the harvest-era policy. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014fKJGuUL8tP58Y6NvoVeSn --- CLAUDE.md | 6 + CMakeLists.txt | 28 ++-- LAWS.md | 1 + ci/smoke_test_bundle.sh | 19 ++- daslib/.lint_config | 5 + daslib/ARCHITECTURE.md | 79 ++++++++++ daslib/REVIEW.md | 2 +- daslib/linq_fold_array.das | 2 +- daslib/linq_fold_common.das | 2 +- daslib/linq_fold_decs.das | 2 +- daslib/linq_fold_json.das | 2 +- daslib/linq_fold_sql.das | 2 +- daslib/linq_fold_table.das | 2 +- daslib/lint_config.das | 36 ++++- history/README.md | 5 +- {daslib => history/linq_fold}/linq_fold.md | 54 ++----- modules/dasClipboard/daslib/.lint_config | 4 + modules/dasImgui/daslib/.lint_config | 4 + modules/dasLLAMA/dasllama/.lint_config | 4 + modules/dasLLVM/daslib/.lint_config | 4 + modules/dasPUGIXML/daslib/.lint_config | 4 + modules/dasPUGIXML/daslib/linq_fold_xml.das | 4 +- modules/dasSMT/daslib/.lint_config | 4 + modules/dasSQLITE/daslib/.lint_config | 4 + modules/dasTerminal/daslib/.lint_config | 4 + modules/dasTreeSitter/daslib/.lint_config | 4 + modules/dasVulkan/daslib/.lint_config | 4 + .../daslib_comment_sweep.md | 0 .../parity_xml_order_distinct_group.das | 2 +- tests/lint/test_lint_config.das | 6 +- utils/lint/README.md | 7 + utils/lint/main.das | 109 +++++++++++-- utils/lint/tests/lint025_rule_docs_only.das | 146 ++++++++++++++++++ 33 files changed, 470 insertions(+), 91 deletions(-) rename {daslib => history/linq_fold}/linq_fold.md (93%) create mode 100644 modules/dasClipboard/daslib/.lint_config create mode 100644 modules/dasImgui/daslib/.lint_config create mode 100644 modules/dasLLAMA/dasllama/.lint_config create mode 100644 modules/dasLLVM/daslib/.lint_config create mode 100644 modules/dasPUGIXML/daslib/.lint_config create mode 100644 modules/dasSMT/daslib/.lint_config create mode 100644 modules/dasSQLITE/daslib/.lint_config create mode 100644 modules/dasTerminal/daslib/.lint_config create mode 100644 modules/dasTreeSitter/daslib/.lint_config create mode 100644 modules/dasVulkan/daslib/.lint_config rename daslib/followup_comment_sweep.md => plans/daslib_comment_sweep.md (100%) create mode 100644 utils/lint/tests/lint025_rule_docs_only.das diff --git a/CLAUDE.md b/CLAUDE.md index 75f7cd4689..a969d2b375 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -30,6 +30,12 @@ Three document types split one grammar, each with a repo-root contract: exception ledger says what is already ruled. Facts formerly carried by comments live here. - **`/history`** - past tense: what happened. Nothing current cites it. +An implementation folder - `daslib/`, a module's `daslib/` or engine folder - holds no `.md` +besides `REVIEW*.md`, `ARCHITECTURE.md`, and `LAWS.md` (lint-enforced: `.lint_config` +`rule_docs_only`). Ledgers and plans live at the module root or in `plans/` (repo root). None +of these documents installs - the SDK bundle gate bans them; `REVIEW_COMMON.md` and +`ARCHITECTURE_COMMON.md` at the install root are the two vendoring exceptions. + The mood test routes misplaced text: must/never binding a diff -> REVIEW.md; present-tense fact -> ARCHITECTURE.md; dated or past-tense -> /history. diff --git a/CMakeLists.txt b/CMakeLists.txt index 709f32e11f..9349542488 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -1922,8 +1922,10 @@ install(DIRECTORY ${PROJECT_SOURCE_DIR}/skills/daslang/ REGEX "/LAWS\\.md$" EXCLUDE PATTERN "*.md" ) -# The review constitution ships with the review skills that route to it. +# The review constitution ships with the review skills that route to it, and the +# architecture-doc contract it names ships beside it so the pointer resolves. install(FILES ${PROJECT_SOURCE_DIR}/REVIEW_COMMON.md DESTINATION ${DAS_INSTALL_DOCDIR}) +install(FILES ${PROJECT_SOURCE_DIR}/ARCHITECTURE_COMMON.md DESTINATION ${DAS_INSTALL_DOCDIR}) install(FILES ${PROJECT_SOURCE_DIR}/LICENSE DESTINATION ${DAS_INSTALL_DOCDIR}) install(FILES ${PROJECT_SOURCE_DIR}/3rdparty/uriparser/COPYING DESTINATION ${DAS_INSTALL_DOCDIR} RENAME URIPARSER.LICENSE) install(FILES ${PROJECT_SOURCE_DIR}/include/dag_noise/LICENSE DESTINATION ${DAS_INSTALL_DOCDIR} RENAME DAG_NOISE.LICENSE) @@ -2301,28 +2303,34 @@ if(NOT ${DAS_TREE_SITTER_DISABLED}) ) endif() -# REVIEW.md / REVIEW.das are repo-internal review machinery and never install; -# the one exception is REVIEW_COMMON.md at the install root (adopting repos vendor -# from it). This must stay the LAST install rule in the root CMakeLists so the -# manifest it checks is complete. +# Repo-internal rule-and-record documents never install: REVIEW.md / REVIEW.das, +# ARCHITECTURE.md, LAWS.md, MASTERPLAN logs, followup/perf/profile ledgers. The two +# exceptions are REVIEW_COMMON.md and ARCHITECTURE_COMMON.md at the install root +# (adopting repos vendor from them). This must stay the LAST install rule in the +# root CMakeLists so the manifest it checks is complete. install(CODE "set(_das_docdir \"${DAS_INSTALL_DOCDIR}\")") install(CODE [[ set(_review_leaks "") if(IS_ABSOLUTE "${_das_docdir}") - set(_root_common "$ENV{DESTDIR}${_das_docdir}/REVIEW_COMMON.md") + set(_root_docdir "$ENV{DESTDIR}${_das_docdir}") else() - set(_root_common "$ENV{DESTDIR}${CMAKE_INSTALL_PREFIX}/${_das_docdir}/REVIEW_COMMON.md") + set(_root_docdir "$ENV{DESTDIR}${CMAKE_INSTALL_PREFIX}/${_das_docdir}") endif() - get_filename_component(_root_common "${_root_common}" ABSOLUTE) + get_filename_component(_root_common "${_root_docdir}/REVIEW_COMMON.md" ABSOLUTE) + get_filename_component(_root_arch_common "${_root_docdir}/ARCHITECTURE_COMMON.md" ABSOLUTE) foreach(_f IN LISTS CMAKE_INSTALL_MANIFEST_FILES) get_filename_component(_name "${_f}" NAME) get_filename_component(_f_abs "${_f}" ABSOLUTE) - if(_name MATCHES "^REVIEW.*\.(md|das)$" AND NOT _f_abs STREQUAL _root_common) + if((_name MATCHES "^REVIEW.*\.(md|das)$" + OR _name MATCHES "^(ARCHITECTURE[A-Za-z_]*|LAWS|MASTERPLAN[A-Za-z_]*|PERF_LEDGER|PROFILE|THINKING)\.md$" + OR _name MATCHES "^followup_.*\.md$") + AND NOT _f_abs STREQUAL _root_common + AND NOT _f_abs STREQUAL _root_arch_common) list(APPEND _review_leaks "${_f}") endif() endforeach() if(_review_leaks) list(JOIN _review_leaks "\n " _review_leaks_text) - message(FATAL_ERROR "install carries repo-internal review files:\n ${_review_leaks_text}") + message(FATAL_ERROR "install carries repo-internal rule/record documents:\n ${_review_leaks_text}") endif() ]]) diff --git a/LAWS.md b/LAWS.md index 11136b1e75..f64a16c122 100644 --- a/LAWS.md +++ b/LAWS.md @@ -20,3 +20,4 @@ compacted, or cited as rules. | 2026-08-27 | daslib/.lint_config, .claude/agents/harvester.md (new) | comment sweep redesign: "remove hard lint rune, but have a process where comments are removed - recommendations are takend - rules and facts migrated to ARCHITECTURE.md and REVIEW.md, and then most important comments reduced to one-liners and restored" + "disjoint. daslib been swept already... anything comment heavy in dasLLAMA" + flashlight for NEEDS RULING: "this is going to be massive, and way outside my ability to focus and scope properly. so only important ones survive and the rest u do your best" | | 2026-08-27 | .claude/agents/harvester.md (RENAME verdict) | "what the previous resque-boat agent did for comments - the most interesting part - is suggest renames. like 'this comment is rename in disguise'. i don't want to loose that at all. its important" - RENAME added to the harvester's verdicts; comment stays until the rename lands | | 2026-08-27 | .claude/agents/harvester.md (RENAME precedence) | "rename should be first option, before any rescue attempts. it is the strongest." - RENAME tested before RULE/FACT/KEEP/DROP/TODO; only a comment no name can carry goes on to filing | +| 2026-08-27 | CLAUDE.md (document system), CMakeLists.txt (install gate), ci/smoke_test_bundle.sh, daslib/.lint_config + utils/lint | "no random .md next to shipped code ... REVIEW.md and REVIEW.das go there, if they exist. so does ARCHITECTURE.md ... but there should be no other .md there whatsoever. lint should catch mechanically ... and related CMAKE install thing is to not install .md unless explicitly listed" + "LAWS.md can stay in the folder just like REVIEW.md, but should never ever be installed as well" - rule_docs_only lint tag; install manifest gate + bundle smoke test ban ARCHITECTURE.md/LAWS.md/ledgers; ARCHITECTURE_COMMON.md ships beside REVIEW_COMMON.md | diff --git a/ci/smoke_test_bundle.sh b/ci/smoke_test_bundle.sh index 27753f851b..7ae788c463 100644 --- a/ci/smoke_test_bundle.sh +++ b/ci/smoke_test_bundle.sh @@ -219,17 +219,24 @@ echo "Shipped skills:" printf ' %-30s ' "bundle layout" if [[ -f "$BUNDLE/skills/daslang/SKILL.md" && -d "$BUNDLE/skills/daslang/references" \ && ! -d "$BUNDLE/skills/internal" && -f "$BUNDLE/.claude/skills/daslang/SKILL.md" \ - && -f "$BUNDLE/REVIEW_COMMON.md" && -f "$BUNDLE/.claude/agents/dragon.md" ]]; then + && -f "$BUNDLE/REVIEW_COMMON.md" && -f "$BUNDLE/ARCHITECTURE_COMMON.md" \ + && -f "$BUNDLE/.claude/agents/dragon.md" ]]; then echo "OK" PASS=$((PASS + 1)) else - echo "FAIL (need skills/daslang/{SKILL.md,references/}, .claude/skills/daslang, .claude/agents/dragon.md, REVIEW_COMMON.md, and no skills/internal/)" + echo "FAIL (need skills/daslang/{SKILL.md,references/}, .claude/skills/daslang, .claude/agents/dragon.md, REVIEW_COMMON.md, ARCHITECTURE_COMMON.md, and no skills/internal/)" FAIL=$((FAIL + 1)) fi -# REVIEW.md / REVIEW.das are 100% internal -- per-folder review machinery never -# ships. REVIEW_COMMON.md is the one exception: adopting repos vendor from it. -printf ' %-30s ' "no REVIEW.md/REVIEW.das" -REVIEW_LEAKS=$(find "$BUNDLE" \( -name "REVIEW*.md" -o -name "REVIEW*.das" \) ! -path "$BUNDLE/REVIEW_COMMON.md" 2>/dev/null) +# Repo-internal rule/record documents never ship: REVIEW.md / REVIEW.das, +# ARCHITECTURE.md, LAWS.md, MASTERPLAN logs, followup/perf/profile ledgers. +# REVIEW_COMMON.md and ARCHITECTURE_COMMON.md at the bundle root are the two +# exceptions: adopting repos vendor from them. +printf ' %-30s ' "no internal rule/record docs" +REVIEW_LEAKS=$(find "$BUNDLE" \( -name "REVIEW*.md" -o -name "REVIEW*.das" \ + -o -name "ARCHITECTURE*.md" -o -name "LAWS.md" -o -name "MASTERPLAN*.md" \ + -o -name "PERF_LEDGER.md" -o -name "PROFILE.md" -o -name "THINKING.md" \ + -o -name "followup_*.md" \) \ + ! -path "$BUNDLE/REVIEW_COMMON.md" ! -path "$BUNDLE/ARCHITECTURE_COMMON.md" 2>/dev/null) if [[ -z "$REVIEW_LEAKS" ]]; then echo "OK" PASS=$((PASS + 1)) diff --git a/daslib/.lint_config b/daslib/.lint_config index ac824f7360..4378b565d7 100644 --- a/daslib/.lint_config +++ b/daslib/.lint_config @@ -2,3 +2,8 @@ # REVIEW.md / ARCHITECTURE.md, site-local one-liners stay - not formatter-deleted. [format] force_clean_comments = false + +# Rule documents only beside the stdlib sources (LINT025): ledgers, plans and notes +# live in plans/ (repo root). +[docs] +rule_docs_only = true diff --git a/daslib/ARCHITECTURE.md b/daslib/ARCHITECTURE.md index ae914280e2..c4fb2cc6c8 100644 --- a/daslib/ARCHITECTURE.md +++ b/daslib/ARCHITECTURE.md @@ -698,3 +698,82 @@ module; entries are anchored to symbols. - **`_table_index_and_init` exists for infer's `default_init_containers` rewrite of non-store `tab[key]`** - it has no daslib call site and is not dead. + +## 37. linq_fold module layout and the SourceAdapter contract + +- **Charters.** `linq_fold` is dispatch only: source recognition (`try_splice_patterns`), + adapter construction, the tier-2 `fold_linq_default` cascade, the `LinqFold` call macro, + and the single registrar `register_all_linq_fold_rows`. `linq_fold_common` holds what no + source can influence - the pattern-table types, the walker, the predicate library, the + chain pre-passes, the generic emit lanes, the `splice_patterns` registry, and the abstract + `SourceAdapter`. Each remaining file owns one source: `linq_fold_array` (array, zip, + array-join), `linq_fold_decs`, `linq_fold_json`, `linq_fold_table`, and pugixml's + `linq_fold_xml`. `linq_fold_sql` is the exception - it recognizes a `[sql_table]` source + and hands the whole chain to the `_sql` macro, so it has no adapter and no emit. A source + file requires `linq_fold_common` and never a sibling source. +- **`try_splice_patterns` tries recognizers in a fixed order** - sql, decs, xml (behind the + pugixml `static_if`), json, table - and the array arm runs last with no recognizer of its + own: it is what claims a chain nobody else claimed. +- **The adapter contract is four abstract methods; everything else on the base has a + default.** `bind_name`, `element_type`, `wrap_source_loop` and `wrap_invoke` are abstract, + so every source answers them. The per-operation hooks (`emit_loop_or_count`, + `emit_reverse_skip_into_tail`, `emit_reverse_last_backward`, `emit_distinct_take_loop`, + `build_group_by_adapter`, `emit_join_hook`) return null on the base, and a null sends the + caller down its generic path. The capability methods (`can_group_by`, `can_join`, + `can_reserve_by_length`, `has_own_loop_or_count_lane`, `supports_direct_return`, + `effective_dispatch`, `defers_materialization`, `count_shortcut`, `invoke_param_type`, + `name_prefix`) are the source answering questions about itself. daslang classes have no + `is`/`as` downcast, so an emit fn holding a `SourceAdapter?` reaches source-specific + behavior only through these methods - there is no place to write a per-source branch, and + adding a source needs no edit to a central enum or switch. +- **The capability defaults are fail-closed, and `supports_direct_return` is the one that + matters.** A wrong `true` emits a mid-loop `return` that escapes a nested-callback walk + (decs's `for_each_archetype`); a wrong `false` only costs the slower state-var + + find-stop + tail path. +- **`loop_source_expr` / `loop_source_name` is what lights up the array-shaped lanes.** The + shared lanes spell their loop and their `length(...)` reads with the name, and read the + expr for compile-time facts - the `type_has_length` reserve gate and the join's srcA + element-type check. Array and table override both, xml and json override the name only, + decs overrides neither. +- **`ProjectedSourceAdapter` absorbs a leading source projection.** When a row captures the + optional `srcsel` slot (`source |> _select(f) |> order/distinct/take`), the dispatcher + wraps the real adapter in this decorator: it binds `projName = f(rawElem)` on top of the + per-element body and delegates loop and invoke to the inner adapter, so every emit sees the + projected element unchanged. It leaves `loop_source_expr` / `loop_source_name` / + `can_reserve_by_length` at the base defaults, which disables the source-direct fast paths - + those would bypass the projection. +- **Rows are per operator family, not per source.** The seven `build_*_rows` fns + (order_family, loop_or_count, reverse, distinct, group_by, join, zip) emplace 17 rows into + the one `splice_patterns` table; only zip's builder lives outside `linq_fold_common`. + Source gating happens inside a row - through a `RequiresPredicate` that asks the adapter + (`can_group_by_source`, `can_join_source`, `array_source`, `non_array_source`) or through + a hook that returns null. Registration order is match priority: the dispatcher walks the + table top to bottom and the first row whose chain matches and whose emit returns non-null + wins, so more specific chains are declared first. A row whose slot chain is a strict prefix + of a later row's shadows that row (`chain_prefix_of`). +- **The walker consumes the whole call list or fails.** `match_pattern` walks the flattened + calls left to right against the row's slots: `c_one` must match and advances both cursors, + `c_opt` advances both on a match and skips the slot otherwise, `c_chain` matches greedily + while the call is in the slot's name set and always succeeds - including with zero matches, + which still creates the (empty) `captures.many` entry, so an emit fn can rely on the key + existing. Any call left unconsumed after the last slot is a no-match. The `requires` + predicates run last, against the populated captures. +- **`Captures.single_name` exists because a pre-pass renames the op without touching the + AST.** `normalize_order_reverse` folds a trailing `reverse` into the order op by swapping + the tuple's `LinqCall` record, leaving the `ExprCall` alone; deriving the name back from + the `ExprCall` (`call_norm_name`) would silently undo the swap. The walker therefore + records the `LinqCall.name` at match time, and any emit fn that cares about the + post-normalize name reads `single_name`. +- **`alias_table` names the op-name groups a `m_alias` slot admits, and a missing name fails + silently.** A terminator the lanes handle but `loop_terminator_family` omits never matches + the terminator slot, so the chain leaves the splice and lands on the tier-2 cascade with no + diagnostic - the symptom is a slow chain, not an error. +- **Predicates and emit fns are named module-level functions, referenced as + `@@<RequiresPredicate> name` / `@@<EmitFn> name`.** An anonymous `@@(...)` lambda compiles + to a `_localfunction_*` symbol the LLVM JIT pass cannot resolve; a named function has a + stable address. +- **The table lane's key and value binds are asymmetric on purpose.** `keys(tab)` yields + non-const elements (writable temp copies), so the key lanes rebind the loop var through a + `let` before the body sees it; `values(tab)` over the const table param already yields + `V& const` and binds directly. Keys are workhorse types, so the copy is free, and + downstream `==const` composition (a `push_clone` of a bare projected key) needs the const. diff --git a/daslib/REVIEW.md b/daslib/REVIEW.md index 3e5e85b6a9..4cdf836b1f 100644 --- a/daslib/REVIEW.md +++ b/daslib/REVIEW.md @@ -1,7 +1,7 @@ # daslib Code Review Checklist **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: -`daslib/ARCHITECTURE.md`. Planned work: `daslib/followup_comment_sweep.md`. +`daslib/ARCHITECTURE.md`. Planned work: `plans/daslib_comment_sweep.md` (repo root). A diff touching the linq family - `linq*.das`, `sql_*.das` - applies `REVIEW_LINQ.md` too. **A diff that adds or changes a lint report path (`perf_warning` / `style_warning` / diff --git a/daslib/linq_fold_array.das b/daslib/linq_fold_array.das index 55662dc856..4cc77550cb 100644 --- a/daslib/linq_fold_array.das +++ b/daslib/linq_fold_array.das @@ -8,7 +8,7 @@ module linq_fold_array shared public //! linq_fold array source adapters: ArrayAdapter / ZipAdapter / ArrayJoinAdapter plus the //! array-shaped emit lanes (counter / accumulator / early-exit / loop-or-count / zip) and -//! their row-builders. Requires linq_fold_common. See daslib/linq_fold.md. +//! their row-builders. Requires linq_fold_common. See daslib/ARCHITECTURE.md sec. 37. require daslib/linq public require daslib/ast_boost diff --git a/daslib/linq_fold_common.das b/daslib/linq_fold_common.das index e264dfac87..e939ed46cf 100644 --- a/daslib/linq_fold_common.das +++ b/daslib/linq_fold_common.das @@ -8,7 +8,7 @@ module linq_fold_common shared public //! linq_fold source-independent kernel: pattern-table types, walker, predicate library, //! chain transforms, generic emit lanes, and the abstract ``SourceAdapter`` base. Carries -//! the shared ``splice_patterns`` registry. See daslib/linq_fold.md "Module layout & adapter contract". +//! the shared ``splice_patterns`` registry. See daslib/ARCHITECTURE.md sec. 37. require daslib/linq public require daslib/ast_boost diff --git a/daslib/linq_fold_decs.das b/daslib/linq_fold_decs.das index a01c0f376f..7815cdf7b9 100644 --- a/daslib/linq_fold_decs.das +++ b/daslib/linq_fold_decs.das @@ -8,7 +8,7 @@ module linq_fold_decs shared public //! linq_fold decs (ECS) source adapters: DecsAdapter / DecsJoinAdapter, the decs-bridge tuple //! scanners/rewriters, and the decs-specific emit hooks (count_shortcut, reverse-skip-into-tail, //! join, min_by/max_by). Terminators ride the generic lanes in linq_fold_common via DecsAdapter. -//! Requires linq_fold_common. See daslib/linq_fold.md. +//! Requires linq_fold_common. See daslib/ARCHITECTURE.md sec. 37. require daslib/linq public require daslib/ast_boost diff --git a/daslib/linq_fold_json.das b/daslib/linq_fold_json.das index 5827474f4c..6690c95847 100644 --- a/daslib/linq_fold_json.das +++ b/daslib/linq_fold_json.das @@ -18,7 +18,7 @@ module linq_fold_json shared public //! ``for_each_archetype``), so this module pulls in no json dependency. The field-pruning //! row-usage scanners (``RowUsageScanner`` / ``collect_row_usage`` / ``RowFieldFlattener`` / //! ``flatten_row_to_locals`` / ``hoist_prelude``) are shared with the XML and table adapters and -//! live in ``daslib/linq_fold_common.das``. See daslib/linq_fold.md. +//! live in ``daslib/linq_fold_common.das``. See daslib/ARCHITECTURE.md sec. 37. require daslib/ast_boost require daslib/ast_match diff --git a/daslib/linq_fold_sql.das b/daslib/linq_fold_sql.das index 8a1453f919..54425e9b4e 100644 --- a/daslib/linq_fold_sql.das +++ b/daslib/linq_fold_sql.das @@ -10,7 +10,7 @@ module linq_fold_sql shared public //! materialized full-table ``array<Row>`` in memory. There is no emitting adapter — ``_sql`` is a //! complete parallel lowering; this module only recognizes the source. Companion to //! ``daslib/linq_fold``; provider-neutral (matches the ``[sql_table]`` annotation, not a runner -//! type), so it loads unconditionally. See daslib/linq_fold.md. +//! type), so it loads unconditionally. See daslib/ARCHITECTURE.md sec. 37. require daslib/linq_fold_common public require daslib/ast_boost diff --git a/daslib/linq_fold_table.das b/daslib/linq_fold_table.das index 365a39f019..371a1e600c 100644 --- a/daslib/linq_fold_table.das +++ b/daslib/linq_fold_table.das @@ -15,7 +15,7 @@ module linq_fold_table shared public //! field-pruning); bare ``count()`` folds to O(1) ``length(tab)``; a plain ``distinct`` over raw keys/kv //! elements is dropped (keys are unique by construction). In-tree companion to ``daslib/linq_fold`` //! (required unconditionally; the matcher returns null for non-table chains). Emits ``keys`` / ``values`` -//! / the kv zip BY NAME at the user's splice site. See history/linq_fold/LINQ_TO_TABLE.md. +//! / the kv zip BY NAME at the user's splice site. See daslib/ARCHITECTURE.md sec. 37. require daslib/ast_boost require daslib/ast_match diff --git a/daslib/lint_config.das b/daslib/lint_config.das index 483be36c9b..e177f79ad3 100644 --- a/daslib/lint_config.das +++ b/daslib/lint_config.das @@ -31,6 +31,11 @@ module lint_config shared private //! directory may hold its own ``.lint_config``; ``format_policy_for`` walks //! from the file's directory to the repository root and the nearest file //! that defines a key wins that key (``[rules]`` stays repo-root-only). +//! +//! The ``[docs]`` table is a FOLDER property — no cascade, no inheritance. +//! ``rule_docs_only = true`` marks a folder that carries rule documents only; +//! ``rule_docs_only_at`` reads that one file, and the LINT025 pass in +//! ``utils/lint/main.das`` reports any other ``.md`` sitting beside the code. require daslib/fio require daslib/json @@ -257,17 +262,38 @@ struct public FormatPolicy { force_clean_comments : bool = false } -def private read_format_table(path : string) : JsonValue? { +def private read_config_table(path, table_name : string) : JsonValue? { let body = fread(path) if (empty(body)) return null var error : string var root : JsonValue? = read_toml(body, error) if (root == null || !(root.value is _object)) return null var root_obj & = unsafe(root.value as _object) - if (!key_exists(root_obj, "format")) return null - var fmt_jv : JsonValue? = root_obj["format"] - if (fmt_jv == null || !(fmt_jv.value is _object)) return null - return fmt_jv + if (!key_exists(root_obj, table_name)) return null + var tbl_jv : JsonValue? = root_obj[table_name] + if (tbl_jv == null || !(tbl_jv.value is _object)) return null + return tbl_jv +} + +def private read_format_table(path : string) : JsonValue? { + return read_config_table(path, "format") +} + +def private read_docs_table(path : string) : JsonValue? { + return read_config_table(path, "docs") +} + +//! True when the ``.lint_config`` at `config_path` sets ``[docs] rule_docs_only = true`` — the folder +//! carries rule documents only (``REVIEW*.md`` / ``ARCHITECTURE*.md`` / ``LAWS.md``). A folder property, +//! NOT the ``[format]`` cascade: only that one file is read, and anything missing reads as false. +def public rule_docs_only_at(config_path : string) : bool { + var docs_jv = read_docs_table(config_path) + return false if (docs_jv == null) + var docs & = unsafe(docs_jv.value as _object) + return false if (!key_exists(docs, "rule_docs_only")) + var val_jv : JsonValue? = docs["rule_docs_only"] + return false if (val_jv == null || !(val_jv.value is _bool)) + return val_jv.value as _bool } def private config_dir_of(file : string) : string { diff --git a/history/README.md b/history/README.md index f2206d71b9..5d614a74c2 100644 --- a/history/README.md +++ b/history/README.md @@ -26,8 +26,8 @@ Process and criteria: `skills/internal/doc_archiving.md`. `modules/dasSpirv/ARCHITECTURE.md` and `history/dasSpirv/MASTERPLAN_LOG.md`) - `dasVulkan/` - the original boost-layer design plan, carried over when the module moved in-tree; the living docs are `modules/dasVulkan/CLAUDE.md` and its `ROADMAP.md` -- `linq_fold/` - the linq_fold arc's plans and audits; the living masterplan is - `daslib/linq_fold.md`, results stay at `benchmarks/sql/results.md` +- `linq_fold/` - the linq_fold arc's plans, audits and the archived masterplan; the living + reference is `daslib/ARCHITECTURE.md` sec. 37, results stay at `benchmarks/sql/results.md` - `examples/` - plans, findings, and research notes behind shipped examples - `tests/` - one-off test-migration audits @@ -142,3 +142,4 @@ listed in the log below - search it first when hunting for a doc. - 2026-08-24 `modules/dasLLAMA/quant_lane_audit_plan.md` -> `history/dasLLAMA/quant_lane_audit_plan.md` - the format x kernel-lane x platform coverage audit (holes plugged; the kq-native and fused-arm work it drove shipped) - 2026-08-27 `modules/dasSpirv/MASTERPLAN.md` (plan sections + implementation log) -> `history/dasSpirv/MASTERPLAN_LOG.md` - the Phase 0-11 SPIR-V emitter build-out, its phase plans, and the tutorial-prerequisite backlog (the doc's present-tense half stayed as `modules/dasSpirv/ARCHITECTURE.md`) - 2026-08-27 `modules/dasMetal/MASTERPLAN.md` (plan sections + implementation log) -> `history/dasMetal/MASTERPLAN_LOG.md` - the Phase 0-8 MSL emitter build-out and the dasLLAMA GPU-resident prefill/decode chase (the doc's present-tense half, including the shared cross-backend ledger, stayed as `modules/dasMetal/ARCHITECTURE.md`) +- 2026-08-27 `daslib/linq_fold.md` -> `history/linq_fold/linq_fold.md` - the PR A-G splice-pattern refactor masterplan, its phase logs and decision log (the module layout, `SourceAdapter` contract and walker contract were distilled into `daslib/ARCHITECTURE.md` sec. 37, which the `linq_fold_*` headers cite) diff --git a/daslib/linq_fold.md b/history/linq_fold/linq_fold.md similarity index 93% rename from daslib/linq_fold.md rename to history/linq_fold/linq_fold.md index a3e097f7c2..d539c32887 100644 --- a/daslib/linq_fold.md +++ b/history/linq_fold/linq_fold.md @@ -1,6 +1,13 @@ -# linq_fold.das refactor - masterplan +# linq_fold.das refactor - archived masterplan -Living document. Update **Status** + **Decision log** as phases ship. +Archived 2026-08-27 from `daslib/linq_fold.md`. The operative half - the module layout, the +`SourceAdapter` contract, and the pattern-table walker contract - was distilled into +`daslib/ARCHITECTURE.md` sec. 37, which the `linq_fold_*` module headers now cite. +Everything below is that document's plan material, phase logs, PR sketches and decision log, +verbatim; the projected alias/predicate tables it carries were snapshots of a planned state +and the shipped code has since diverged from them. + +--- ## Status @@ -58,23 +65,6 @@ Match-side (Phase E) and emit-side (Phase F) are uniform. Phase G makes the **so Behavior-preserving (not byte-identical for decs): full `tests/` green on INTERP+JIT+AOT; `benchmarks/sql` decs/array/zip/xml cells within thermal noise with identical alloc profiles. The array/zip/xml seam commits (`LoopDispatch` threading) are AST byte-identical. - [ ] **PR G4** (separate) - `linq_fold_xml.das` validates the interface (dasPUGIXML source; field access via `operator .` overloads). -### Module layout & adapter contract - -The adapter is an abstract `class SourceAdapter` (`[macro_interface]`, so every method is macro-callable - no per-method `[macro_function]`). One subclass per data source carries that source's data as fields. The contract is four virtual methods: - -- `bind_name(at) : string` - per-element bind name. -- `element_type() : TypeDeclPtr` - source element type. -- `wrap_source_loop(loopShape : LoopDispatch; var body; at) : Expression?` - emit the per-element iteration (array `for`, decs `for_each_archetype{,_find}`, zip lockstep, joins hash+probe). `loopShape` is the loop-framing knob consumed only by nested-callback sources (decs); direct-return sources frame an unconditional loop and ignore it. -- `wrap_invoke(var stmts; retType; wrapIter; at) : Expression?` - outer invoke binding sources as params. - -Emit fns hold a `SourceAdapter?` (via `EmitCtx.src` or an `adapter` local) and call these virtually. **daslang classes have no `is`/`as` downcast** (variant-only), so source-specific data is never pulled off a base pointer by downcasting - it goes through virtual methods. Beyond the 4 dispatch methods the base also declares 6 default-null **per-operation hook methods** (`emit_loop_or_count` / `emit_reverse_skip_into_tail` / `emit_reverse_last_backward` / `emit_distinct_take_loop` / `build_group_by_adapter` / `emit_join_hook`) that the owning source overrides; the generic lane falls back to its inline (array) body when the hook returns null. (`XmlAdapter` overrides the two reverse hooks with a **backward DOM walk** - `last_child`/`previous_sibling`, both O(1) in pugixml: `emit_reverse_skip_into_tail` collects only the last N children for `reverse |> take(N)` (m5f `reverse_take` 88.9 -> 0.0 ns/op), and `emit_reverse_last_backward` returns the last element in one step for a no-predicate `last()` / `reverse |> first`. Predicated `[where] |> last` stays on the forward walk - reverse DOM traversal is ~2x cache-hostile per node, profiled - and the named 3-arg `from_xml_node` form falls back to the buffer path since pugixml has no last-named-child primitive.) (`emit_join_hook` is the standalone-join dispatch: the single `join_general` pattern's thin `emit_join` routes to it, so each source supplies its own join body - array `for`+2-param invoke, decs `for_each_archetype`, XML field-pruned DOM walk - with no parallel per-source join pattern.) It also declares **capability methods** the source answers about itself - `can_group_by` / `can_join` / `can_reserve_by_length` / `has_own_loop_or_count_lane` (bool, default false) and `name_prefix` (string) - which replaced the old `kind() : AdapterKind` enum + per-site switches, so a new source only implements the methods (no central enum to extend). The `can_group_by` / `can_join` capabilities are queried from the `can_group_by_source` / `can_join_source` `RequiresPredicate`s (which thread the adapter), so the single `group_by` / `join_general` pattern admits any capable source and the adapter's `build_group_by_adapter` / `emit_join_hook` does the source/srcb-shape gating (null -> tier-2). The **generic-lane source feed** is the getter pair `loop_source_expr()`/`loop_source_name()` (default null/"" on base; array/table override both, xml/json name-only): the shared array-shaped lanes (counter / early-exit / dedup / order family / hashed join) emit their loops and `length()` reads against the name and read the expr for compile-time facts (reserve hints, srcB element validation), so overriding the pair is what lights those lanes up for a source; the decs-specific getters were removed in G2a so the base (and thus `linq_fold_common`) is free of `DecsAdapter`/ECS coupling. One decorator subclass lives in `linq_fold_common`: `ProjectedSourceAdapter` wraps any inner adapter to absorb a leading `_select(f)` source projection (the `srcsel` slot) - it binds `projName = f(rawElem)` atop the per-element body and delegates `wrap_source_loop`/`wrap_invoke`/`name_prefix` to the inner adapter, leaving the base no-op `loop_source_expr`/`loop_source_name`/`can_reserve_by_length` so source-direct fast paths (which would bypass the projection) stay disabled. This lets order/distinct splices fuse over `source |> _select(f) |> ...` for any source. - -**The capability defaults are fail-closed, and `supports_direct_return` is the one that matters.** A wrong `true` mis-emits a `return` that escapes a nested-callback walk; a wrong `false` only costs the slower state-var + find-stop + tail path. A new source leaves it at the default until its loop really is a plain `for`. - -**Table lane binds are asymmetric on purpose.** `keys(tab)` yields NON-const elements (writable temp copies), so the key lanes rebind the loop var through a `let` before the body sees it; `values(tab)` over the const table param already yields `V& const` and binds directly. The rebind is not redundant boilerplate - keys are workhorse types so the copy is free, and downstream `==const` composition (`push_clone` of a bare projected key) needs the const. - -**Realized module layout (post-G3d):** `linq_fold_common` (kernel + abstract base + adapter-pure generic lanes - terminator/fold-array plus the source-generic loop_or_count / counter / accumulator / early-exit lanes, with `LoopDispatch` + the per-op `!supports_direct_return` state path that lets nested-callback sources ride the early-exit lane - + `splice_patterns` + `DecsBridgeShape`/`extract_decs_bridge`) <- `linq_fold_array` (Array/Zip/ArrayJoin adapters + the zip/join emit `emit_zip`/`emit_array_join` + array row-builders) and `linq_fold_decs` (Decs/DecsJoin adapters + decs-bridge visitors + the decs dispatcher `emit_loop_or_count_lane_decs` + the decs-specific hooks `emit_decs_count_archsize`/`emit_decs_reverse_skip_into_tail`/`emit_decs_join_impl`/`emit_decs_min_max_by` - the parallel terminator scaffold is gone, decs rides the generic lanes via `DecsAdapter`); the engine `linq_fold` requires all three and holds only the dispatcher + the `LinqFold` macro + the single `register_all_linq_fold_rows`. Adding a source = a new `linq_fold_<src>.das` subclass module + one `require` + one `build_<src>_rows()` call in the engine registrar. Later sources follow that recipe: `linq_fold_json` (`JsonAdapter`/`JsonJoinAdapter`), `pugixml/linq_fold_xml` (`XmlAdapter`, optional), `daslib/linq_fold_sql` (pass-through detector, provider-neutral), and `linq_fold_table` (`TableAdapter` over `each_kv`/`keys`/`values` heads - kv usage-pruned slot walks, no new rows; arc plan in `history/linq_fold/LINQ_TO_TABLE.md`). - ## Goal Split `_fold` splice machinery into two layers: @@ -181,29 +171,9 @@ var splice_patterns : array<SplicePattern> // PR D: collapsed from per-plan Predicates and emit archetypes are NAMED module-level `def` functions wrapped at use sites with `@@<RequiresPredicate>` / `@@<EmitFn>` (anonymous `@@(...)` lambdas produce `_localfunction_*` symbols that the LLVM JIT pass can't resolve - named functions take a stable address). -### Walker contract - -```das -def match_pattern(p : SplicePattern; - var calls : array<tuple<ExprCall?; LinqCall?>>; - var top : Expression?) : MatchResult -``` - -Walks `calls` left-to-right. For each slot: - -- `one` - current call must match (name + arity if specified); both cursors advance. -- `optional` - if current call matches, both cursors advance; otherwise the slot is skipped without consuming. -- `chain` (PR B1) - greedy match-while-in-set. Captured as `array<ExprCall?>` into `captures.many[capture_name]`. Always succeeds (0+); empty match still creates the `many` entry so emit fns can rely on `c.many |> key_exists("...")`. Pairs with `m_one_of` via the `slot_chain_of(names, cap)` convenience constructor. - -After all slots, no unconsumed calls remain. If any of the above fails -> `MatchResult(no_match = null)`. - -Then each `RequiresPredicate` in `p.requires` is evaluated against the populated `Captures` and the peeled `top`. All must return true. If any fails -> `MatchResult(no_match = null)`. - -Returns `MatchResult(matched <- captures)` on full success (move semantics - `Captures` is a table). Caller binds `var r <- match_pattern(...)` and reads via `if (r is matched) { let c & = r as matched; ... }`. - ### Alias table (named op-name groups) -The snippet below is the projected end-state at PR D. The authoritative live list is the `alias_table` literal in [daslib/linq_fold.das](linq_fold.das). Status reflects what's populated through PR B1. +The snippet below is the projected end-state at PR D. The authoritative live list is the `alias_table` literal in [daslib/linq_fold.das](../../daslib/linq_fold.das). Status reflects what's populated through PR B1. ```das // projected end-state at PR D @@ -224,7 +194,7 @@ var alias_table : table<string; array<string>> <- { ### Predicate library -Module-level named `RequiresPredicate` constants for reuse across patterns. As with `alias_table`, this table shows the projected end-state - see [daslib/linq_fold.das](linq_fold.das) for what's actually defined today (PR A: `array_source`, `take_arg_is_int`, `no_terminator`). +Module-level named `RequiresPredicate` constants for reuse across patterns. As with `alias_table`, this table shows the projected end-state - see [daslib/linq_fold.das](../../daslib/linq_fold.das) for what's actually defined today (PR A: `array_source`, `take_arg_is_int`, `no_terminator`). | Name | Status | Meaning | |---|---|---| @@ -416,7 +386,7 @@ Both planner bodies (~165 LOC across array + decs) hard-deleted; new code (rows **Branch:** `bbatkin/linq-fold-pattern-table-pre` -**Scope (delivered):** Final collapse promised at [linq_fold.md:212](daslib/linq_fold.md#L212). 7 per-plan tables -> 1 `splice_patterns`; 12 stub fns + 12-line `LinqFold.visit` cascade -> 1 `try_splice_patterns` dispatcher. +**Scope (delivered):** Final collapse promised at [linq_fold.md:212](linq_fold.md#L212). 7 per-plan tables -> 1 `splice_patterns`; 12 stub fns + 12-line `LinqFold.visit` cascade -> 1 `try_splice_patterns` dispatcher. **Commit 1 - table consolidation:** - Delete the 7 `var private plan_<X>_patterns : array<SplicePattern>` declarations; keep `splice_patterns` (previously declared empty since PR A). diff --git a/modules/dasClipboard/daslib/.lint_config b/modules/dasClipboard/daslib/.lint_config new file mode 100644 index 0000000000..005caa12b7 --- /dev/null +++ b/modules/dasClipboard/daslib/.lint_config @@ -0,0 +1,4 @@ +# Rule documents only beside the shipped module code (LINT025): ledgers, plans and +# notes live at the module root or in plans/ (repo root). +[docs] +rule_docs_only = true diff --git a/modules/dasImgui/daslib/.lint_config b/modules/dasImgui/daslib/.lint_config new file mode 100644 index 0000000000..005caa12b7 --- /dev/null +++ b/modules/dasImgui/daslib/.lint_config @@ -0,0 +1,4 @@ +# Rule documents only beside the shipped module code (LINT025): ledgers, plans and +# notes live at the module root or in plans/ (repo root). +[docs] +rule_docs_only = true diff --git a/modules/dasLLAMA/dasllama/.lint_config b/modules/dasLLAMA/dasllama/.lint_config new file mode 100644 index 0000000000..005caa12b7 --- /dev/null +++ b/modules/dasLLAMA/dasllama/.lint_config @@ -0,0 +1,4 @@ +# Rule documents only beside the shipped module code (LINT025): ledgers, plans and +# notes live at the module root or in plans/ (repo root). +[docs] +rule_docs_only = true diff --git a/modules/dasLLVM/daslib/.lint_config b/modules/dasLLVM/daslib/.lint_config new file mode 100644 index 0000000000..005caa12b7 --- /dev/null +++ b/modules/dasLLVM/daslib/.lint_config @@ -0,0 +1,4 @@ +# Rule documents only beside the shipped module code (LINT025): ledgers, plans and +# notes live at the module root or in plans/ (repo root). +[docs] +rule_docs_only = true diff --git a/modules/dasPUGIXML/daslib/.lint_config b/modules/dasPUGIXML/daslib/.lint_config new file mode 100644 index 0000000000..005caa12b7 --- /dev/null +++ b/modules/dasPUGIXML/daslib/.lint_config @@ -0,0 +1,4 @@ +# Rule documents only beside the shipped module code (LINT025): ledgers, plans and +# notes live at the module root or in plans/ (repo root). +[docs] +rule_docs_only = true diff --git a/modules/dasPUGIXML/daslib/linq_fold_xml.das b/modules/dasPUGIXML/daslib/linq_fold_xml.das index 63d8361413..8a9d70b5a4 100644 --- a/modules/dasPUGIXML/daslib/linq_fold_xml.das +++ b/modules/dasPUGIXML/daslib/linq_fold_xml.das @@ -10,7 +10,7 @@ module linq_fold_xml shared public //! ``from_xml_node(node[, name], type<Row>)`` chain emit an inlined DOM child-element walk (mirroring //! ``PUGIXML_boost``'s generator body) instead of driving the generator + an intermediate array. //! Optional companion to ``daslib/linq_fold`` — pulled in via ``require ?pugixml`` and gated by -//! ``static_if (typeinfo builtin_module_exists(pugixml))``. See daslib/linq_fold.md. +//! ``static_if (typeinfo builtin_module_exists(pugixml))``. See daslib/ARCHITECTURE.md sec. 37. require pugixml public require pugixml/PUGIXML_boost // nolint:STYLE030 — build_xml_row / read_xml_field / peek_xml_field are emitted via qmacro, invisible to the require-usage scan @@ -64,7 +64,7 @@ def private try_xml_under_guard_materialize(var body : Expression?; bindName : s // Fills `prelude` (the hoisted default-row decl, flatten path only) + `inner` (per-element, ends with // `body`): allUsed -> full build_xml_row; no field refs -> body only; pruned -> one `let xf_<f>` read -// per referenced field with body's `it.<f>` rewritten to it (struct dropped). See linq_fold.md. +// per referenced field with body's `it.<f>` rewritten to it (struct dropped). See daslib/ARCHITECTURE.md sec. 37. [macro_function] def private build_xml_materializer(var body : Expression?; bindName : string; declT : TypeDeclPtr; defName, chName : string; at : LineInfo; diff --git a/modules/dasSMT/daslib/.lint_config b/modules/dasSMT/daslib/.lint_config new file mode 100644 index 0000000000..005caa12b7 --- /dev/null +++ b/modules/dasSMT/daslib/.lint_config @@ -0,0 +1,4 @@ +# Rule documents only beside the shipped module code (LINT025): ledgers, plans and +# notes live at the module root or in plans/ (repo root). +[docs] +rule_docs_only = true diff --git a/modules/dasSQLITE/daslib/.lint_config b/modules/dasSQLITE/daslib/.lint_config new file mode 100644 index 0000000000..005caa12b7 --- /dev/null +++ b/modules/dasSQLITE/daslib/.lint_config @@ -0,0 +1,4 @@ +# Rule documents only beside the shipped module code (LINT025): ledgers, plans and +# notes live at the module root or in plans/ (repo root). +[docs] +rule_docs_only = true diff --git a/modules/dasTerminal/daslib/.lint_config b/modules/dasTerminal/daslib/.lint_config new file mode 100644 index 0000000000..005caa12b7 --- /dev/null +++ b/modules/dasTerminal/daslib/.lint_config @@ -0,0 +1,4 @@ +# Rule documents only beside the shipped module code (LINT025): ledgers, plans and +# notes live at the module root or in plans/ (repo root). +[docs] +rule_docs_only = true diff --git a/modules/dasTreeSitter/daslib/.lint_config b/modules/dasTreeSitter/daslib/.lint_config new file mode 100644 index 0000000000..005caa12b7 --- /dev/null +++ b/modules/dasTreeSitter/daslib/.lint_config @@ -0,0 +1,4 @@ +# Rule documents only beside the shipped module code (LINT025): ledgers, plans and +# notes live at the module root or in plans/ (repo root). +[docs] +rule_docs_only = true diff --git a/modules/dasVulkan/daslib/.lint_config b/modules/dasVulkan/daslib/.lint_config new file mode 100644 index 0000000000..005caa12b7 --- /dev/null +++ b/modules/dasVulkan/daslib/.lint_config @@ -0,0 +1,4 @@ +# Rule documents only beside the shipped module code (LINT025): ledgers, plans and +# notes live at the module root or in plans/ (repo root). +[docs] +rule_docs_only = true diff --git a/daslib/followup_comment_sweep.md b/plans/daslib_comment_sweep.md similarity index 100% rename from daslib/followup_comment_sweep.md rename to plans/daslib_comment_sweep.md diff --git a/tests/dasPUGIXML/parity_xml_order_distinct_group.das b/tests/dasPUGIXML/parity_xml_order_distinct_group.das index 14f6f2cdc2..7cc580ba51 100644 --- a/tests/dasPUGIXML/parity_xml_order_distinct_group.das +++ b/tests/dasPUGIXML/parity_xml_order_distinct_group.das @@ -220,7 +220,7 @@ def parity_order_by_take(t : T?) { } } -// ===== Deferred-materialization (handle-buffering) bounded-heap variants — see linq_fold.md ===== +// ===== Deferred-materialization (handle-buffering) bounded-heap variants — see daslib/ARCHITECTURE.md sec. 37 ===== [test] def parity_order_by_descending_take(t : T?) { diff --git a/tests/lint/test_lint_config.das b/tests/lint/test_lint_config.das index cc42d3663f..69e0f6ae43 100644 --- a/tests/lint/test_lint_config.das +++ b/tests/lint/test_lint_config.das @@ -315,10 +315,10 @@ def test_partition_by_format_policy(t : T?) { } [test] -def test_daslib_policy_is_comment_clean(t : T?) { - t |> run("daslib ships with force_clean_comments on") @@(t : T?) { +def test_daslib_policy_comments_harvested(t : T?) { + t |> run("daslib ships with force_clean_comments off") @@(t : T?) { let pol = format_policy_for(path_join(get_das_root(), "daslib/lint.das")) t |> success(pol.format_enabled) - t |> success(pol.force_clean_comments, "daslib/.lint_config arms the strip; turning it off un-ratchets the comment sweep") + t |> success(!pol.force_clean_comments, "daslib comments are harvested, not formatter-stripped; re-arming the strip is a decision, not drift") } } diff --git a/utils/lint/README.md b/utils/lint/README.md index e88066ab4e..7453882a41 100644 --- a/utils/lint/README.md +++ b/utils/lint/README.md @@ -10,6 +10,13 @@ Run: Exit 2 on any warning - CI's changed-files gate and the nightly full sweep both key on it. The SDK also ships a prebuilt `bin/lint.exe`. +One rule is the runner's own, because it is about folders rather than code: **LINT025**. A +directory whose `.lint_config` carries `[docs] rule_docs_only = true` may hold only rule +documents (`REVIEW*.md`, `ARCHITECTURE*.md`, `LAWS.md`); any other `.md` beside the sources +is reported once per invocation, from a walk of the directory roots the run was given. The +key is a folder property - it never cascades, unlike `[format]`. Fixture: `tests/lint025_*`, +which drives the CLI over a planted tree. + Design: the runner stays thin - rules live in the daslib modules (authoring rails: `skills/internal/perf_lint_authoring.md`, `skills/internal/style_lint_authoring.md`); suppression policy is `skills/perf_lint.md` / `skills/style_lint.md`. `tests/` here are the diff --git a/utils/lint/main.das b/utils/lint/main.das index 000357ad8e..4cfe489f70 100644 --- a/utils/lint/main.das +++ b/utils/lint/main.das @@ -175,6 +175,72 @@ def scan_das_files(path : string; var files : array<string>; var cache : table<s } } +// The .md basenames a `[docs] rule_docs_only = true` folder may carry: the folder's +// checklists (REVIEW.md and its routed companions), its architecture doc (and companion +// splits), and the rulings sidecar. Everything else is a ledger, a plan or a note. +def is_rule_doc_name(name : string) : bool { + return false if (!(name |> ends_with(".md"))) + return (name == "REVIEW.md" || name == "ARCHITECTURE.md" || name == "LAWS.md" + || name |> starts_with("REVIEW_") || name |> starts_with("ARCHITECTURE_")) +} + +// Directories under `path` whose OWN .lint_config sets [docs] rule_docs_only = true. +// Folder property, no cascade - each directory answers for itself. Skip rules mirror +// scan_das_files so both passes see the same tree. +def scan_docs_folders(path : string; var folders : array<string>; var seen : table<string; void?>) { + let st = stat(path) + return if (!st.is_valid || !st.is_dir || seen |> key_exists(path)) + seen |> insert(path, null) + if (rule_docs_only_at("{path}/.lint_config")) { + folders |> push(path) + } + fio::dir(path) $(name) { + return if (name == "." || name == ".." || name |> starts_with("_") || is_skip_dir(name)) + let full = "{path}/{name}" + let fst = stat(full) + if (fst.is_valid && fst.is_dir) { + scan_docs_folders(full, folders, seen) + } + } +} + +// The .md files in `folder` (non-recursive) that are not rule documents. +def collect_stray_rule_docs(folder : string; var out : array<string>) { + fio::dir(folder) $(name) { + return if (name == "." || name == ".." || !(name |> ends_with(".md")) || is_rule_doc_name(name)) + let full = "{folder}/{name}" + let fst = stat(full) + if (fst.is_valid && fst.is_reg) { + out |> push(full) + } + } +} + +// Folder-docs pass: one sweep per invocation over the directory roots the run was given. +// A ledger or plan parked beside shipped code is read by nobody and goes stale unnoticed, +// so a tagged folder admits rule documents only. Returns the finding count. +def run_rule_docs_pass(roots : array<string>; disabled_codes, enabled_codes : table<string>) : int { + return 0 if (key_exists(disabled_codes, "LINT025") + || (!empty(enabled_codes) && !key_exists(enabled_codes, "LINT025"))) + var folders : array<string> + var seen : table<string; void?> + for (root in roots) { + scan_docs_folders(root, folders, seen) + } + folders |> sort + var found = 0 + for (folder in folders) { + var strays : array<string> + collect_stray_rule_docs(folder, strays) + strays |> sort + for (md in strays) { + print("{md}: LINT025: stray .md beside shipped code - rule documents only (REVIEW*.md, ARCHITECTURE*.md, LAWS.md); ledgers and plans live at the module root or plans/ (repo root)\n") + found ++ + } + } + return found +} + // Validates a rule code: ^(LINT|PERF|STYLE)\d{3}$. Returns true if shape matches. def is_valid_rule_code(code : string) : bool { let n = length(code) @@ -559,14 +625,14 @@ def resolve_rule_filters(cfg : Config; var disabled_codes : table<string>&; var return true } -// Driver-mode positionals → sorted .das file list. Returns a process exit code, or -1 to proceed. -def collect_input_files(var files : array<string>&) : int { +// Driver-mode positionals → sorted .das file list, plus the directory roots among them +// (`dirs`, the folder-docs pass's scan roots). Returns a process exit code, or -1 to proceed. +def collect_input_files(var files : array<string>&; var dirs : array<string>&) : int { let cli_args <- get_user_args() let paths <- [for (arg in cli_args); arg; where !starts_with(arg, "-")] if (empty(paths)) { print("Error: no files or directories specified\n\n") - print_help(get_command_info(type<Config>), "utils/lint/main.das") - return 1 + return print_lint_help(1) } var cache : table<string; void?> var skipped = 0 @@ -575,7 +641,12 @@ def collect_input_files(var files : array<string>&) : int { // same-dir requires under a relative key while module-mount-routed requires register // absolute, tripping error 20605 "several modules with the same name" (e.g. dasPEG) let full = get_full_file_name(p) - scan_das_files(empty(full) ? p : replace(full, "\\", "/"), files, cache, skipped) + let target = empty(full) ? p : replace(full, "\\", "/") + let st = stat(target) + if (st.is_valid && st.is_dir) { + dirs |> push(target) + } + scan_das_files(target, files, cache, skipped) } if (empty(files)) { // All inputs were intentionally skipped (e.g. builtin.das) -- success. @@ -646,21 +717,29 @@ def print_summary(files_count, total_issues, total_errors, total_skipped, total_ } } +// --help / bad-argv banner. Returns the process exit code so main stays a dispatcher. +def print_lint_help(rc : int) : int { + if (rc == 0) { + print("Unified lint checker for daslang files.\n") + print("Runs paranoid, performance, and style lint passes with recursive directory scanning.\n\n") + } + print_help(get_command_info(type<Config>), "utils/lint/main.das") + if (rc == 0) { + print("\nPositional arguments: one or more files or directories to lint.\n") + } + return rc +} + [export] def main() : int { var r <- parse_args(type<Config>) if (r |> is_err) { print("error: {r |> unwrap_err}\n\n") - print_help(get_command_info(type<Config>), "utils/lint/main.das") - return 1 + return print_lint_help(1) } let cfg <- r |> move_unwrap if (cfg.help) { - print("Unified lint checker for daslang files.\n") - print("Runs paranoid, performance, and style lint passes with recursive directory scanning.\n\n") - print_help(get_command_info(type<Config>), "utils/lint/main.das") - print("\nPositional arguments: one or more files or directories to lint.\n") - return 0 + return print_lint_help(0) } let any_only = cfg.paranoid_only || cfg.perf_only || cfg.style_only let run_paranoid = !any_only || cfg.paranoid_only @@ -685,7 +764,8 @@ def main() : int { // Driver mode: scan positional paths, collect file list. var files : array<string> - let collect_rc = collect_input_files(files) + var docs_roots : array<string> + let collect_rc = collect_input_files(files, docs_roots) if (collect_rc >= 0) return collect_rc var total_issues = 0 @@ -725,5 +805,8 @@ def main() : int { } } + // Folder policy, not file policy: runs once per invocation, in both driver modes. + total_issues += run_rule_docs_pass(docs_roots, disabled_codes, enabled_codes) + return print_summary(length(files), total_issues, total_errors, total_skipped, total_excluded) } diff --git a/utils/lint/tests/lint025_rule_docs_only.das b/utils/lint/tests/lint025_rule_docs_only.das new file mode 100644 index 0000000000..aca8f88af2 --- /dev/null +++ b/utils/lint/tests/lint025_rule_docs_only.das @@ -0,0 +1,146 @@ +options gen2 +options indenting = 4 +options no_unused_function_arguments = false + +require dastest/testing_boost public + +require strings +require daslib/fio + +// LINT025 is a FOLDER rule the runner enforces with its own tree walk, not a compile pass, +// so it cannot ride the `expect 50503:N` fixture form the rule fixtures here use - the +// check is driven through the CLI over a planted tree. +// +// The tree carries three folders: `tagged` (rule_docs_only = true, one stray .md beside the +// five allowed names), `off` (rule_docs_only = false, one stray) and `plain` (no config, one +// stray). Exactly one finding may come back, naming `tagged/notes.md`. + +var private _tmp_dir : string +var private _tmp_dir_inited = false + + +def private tmp_root() : string { + if (!_tmp_dir_inited) { + let r = create_temp_directory_result("lint025_rule_docs_only") + // Fail-fast: an empty path would resolve against the working directory and plant + // the fixture tree in the worktree. + if (!(r is value)) { + panic("could not create temp dir for the LINT025 test: {unsafe(r.error)}") + } + _tmp_dir = unsafe(r.value) + _tmp_dir_inited = true + } + return _tmp_dir +} + + +[finalize] +def cleanup_temp_dir { + if (_tmp_dir_inited && !empty(_tmp_dir)) { + rmdir_rec(_tmp_dir) + } +} + + +def private plant(dir, name, content : string) { + fopen(path_join(dir, name), "wb") $(f) { + if (f != null) { + fwrite(f, content) + } + } +} + + +// One folder: its own .lint_config (skipped when `config` is empty), a lint-clean .das so +// the run has something to lint, and the .md files the folder carries. +def private plant_folder(root, name, config : string; docs : array<string>) : string { + let dir = path_join(root, name) + mkdir_rec(dir) + if (!empty(config)) { + plant(dir, ".lint_config", config) + } + plant(dir, "ok.das", "options gen2\n\n[export]\ndef main \{\n print(\"ok\\n\")\n\}\n") + for (doc in docs) { + plant(dir, doc, "# {doc}\n") + } + return dir +} + + +def private build_tree() : string { + let root = path_join(tmp_root(), "tree") + mkdir_rec(root) + plant_folder(root, "tagged", "[docs]\nrule_docs_only = true\n", + ["notes.md", "REVIEW.md", "REVIEW_LINQ.md", "ARCHITECTURE.md", "ARCHITECTURE_DEDUP.md", "LAWS.md"]) + plant_folder(root, "off", "[docs]\nrule_docs_only = false\n", ["notes.md"]) + plant_folder(root, "plain", "", ["notes.md"]) + return root +} + + +// argv[0] is the running interpreter - dastest itself is spawned as +// `daslang(.exe) dastest/dastest.das ...`, so it names the binary to spawn. +def private das_exe() : string { + let args <- get_command_line_arguments() + return empty(args) ? "" : args[0] +} + + +def private run_lint(root : string; var output : string&) : int { + let argv <- [das_exe(), "-dasroot", get_das_root(), "{get_das_root()}/utils/lint/main.das", "--", "-q", root] + return unsafe(popen_argv(argv, 0.0, $(f) { + if (f != null) { + output = fread(f) + } + })) +} + + +def private count_of(text, sub : string) : int { + var n = 0 + var from = 0 + while (true) { + let at = find(text, sub, from) + break if (at < 0) + n ++ + from = at + length(sub) + } + return n +} + + +[test] +def test_rule_docs_only_folder(t : T?) { + t |> run("a stray .md is reported once, and only in the tagged folder") @(t : T?) { + let root = build_tree() + var out : string + let rc = run_lint(root, out) + t |> equal(count_of(out, "LINT025"), 1, "exactly one stray .md in the planted tree\n{out}") + t |> success(find(out, "tagged/notes.md: LINT025: stray .md beside shipped code") >= 0, + "the finding names the tagged folder's stray\n{out}") + t |> success(find(out, "off/notes.md") < 0, "rule_docs_only = false reports nothing\n{out}") + t |> success(find(out, "plain/notes.md") < 0, "an untagged folder reports nothing\n{out}") + t |> equal(rc, 2, "a finding sets the issue exit code\n{out}") + } + t |> run("the rule documents themselves are never findings") @(t : T?) { + let root = build_tree() + var out : string + run_lint(root, out) + for (doc in ["REVIEW.md", "REVIEW_LINQ.md", "ARCHITECTURE.md", "ARCHITECTURE_DEDUP.md", "LAWS.md"]) { + t |> success(find(out, "/{doc}: LINT025") < 0, "{doc} is a rule document\n{out}") + } + } + t |> run("--disable LINT025 silences the pass") @(t : T?) { + let root = build_tree() + var out : string + let argv <- [das_exe(), "-dasroot", get_das_root(), "{get_das_root()}/utils/lint/main.das", + "--", "-q", "--disable", "LINT025", root] + let rc = unsafe(popen_argv(argv, 0.0, $(f) { + if (f != null) { + out = fread(f) + } + })) + t |> equal(count_of(out, "LINT025"), 0, "the disabled rule reports nothing\n{out}") + t |> equal(rc, 0, "and the run is clean\n{out}") + } +} From c8b36a4d96782ddee6de5e88ba3122124b8118f3 Mon Sep 17 00:00:00 2001 From: Boris Batkin <bbatkin@gmail.com> Date: Thu, 27 Aug 2026 05:18:29 -0700 Subject: [PATCH 12/15] LINT025 runs for file positionals and das-free folders External review caught two invocation gaps: a file positional never armed its folder (the PR lint shape passes changed files one by one), and a tagged folder holding only a stray .md errored "no .das files found" before the folder pass could rule. A file positional now arms its parent, and directory roots proceed to the docs pass with an empty source set. One test arm per gap. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014fKJGuUL8tP58Y6NvoVeSn --- utils/lint/main.das | 14 ++++++++++---- utils/lint/tests/lint025_rule_docs_only.das | 20 ++++++++++++++++++++ 2 files changed, 30 insertions(+), 4 deletions(-) diff --git a/utils/lint/main.das b/utils/lint/main.das index 4cfe489f70..ebf28e02b5 100644 --- a/utils/lint/main.das +++ b/utils/lint/main.das @@ -625,8 +625,9 @@ def resolve_rule_filters(cfg : Config; var disabled_codes : table<string>&; var return true } -// Driver-mode positionals → sorted .das file list, plus the directory roots among them -// (`dirs`, the folder-docs pass's scan roots). Returns a process exit code, or -1 to proceed. +// Driver-mode positionals → sorted .das file list, plus the folder-docs pass's scan roots +// (`dirs`: directory positionals, and each file positional's folder). Returns a process +// exit code, or -1 to proceed. def collect_input_files(var files : array<string>&; var dirs : array<string>&) : int { let cli_args <- get_user_args() let paths <- [for (arg in cli_args); arg; where !starts_with(arg, "-")] @@ -645,13 +646,18 @@ def collect_input_files(var files : array<string>&; var dirs : array<string>&) : let st = stat(target) if (st.is_valid && st.is_dir) { dirs |> push(target) + } elif (st.is_valid) { + // a file positional arms its folder too - the PR lint shape passes changed + // files one by one, and LINT025 must not go silent there + dirs |> push(dir_name(target)) } scan_das_files(target, files, cache, skipped) } - if (empty(files)) { + if (empty(files) && empty(dirs)) { // All inputs were intentionally skipped (e.g. builtin.das) -- success. // Distinguishes a clean "nothing to lint" from a genuine "no .das at the - // given path", which still errors. + // given path", which still errors. A directory root proceeds even with no + // .das under it: the folder-docs pass still owes its verdict. return 0 if (skipped > 0) print("Error: no .das files found\n") return 1 diff --git a/utils/lint/tests/lint025_rule_docs_only.das b/utils/lint/tests/lint025_rule_docs_only.das index aca8f88af2..abe773339f 100644 --- a/utils/lint/tests/lint025_rule_docs_only.das +++ b/utils/lint/tests/lint025_rule_docs_only.das @@ -130,6 +130,26 @@ def test_rule_docs_only_folder(t : T?) { t |> success(find(out, "/{doc}: LINT025") < 0, "{doc} is a rule document\n{out}") } } + t |> run("a file positional arms its folder - the PR lint shape") @(t : T?) { + let root = build_tree() + var out : string + let rc = run_lint(path_join(path_join(root, "tagged"), "ok.das"), out) + t |> equal(count_of(out, "LINT025"), 1, "linting one file still checks its folder\n{out}") + t |> success(find(out, "tagged/notes.md: LINT025") >= 0, "the finding names the stray\n{out}") + t |> equal(rc, 2, "a finding sets the issue exit code\n{out}") + } + t |> run("a tagged folder with no .das still gets its verdict") @(t : T?) { + let root = build_tree() + let dir = path_join(root, "mdonly") + mkdir_rec(dir) + plant(dir, ".lint_config", "[docs]\nrule_docs_only = true\n") + plant(dir, "stray.md", "# stray\n") + var out : string + let rc = run_lint(dir, out) + t |> success(find(out, "no .das files found") < 0, "an armed folder is not an empty input\n{out}") + t |> success(find(out, "mdonly/stray.md: LINT025") >= 0, "the finding names the stray\n{out}") + t |> equal(rc, 2, "a finding sets the issue exit code\n{out}") + } t |> run("--disable LINT025 silences the pass") @(t : T?) { let root = build_tree() var out : string From e638f4a415b5f96b67bcc3b3ce410708f7eed8d3 Mon Sep 17 00:00:00 2001 From: Boris Batkin <bbatkin@gmail.com> Date: Thu, 27 Aug 2026 05:29:02 -0700 Subject: [PATCH 13/15] audit round fixes: LINT025 joins the doc triple; config-tree leak closed The review-md auditors' findings applied. LINT025 gets its lint.rst section, and utils/lint/main.das joins the gate's RULE_MODULES so the fixture-and-rst triple binds the runner's ids too; the checklist now binds both directions (shrinking the scanned set, or emitting outside it). The .lint_config readers no longer leak the parsed TOML tree - a callback shape frees the root before returning, so no borrowed view can escape; covered by the existing cascade pins plus a no-cascade arm. The ci weakening rule states the property instead of three cases; daslib gains the dead-id-residue duty. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014fKJGuUL8tP58Y6NvoVeSn --- ARCHITECTURE_COMMON.md | 3 +- ci/REVIEW.md | 6 +-- ci/smoke_test_bundle.sh | 2 +- daslib/REVIEW.md | 5 ++ daslib/lint_config.das | 52 +++++++++++---------- doc/reflections/das2rst.das | 2 +- doc/source/reference/language/lint.rst | 18 +++++++ utils/lint/REVIEW.das | 2 +- utils/lint/REVIEW.md | 19 ++++---- utils/lint/tests/lint025_rule_docs_only.das | 9 ++++ 10 files changed, 77 insertions(+), 41 deletions(-) diff --git a/ARCHITECTURE_COMMON.md b/ARCHITECTURE_COMMON.md index 9d4965a4f5..34683c7fd4 100644 --- a/ARCHITECTURE_COMMON.md +++ b/ARCHITECTURE_COMMON.md @@ -9,8 +9,7 @@ sibling `REVIEW_COMMON.md` (repo root) is the checklists' contract. why that shape wins.** The mood test: a sentence that binds a diff with must/never/"is a defect" is a rule - move it to the folder's `REVIEW.md`. A passage that narrates what happened - past tense, or dated as a record of an event - is history - archive it to -`/history` (`skills/internal/doc_archiving.md`); a `probe-verified <date>` tag or a version -stamp records no event and stays. A promise, phase, or plan is planned work - move it to the +`/history`; a `probe-verified <date>` tag or a version stamp records no event and stays. A promise, phase, or plan is planned work - move it to the folder's follow-up ledger. **What belongs in an architecture doc: file charters, mechanisms, definitions, inventories, diff --git a/ci/REVIEW.md b/ci/REVIEW.md index 9ac90749e7..3bc73d5c3a 100644 --- a/ci/REVIEW.md +++ b/ci/REVIEW.md @@ -3,6 +3,6 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `CLAUDE.md` (repo root). -**A diff that weakens `smoke_test_bundle.sh` - the script that checks a built release bundle - -is a defect.** Weakening means dropping a check, loosening an assertion, or shortening a list -the script walks. +**A diff that shrinks what `smoke_test_bundle.sh` - the script that checks a built release +bundle - rejects is a defect**: every bundle it failed before the diff still fails. A new +exemption may name only a file no existing check matched. diff --git a/ci/smoke_test_bundle.sh b/ci/smoke_test_bundle.sh index 7ae788c463..22a9bf8a1d 100644 --- a/ci/smoke_test_bundle.sh +++ b/ci/smoke_test_bundle.sh @@ -241,7 +241,7 @@ if [[ -z "$REVIEW_LEAKS" ]]; then echo "OK" PASS=$((PASS + 1)) else - echo "FAIL (internal review files in bundle):" + echo "FAIL (internal rule/record docs in bundle):" echo "$REVIEW_LEAKS" | sed 's/^/ /' FAIL=$((FAIL + 1)) fi diff --git a/daslib/REVIEW.md b/daslib/REVIEW.md index 4cdf836b1f..69f61521dc 100644 --- a/daslib/REVIEW.md +++ b/daslib/REVIEW.md @@ -70,6 +70,11 @@ the formatter's kept set. in the same change.** An unlisted module's ids are never scanned, so its fixture-and-rst check never runs. +**A diff that deletes a rule id's last emitting spelling from a rule module also removes that +id's fixture and its `doc/source/reference/language/lint.rst` (repo root) section, in the same +change** - the gate checks only ids it can still find, so a dead id's residue outlives it +silently. + **Never record a LINT010 store in `preVisitExprVar` - record it in the POST-visit of `ExprCopy`/`ExprClone`/`ExprMove`.** The LHS's variable fires before the RHS is walked, so an early record lets the RHS's own read clear the store it just recorded. diff --git a/daslib/lint_config.das b/daslib/lint_config.das index e177f79ad3..051fed19e3 100644 --- a/daslib/lint_config.das +++ b/daslib/lint_config.das @@ -262,38 +262,41 @@ struct public FormatPolicy { force_clean_comments : bool = false } -def private read_config_table(path, table_name : string) : JsonValue? { +// Parses `path`, runs `blk` over its [table_name] table when present, then frees the +// parsed tree - read_toml mints it, and no view may outlive this scope. +def private with_config_table(path, table_name : string; blk : block<(var tbl : JsonValue?) : void>) { let body = fread(path) - if (empty(body)) return null + return if (empty(body)) var error : string var root : JsonValue? = read_toml(body, error) - if (root == null || !(root.value is _object)) return null - var root_obj & = unsafe(root.value as _object) - if (!key_exists(root_obj, table_name)) return null - var tbl_jv : JsonValue? = root_obj[table_name] - if (tbl_jv == null || !(tbl_jv.value is _object)) return null - return tbl_jv -} - -def private read_format_table(path : string) : JsonValue? { - return read_config_table(path, "format") -} - -def private read_docs_table(path : string) : JsonValue? { - return read_config_table(path, "docs") + return if (root == null) + if (root.value is _object) { + var root_obj & = unsafe(root.value as _object) + if (key_exists(root_obj, table_name)) { + var tbl_jv : JsonValue? = root_obj[table_name] + if (tbl_jv != null && tbl_jv.value is _object) { + blk |> invoke(tbl_jv) + } + } + } + unsafe { + delete root + } } //! True when the ``.lint_config`` at `config_path` sets ``[docs] rule_docs_only = true`` — the folder //! carries rule documents only (``REVIEW*.md`` / ``ARCHITECTURE*.md`` / ``LAWS.md``). A folder property, //! NOT the ``[format]`` cascade: only that one file is read, and anything missing reads as false. def public rule_docs_only_at(config_path : string) : bool { - var docs_jv = read_docs_table(config_path) - return false if (docs_jv == null) - var docs & = unsafe(docs_jv.value as _object) - return false if (!key_exists(docs, "rule_docs_only")) - var val_jv : JsonValue? = docs["rule_docs_only"] - return false if (val_jv == null || !(val_jv.value is _bool)) - return val_jv.value as _bool + var result = false + with_config_table(config_path, "docs") $(var docs_jv) { + var docs & = unsafe(docs_jv.value as _object) + return if (!key_exists(docs, "rule_docs_only")) + var val_jv : JsonValue? = docs["rule_docs_only"] + return if (val_jv == null || !(val_jv.value is _bool)) + result = val_jv.value as _bool + } + return result } def private config_dir_of(file : string) : string { @@ -350,8 +353,7 @@ def public format_policy_for(file : string) : FormatPolicy { var dir = config_dir_of(file) while (true) { let cfg = dir == "." ? ".lint_config" : "{dir}/.lint_config" - var fmt_jv = read_format_table(cfg) - if (fmt_jv != null) { + with_config_table(cfg, "format") $(var fmt_jv) { take_format_key(fmt_jv, "format_enabled", pol.format_enabled, enabled_decided) take_format_key(fmt_jv, "force_clean_comments", pol.force_clean_comments, clean_decided) } diff --git a/doc/reflections/das2rst.das b/doc/reflections/das2rst.das index 1cd5cb4b1d..9f90b0089a 100644 --- a/doc/reflections/das2rst.das +++ b/doc/reflections/das2rst.das @@ -1364,7 +1364,7 @@ def document_module_lint(_root : string) { def document_module_lint_config(_root : string) { var mod = find_module("lint_config") var groups <- array<DocGroup>( - group_by_regex("Configuration", mod, %regex~(load_lint_config|load_lint_config_from_path|load_env_disabled|seed_default_disabled|build_lint_macro_disabled|lint_config_forces_on)$%%), + group_by_regex("Configuration", mod, %regex~(load_lint_config|load_lint_config_from_path|load_env_disabled|seed_default_disabled|build_lint_macro_disabled|lint_config_forces_on|rule_docs_only_at)$%%), group_by_regex("Path excludes", mod, %regex~(load_path_excludes_from_path|matches_path_excludes|is_lint_path_excluded)$%%), group_by_regex("Path-based rule defaults", mod, %regex~(is_daslib_source|is_shipped_library_source|is_core_library_source)$%%), group_by_regex("Lint-surface predicates", mod, %regex~(is_user_authored_body|is_lint_fixture_name|lint_file_skip_reason)$%%), diff --git a/doc/source/reference/language/lint.rst b/doc/source/reference/language/lint.rst index 57df7906f5..0708d90167 100644 --- a/doc/source/reference/language/lint.rst +++ b/doc/source/reference/language/lint.rst @@ -757,6 +757,24 @@ its signal by staying silent there — widen those by hand where they are byte counts. ``int64(a + b)`` has no product and is not this rule's shape; an operand that is already 64-bit is silent. +LINT025 — stray ``.md`` beside shipped code +=========================================== + +A folder rule, not a code rule: a folder whose ``.lint_config`` carries +``[docs] rule_docs_only = true`` may hold only rule documents — +``REVIEW*.md``, ``ARCHITECTURE*.md``, ``LAWS.md``. Any other ``.md`` there +fires. Ledgers, plans, and notes live at the module root or in ``plans/`` +(repo root), where they cannot ride an install glob into the SDK or bury the +documents a reviewer must read. The pass runs once per lint invocation over +the directories being linted and the folders of the files being linted; it +never fires in an untagged folder. + +.. code-block:: toml + + # .lint_config beside the shipped sources + [docs] + rule_docs_only = true + LINT018 — narrowed size argument of a call with a 64-bit overload ================================================================== diff --git a/utils/lint/REVIEW.das b/utils/lint/REVIEW.das index 18306dc09f..e7c2097e1b 100644 --- a/utils/lint/REVIEW.das +++ b/utils/lint/REVIEW.das @@ -11,7 +11,7 @@ require dastest/review_gate // Run from the repo root: bin/daslang utils/lint/REVIEW.das - exit 0 clean, 1 with findings. // Every rule id a lint module emits has a fixture and a lint.rst section. -let RULE_MODULES <- [ "daslib/lint.das", "daslib/perf_lint.das", "daslib/style_lint.das", "daslib/dupe_detect.das" ] +let RULE_MODULES <- [ "daslib/lint.das", "daslib/perf_lint.das", "daslib/style_lint.das", "daslib/dupe_detect.das", "utils/lint/main.das" ] let RST = "doc/source/reference/language/lint.rst" let FIXTURE_DIRS <- [ "utils/lint/tests", "tests/lint" ] // fixtures whose file name does not carry the id (README.md lists them) diff --git a/utils/lint/REVIEW.md b/utils/lint/REVIEW.md index 6297fe9f9e..2ce292db35 100644 --- a/utils/lint/REVIEW.md +++ b/utils/lint/REVIEW.md @@ -3,11 +3,14 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `README.md`. -**A diff that removes `lint` from `DAS_UTILS` or `DAS_UTILS_SHIPPED_EXES` -(`utils/CMakeLists.txt`) is a defect** - dropping either entry breaks the release bundle. - -**A diff that drops a rule id from the set `REVIEW.das` (beside this file) finds - by -editing the gate, or by removing the id's only spelling the gate can find in the source - -is a defect.** The gate checks that every rule id a lint module emits has a fixture and a -section in `doc/source/reference/language/lint.rst` (repo root), so a dropped id retires -that check with no warning. +**A diff that removes `lint` from `DAS_UTILS_SHIPPED_EXES` in `utils/CMakeLists.txt` is a +defect.** The removal drops the prebuilt lint exe from the release bundle, and the gate +cannot see a removed entry. + +**A diff that shrinks the set of rule ids `REVIEW.das` (beside this file) scans is a +defect** - whether by editing the gate or by deleting an id's last scannable spelling. The +gate checks that every emitted rule id has a fixture and a `doc/source/reference/language/lint.rst` +(repo root) section, so a shrunk set retires those checks with no warning. + +**A diff that makes a file emit a rule id the gate `REVIEW.das` (beside this file) does not +scan adds that file to the gate's module set, in the same change.** diff --git a/utils/lint/tests/lint025_rule_docs_only.das b/utils/lint/tests/lint025_rule_docs_only.das index abe773339f..9ff6c9d8b3 100644 --- a/utils/lint/tests/lint025_rule_docs_only.das +++ b/utils/lint/tests/lint025_rule_docs_only.das @@ -138,6 +138,15 @@ def test_rule_docs_only_folder(t : T?) { t |> success(find(out, "tagged/notes.md: LINT025") >= 0, "the finding names the stray\n{out}") t |> equal(rc, 2, "a finding sets the issue exit code\n{out}") } + t |> run("the tag is a folder property - a subfolder does not inherit it") @(t : T?) { + let root = build_tree() + let nested = path_join(path_join(root, "tagged"), "nested") + mkdir_rec(nested) + plant(nested, "stray.md", "# stray\n") + var out : string + run_lint(root, out) + t |> success(find(out, "nested/stray.md") < 0, "no cascade into an untagged subfolder\n{out}") + } t |> run("a tagged folder with no .das still gets its verdict") @(t : T?) { let root = build_tree() let dir = path_join(root, "mdonly") From 542834c462c8a012c73e133cbb2456a7ae3768dc Mon Sep 17 00:00:00 2001 From: Boris Batkin <bbatkin@gmail.com> Date: Thu, 27 Aug 2026 05:35:25 -0700 Subject: [PATCH 14/15] tdd audit fixes: nano ARCHITECTURE.md stops shipping; five branches pinned The audit's negative controls found the extended install gate red on its own branch - nano's directory install shipped ARCHITECTURE.md, so the gate's own ban fired at install time. The exclusion joins REVIEW.md in the nano block; a fresh-configure install now passes the gate and the bundle smoke's two new checks on a real bundle. The untested branches got arms: overlapping-root dedup, the _-prefixed prune, non-bool rule_docs_only values, --enable whitelist mode; the dead .md guard in is_rule_doc_name is gone; the test split satisfies STYLE038. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014fKJGuUL8tP58Y6NvoVeSn --- CMakeLists.txt | 7 +-- utils/lint/main.das | 1 - utils/lint/tests/lint025_rule_docs_only.das | 49 +++++++++++++++++++++ 3 files changed, 53 insertions(+), 4 deletions(-) diff --git a/CMakeLists.txt b/CMakeLists.txt index 9349542488..f0162b6180 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -1822,10 +1822,11 @@ install(DIRECTORY ${PROJECT_SOURCE_DIR}/nano/ PATTERN "*.cpp" PATTERN "*.md" PATTERN "CMakeLists.txt" - # REVIEW.md reviews a change to this repo's copy of nano; an SDK carries - # the library, not the process of changing it. (REVIEW.das is not matched - # by any pattern above, so it never ships either.) + # REVIEW.md and ARCHITECTURE.md review and describe changes to this repo's + # copy of nano; an SDK carries the library, not the process of changing it. + # (REVIEW.das is not matched by any pattern above, so it never ships either.) REGEX "/REVIEW\\.md$" EXCLUDE + REGEX "/ARCHITECTURE\\.md$" EXCLUDE ) # The shared runtime sources nano compiles. They live under src/, which the SDK # otherwise does not carry; nano/CMakeLists.txt names them one by one, and diff --git a/utils/lint/main.das b/utils/lint/main.das index ebf28e02b5..3cb9a223cd 100644 --- a/utils/lint/main.das +++ b/utils/lint/main.das @@ -179,7 +179,6 @@ def scan_das_files(path : string; var files : array<string>; var cache : table<s // checklists (REVIEW.md and its routed companions), its architecture doc (and companion // splits), and the rulings sidecar. Everything else is a ledger, a plan or a note. def is_rule_doc_name(name : string) : bool { - return false if (!(name |> ends_with(".md"))) return (name == "REVIEW.md" || name == "ARCHITECTURE.md" || name == "LAWS.md" || name |> starts_with("REVIEW_") || name |> starts_with("ARCHITECTURE_")) } diff --git a/utils/lint/tests/lint025_rule_docs_only.das b/utils/lint/tests/lint025_rule_docs_only.das index 9ff6c9d8b3..fa5854502f 100644 --- a/utils/lint/tests/lint025_rule_docs_only.das +++ b/utils/lint/tests/lint025_rule_docs_only.das @@ -159,6 +159,55 @@ def test_rule_docs_only_folder(t : T?) { t |> success(find(out, "mdonly/stray.md: LINT025") >= 0, "the finding names the stray\n{out}") t |> equal(rc, 2, "a finding sets the issue exit code\n{out}") } +} + + +[test] +def test_rule_docs_only_modes(t : T?) { + t |> run("overlapping scan roots report each stray once") @(t : T?) { + let root = build_tree() + var out : string + let argv <- [das_exe(), "-dasroot", get_das_root(), "{get_das_root()}/utils/lint/main.das", + "--", "-q", root, path_join(root, "tagged")] + unsafe(popen_argv(argv, 0.0, $(f) { + if (f != null) { + out = fread(f) + } + })) + t |> equal(count_of(out, "tagged/notes.md"), 1, "the seen set dedups overlapping roots\n{out}") + } + t |> run("a tagged folder under a skipped directory is not scanned") @(t : T?) { + let root = build_tree() + let hidden = path_join(path_join(root, "_gen"), "inner") + mkdir_rec(hidden) + plant(hidden, ".lint_config", "[docs]\nrule_docs_only = true\n") + plant(hidden, "stray.md", "# stray\n") + var out : string + run_lint(root, out) + t |> success(find(out, "inner/stray.md") < 0, "the walk prunes _-prefixed dirs like the das scan\n{out}") + } + t |> run("a non-bool rule_docs_only value reads as false") @(t : T?) { + let root = build_tree() + let dir = path_join(root, "strval") + mkdir_rec(dir) + plant(dir, ".lint_config", "[docs]\nrule_docs_only = \"true\"\n") + plant(dir, "stray.md", "# stray\n") + var out : string + run_lint(root, out) + t |> success(find(out, "strval/stray.md") < 0, "only a bool true arms the folder\n{out}") + } + t |> run("--enable whitelist mode silences an unlisted LINT025") @(t : T?) { + let root = build_tree() + var out : string + let argv <- [das_exe(), "-dasroot", get_das_root(), "{get_das_root()}/utils/lint/main.das", + "--", "-q", "--enable", "LINT001", root] + unsafe(popen_argv(argv, 0.0, $(f) { + if (f != null) { + out = fread(f) + } + })) + t |> equal(count_of(out, "LINT025"), 0, "whitelist mode without LINT025 reports nothing\n{out}") + } t |> run("--disable LINT025 silences the pass") @(t : T?) { let root = build_tree() var out : string From 3981910d4e5b805ef7b13698469e40e1f01f1cc4 Mon Sep 17 00:00:00 2001 From: Boris Batkin <bbatkin@gmail.com> Date: Thu, 27 Aug 2026 05:46:37 -0700 Subject: [PATCH 15/15] bundle smoke: restore survives a concurrently recreated lib/ The hide/restore trap moved lib/ back with mv; a build recreating lib/ meanwhile made that a nesting move and left the tree without its dylibs. The restore now merges into an existing lib/. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014fKJGuUL8tP58Y6NvoVeSn --- ci/smoke_test_bundle.sh | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/ci/smoke_test_bundle.sh b/ci/smoke_test_bundle.sh index 22a9bf8a1d..b9d1873114 100644 --- a/ci/smoke_test_bundle.sh +++ b/ci/smoke_test_bundle.sh @@ -138,7 +138,15 @@ if [[ -z "$CPP_SUFFIX" && -d "$BUILD_LIB" ]]; then fi restore_and_clean() { rm -f "$LOG" "$LOG.suite" "$LOG.lint" - if [[ -n "$HIDDEN_LIB" && -d "$HIDDEN_LIB" ]]; then mv "$HIDDEN_LIB" "$BUILD_LIB"; fi + if [[ -n "$HIDDEN_LIB" && -d "$HIDDEN_LIB" ]]; then + # A concurrent build may have recreated lib/ meanwhile; mv into an existing + # dir would NEST the hidden copy instead of restoring it - merge then. + if [[ -e "$BUILD_LIB" ]]; then + cp -a "$HIDDEN_LIB"/. "$BUILD_LIB"/ && rm -rf "$HIDDEN_LIB" + else + mv "$HIDDEN_LIB" "$BUILD_LIB" + fi + fi } trap restore_and_clean EXIT