diff --git a/docs/developers/port-allocation.md b/docs/developers/port-allocation.md index 281095fe..962fd3e6 100644 --- a/docs/developers/port-allocation.md +++ b/docs/developers/port-allocation.md @@ -114,6 +114,126 @@ These bundles use `network_mode: host`. They consume whatever ports their upstre - `tailscale` (MagicDNS, peer connections) - `crowdsec-firewall-bouncer` (deferred to PR 4.5 — upstream does not publish a Docker image; needs a custom Dockerfile and a tested unwind command verified on a throwaway host) — will need host network to manipulate iptables/nftables +## Second host: raven + +Until 2026-09-09 this registry described one machine. The two-host production design +(`docs/superpowers/specs/2026-09-09-two-host-production-and-heavy-model-modes.md`) makes **raven** a production +host, so ports now need a host qualifier to mean anything. + +Raven's ports live in their own namespace and do not collide with crow's. The first column below is deliberately +`raven:` rather than a bare number, because `scripts/check-port-allocation.js` reads bare numbers in the +first cell as **crow** allocations. Keeping raven rows unparseable to it is correct today and is a stopgap: +**making the checker host-aware is follow-up work**, and until it lands, a raven port is only verified by looking +at raven. + +### Production and reserved (raven) + +| port | bind | what | status | +|---|---|---|---| +| raven:8030 | 0.0.0.0 | Qwen3.8-Flash-Next @1M, production (native systemd, not a container) | planned | +| raven:8031 | 0.0.0.0 | Flash-Next two-box master (window mode) | reserved | +| raven:8032 | 0.0.0.0 | DeepSeek-V4-Flash two-box master (window mode) | reserved | +| raven:8033 | 0.0.0.0 | GLM-5.3-Flash two-box master (window mode) | reserved | + +### Benchmark-transient, held only inside a window + +Bound by `pi-lab/scripts/`, never by a compose file, so `check-port-allocation.js` cannot see any of them. Listed +because a registry that covers a host while omitting ports in regular use is worse than one that omits the host. + +| port | bind | what | +|---|---|---| +| raven:8021 | 127.0.0.1 | **two-box master port**, the standard across R1 to R26 (25 `raven-*` scripts). Thursday's W0 binds it | +| raven:8035 | 127.0.0.1 | R24 result-check server | +| raven:8036 | 127.0.0.1 | R25 / R25b single-box stack runs. This is the Flash-Next config chosen for production, so 8030 and 8036 are the same shape on different ports | +| raven:8037 | 127.0.0.1 | R23 knob-sanity (`KS_PORT`), moved here off 8031 when this section reserved that range | +| raven:8098 | 0.0.0.0 | zoo arm endpoint, two-box master and single-box arms (`ZOO_PORT`) | +| raven:8099 | 127.0.0.1 | Q4 MTP smoke | +| raven:50052 | 10.99.0.2 (USB4) | `ggml-rpc-server` when **raven** is the worker | +| raven:50053 | 127.0.0.1 | second `ggml-rpc-server`, for arms running two workers on raven's one GPU | +| crow:8012, 8013 | 127.0.0.1 | GLM-5.2 IQ2 and IQ4 windows (`glm52-window.sh` and friends) | +| crow:8020 | 127.0.0.1 | DeepSeek-V4-Flash windowed serve (`dsv4-window.sh`). The `crow-dsv4` provider row points here | +| crow:8021 | 127.0.0.1 | phase-0 Vulkan probes, run as duties under `dsv4-window.sh` (`PHASE0_PORT`) | +| crow:8022, 8024, 8025, 8026, 8027 | 127.0.0.1 | DSv4 Vulkan hyper-connection investigation duties (`dsv4-vk-hc-*`, `dsv4-ubatch-confirm`) | +| crow:8023 | 127.0.0.1 | DSv4 top-k A/B and HC verify (`TOPK_PORT`, `HCV_PORT`) | +| crow:8099 | 127.0.0.1 | GTT ladder duty under `dsv4-window --duty` | + +### Three ports bind on BOTH machines + +`8021`, `8099` and `50052` each mean two different things depending on the box. 8021 and 8099 are the same number +on the same loopback on two hosts, so nothing about the number distinguishes them; 50052 at least differs by bind +address (`10.99.0.1` is crow, `10.99.0.2` is raven). This is the whole argument for the `raven:`/`crow:` prefix, +and it is why the prefix is worth keeping even after the checker learns host-awareness. A bare number here is not +an allocation, it is an ambiguity. + +Verified free on raven 2026-09-09: 8030 through 8033, 8035, 8036, 8037, 8098 and 8099. Raven listens only on 22, +53, 631 and two ephemeral ports. + +### A constraint this registry cannot express + +**Raven cannot host 8030 and 8036 at the same time, however free both ports are.** They are the same Flash-Next +single-box config, one as the proposed production port and one as the R25/R25b benchmark port, and each wants +about 92.6 GiB on a 124 GiB box. Two free ports, one machine's worth of memory. Allocating a port is not the same +as being able to run the thing, and no port table can say so. + +Read it as the general case rather than one awkward pair: on a single-tenant box, port availability is a necessary +condition and never a sufficient one. Same class as the ordering constraint in +`docs/superpowers/specs/2026-09-09-two-host-production-and-heavy-model-modes.md` §3.0, where the port would have +been free and the arm would still have been unsafe. + +### Three gaps this section exposes, all worth closing + +1. **Crow's own model ports are largely unlisted.** 8003 (35b), 8006 (27b solo) and 8014 (27b 512k) are absent + from the allocation table; only 8010 (27b copilot) is recorded. They bind the tailnet IP from `crow-addons/` + composes, which the conventions above already flag as a separate registry, so nothing catches them. +2. **Crow's own benchmark ports were unlisted too**, and are now in the table above rather than the main one, + because they are script-bound and transient. There are eleven of them. +3. **8098 is ambiguous across hosts.** The table allocates it to searxng on crow's loopback, while the two-box + benchmark zoo arms conventionally use 8098 on raven. Both are correct today because they are different + machines, and neither the table nor the checker can say so. This is precisely the class of silent + double-allocation the conventions section warns about, one host further out. + +### How this list was built, because the method outlasts the list + +Four passes each found more, and the last two were humans reading their own trees: + +1. `PORT=` assignments: four ports. +2. Adding `--port` and `host:port` forms: five more (8022, 8024 through 8027). +3. Adding the shell-default form `${VAR:-NNNN}`: this is the easiest to miss, because a scan for `PORT=8021`, + `port 8021` or `:8021` returns **nothing** on a file that binds 8021 all day via `PORT="${PHASE0_PORT:-8021}"`. + That form is also multiplying, since parameterising a hardcoded port is the right fix for the ambiguities above + and creates a new hiding place every time. +4. Two people reading their own trees: four more ports, one wrong host attribution, and one live collision inside + a range this section had just reserved. + +**Treat any scan of this kind as a lower bound.** If the checker grows host-awareness, its companion should read +invocations and shell defaults as well as assignments. + +### The failure mode behind every mistake made here + +Building this section produced four errors, and not one was a missing grep: + +1. A trailing comment on a shell assignment line swallowed the assignments after it. `bash -n` passed; it would + have failed at first use under `set -u`. +2. A categorical negative, "not one script on that host uses it", was asserted from a pattern that could not have + matched the files in question. +3. A string replace reported success without applying, because its pattern did not match the file's line + wrapping. Every other edit in the same script worked, so the run looked clean. +4. A grep confirming a correct edit returned zero, because the grep was single-line and the text wrapped across a + newline. Re-probing whitespace-flattened text found it. + +All four are one failure: **the check and the thing checked disagree about shape, and the check returns the +reassuring answer.** The fourth is the most dangerous, because it makes a correct edit look failed, and the +natural response is to apply it again. + +Three habits follow, and they are cheap. Assert the match count before substituting, so a no-op is loud rather +than silent. Verify against text normalised the same way the edit was written, since a single-line probe cannot +see a wrapped phrase. And never assert a categorical negative from a pattern-based scan; the honest form is "this +pattern found none", which is a different claim. + +A related trap, from the same evening: a scan **surfaced** a colliding port and its author read the hit as +confirming their own reservation rather than as a conflict. That turns evidence into confirmation, and it is worse +than missing the port outright. + ## Process for amending this file 1. Pick an unallocated port in a sensible range (admin UIs in 3000-3099, backend APIs in 8000-8099, metrics in 19000-19999). diff --git a/docs/superpowers/specs/2026-09-09-two-host-production-and-heavy-model-modes.md b/docs/superpowers/specs/2026-09-09-two-host-production-and-heavy-model-modes.md new file mode 100644 index 00000000..3170adde --- /dev/null +++ b/docs/superpowers/specs/2026-09-09-two-host-production-and-heavy-model-modes.md @@ -0,0 +1,357 @@ +# Spec: two-host production, and heavy-model modes on top of it + +Status: draft for review. Author: `~/r4-tehcy` session, 2026-09-09 evening, from Kevin's direction across +three sessions (`~/r4-tehcy`, `pi-lab-36`, `crow-34`). +Implementation owner: crow (format, orchestrator routing, catalog). Harness changes: pi-lab. First consumer: R4. + +## 1. The configuration Kevin chose + +Verbatim: run the normal production bots on crow, run Qwen3.8-Flash-Next at 1M on raven as normal production, +with the option to evict production on both boxes to run either GLM or DSv4. That is the standard configuration. + +So there are three states, and only three: + +| state | crow | raven | +|---|---|---| +| **standard** | prod bots (35b, vLLM 4b, embed, gemma) | Flash-Next Q4_K_XL @1M | +| **heavy: GLM** | evicted | evicted, GLM-5.3-Flash two-box master | +| **heavy: DSv4** | evicted, RPC worker | evicted, DSv4-Flash two-box master | + +The standard state is the important one: both a fast runner and a 1M-context model are available at the same +time, because they sit on different boxes. No eviction, no RPC, no shared GPU. + +## 2. Why this works, with the numbers + +Per-box GTT is 124 GiB on both machines. Measured, from +`docs/research/2026-08-30-second-strix-halo-box-plan.md` and the serving doc +`~/pi-lab/docs/research/2026-09-09-heavy-model-serving-configs.md`: + +- **Flash-Next Q4_K_XL fits 1M on one box.** Body 76.8 GiB plus 26.9 GiB lazy PLE, about 106 GiB resident at 1M, + leaving **17.7 GiB free on raven**. Rungs measured on raven: 262k leaves 31.0 GiB free, 524k 23.4, 1M 17.7. +- **Single-box beats two-box for this model.** Raven single-box at 120 W measures 354 / 24.7 short and + 234 / 18.4 at depth, ahead of the two-box split on both prefill and decode (§7.16). Two-box earns its keep only + as headroom for the Q5 quant or to host the MTP head on the master. So the 1M production config needs one box, + which is what leaves crow untouched. +- **Flash-Next single-box has never wedged.** All three GPU wedges were DSv4 two-box at 86 to 91 GiB per box, + confirmed against CROW-SCHEDULE's own incident rows, which name DSv4 and dspark and never Flash-Next or GLM. +- **Crow prod is 61 to 62 GiB of GTT**, measured two ways. A live read gives 60.5 GiB; two independent window + teardowns gave MemAvailable swings of 64 and 61 GiB on evicting all four containers, with 17 MiB of GTT residue + left behind. Of that, the vLLM 4b is 15.2 GiB by KFD accounting, which puts **the 35b at roughly 43 GiB** by + subtraction. Cross-checked from the other direction: its weights are 27.16 GB plus a 0.9 GB mmproj, about + 26.1 GiB, and at `-c 262144` another ~17 GiB of KV and compute buffers is plausible. The Vulkan containers do not + appear in ROCm accounting at all, so 43 GiB is a **bound rather than an isolated measurement**, and every gate in + this spec treats it as one. +- **GLM and DSv4 cannot join the standard state.** Their weights exceed one box at every usable quant + (GLM IQ4_XS 146 GiB, DSv4 IQ4_XS 128 GiB against 124 GiB MemTotal), so they are structurally two-box, and the + second box is crow. A crow-side worker share of 46 to 56 GiB does not fit beside a 43 GiB 35b plus the rest of + prod, and the splits that would make it fit (about 0.73/0.27 toward raven) are untested and put raven at its + ceiling. Hence: they get their own state, with both boxes evicted. + +**Assets are already in place.** Raven holds `qwen38-flash-next` UD-Q4_K_XL (104 GB) and UD-Q5_K_XL (148 GB), both +MTP heads including `shared-Q8_0`, plus `dsv4-flash-vision-exp`, `dsv4-flash-0731` and `glm53-flash`, with 614 GB +free at 66 percent. Crow has 168 GB free at 91 percent and does **not** hold the Flash-Next Q4/Q5 quants. Nothing +needs downloading for this design. + +## 3. What the standard state requires + +### 3.0 Ordering constraint: the two-host window lands FIRST + +**Flash-Next production must not go onto raven until `dsv4-window.sh` can evict and restore raven prod.** This is a +hard ordering constraint rather than a preference. + +`dsv4-window.sh` evicts and restores crow prod only, and its `preflight` requires raven to be idle with under 2 GiB +of GTT residue. The moment raven carries a production service, every two-box arm either fails pre-flight on "raven +busy", which is the good outcome, or contends with a live production service, which is the bad one. Thursday's +chain is safe only because raven is idle today. + +So the deployment order is: teach the window two hosts, then stand up Flash-Next on raven. Doing it the other way +round breaks the two-box benchmark program on its next run. + +### 3.1 Flash-Next as a production service on raven + +Raven has run no production services until now. This makes it a second production host. + +Launch, from the validated single-box config (tree `~/llama-max-stack`, or b10715 for the MTP head): + +``` +llama-server -m /Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf \ + -c 1048576 -b 4096 -ub 2048 -ctk q8_0 -ctv q8_0 \ + --rope-scaling yarn --rope-scale 4 --yarn-orig-ctx 262144 \ + --override-kv qwen4exp.attention.indexer.top_k=int:1024 \ + --spec-type draft-mtp -md /mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf --spec-draft-n-max 2 \ + -ngl 999 -fa on -np 1 --jinja --host 0.0.0.0 --port +``` + +**Run it natively on raven's host, not in a container.** The bind-mount into the stock Vulkan image works +(pi-lab probed it: `llama-bench --list-devices` enumerates the GPU identically). The image does carry Mesa 25.3.6 +against raven's host 25.2.8, and H.4 measured image choice moving 27b prefill by 36 to 57 percent. Every validated +number for this config was taken on the host. A native systemd unit keeps the measurements valid and removes the +container from the trust chain. The existing benchmark arms already run exactly this way. + +Open items for the implementer: +- **Port 8030, confirmed free on raven** (2026-09-09: only 22, 53, 631 and two ephemeral ports listen there). + 8031 through 8033 are free too. Closing this before the provider row exists matters, because a provider row + pointing at an occupied port is a subtler form of the `crow-dsv4` bug this spec retires. +- **Provider row** pointing at `http://10.0.0.126:/v1`. This row **does** replicate to every paired + instance, and that is desirable — but membership in `SYNCED_TABLES` is not what decides it. `shouldSyncRow` + does, and it exempts loopback endpoints; this row syncs because `10.0.0.126` is routable, where the + `crow-dsv4` row §7 retires never synced at all. Two requirements follow, and both were violated on the first + attempt (2026-09-10): + - Set **`host = 'raven'`**, on every instance that carries the row. `host` rides the wire, so whichever copy + wins on `lamport_ts` imposes its value fleet-wide, and `host = 'local'` tells + `resolveWarmableProviderName` the endpoint is startable on the local box when it is not. + + **Setting it explicitly is not a checklist item — it is a workaround for an unsound inference.** When a + caller omits `host`, `inferHost` (`shared/providers-db.js:50`) supplies one, and it classifies every + `10.`, `192.168.` and `100.` address as `local`. That is the whole RFC1918 space plus the entire Tailscale + CGNAT range: by that rule raven, grackle, black-swan, colibri and mockingbird are all "local" to every box + in the lab. The only two answers it can return honestly are loopback and public. Existing rows are shielded + only by `if (existingHost) return existingHost` and by long-lived rows carrying explicit hosts, so the + exposure is the seed path — which means the next new id reproduces it. Until that inference is fixed, treat + an omitted `host` on any non-loopback row as a wrong value rather than a missing one. + - **Publish the row only once the endpoint is reachable from the instances that will inherit it.** raven's + `ufw` rule currently admits crow alone; grackle and black-swan are paired with crow and cannot reach 8030. + Either widen the rule or accept that those instances carry an entry that probes as down. See §7 for the + full argument and the reason `gpu_policy.local_only` is the wrong lever. +- **Restart policy and a memory watchdog.** 17.7 GiB is the permanent headroom on raven in this state. It is a + shape that has run for hours (R25/R25b) and never wedged, and it is still the tightest standing configuration in + the lab. +- **MTP at 1M carries a known quality caveat**: output is not byte-identical between MTP and non-MTP at 1M + (1752 versus 1777 tokens on prompt A), where it was identical at 262k. It has not shown up as a quality loss: + the 1M arms scored 8/9 twice, the same as 262k, missing the same case. MTP is worth 1.82x on prompt A, so keep it, + but record the caveat rather than discovering it. +- **8030 and 8036 are mutually exclusive on raven.** 8036 is the R25/R25b benchmark port for this same + single-box Flash-Next config. Both ports being free is irrelevant: each instance wants about 92.6 GiB on a + 124 GiB box. Any R25-lineage arm must treat the production service as something to evict, exactly like the + two-box arms in 3.0. +- **A 262k fallback is worth defining.** 262k leaves 31.0 GiB free instead of 17.7 and is the more exercised rung. + If 1M proves uncomfortable in standing use, 262k is the same service with one flag changed. + +### 3.2 Group and eviction convention + +`group` and `evicts` in `settings.localModels` are global strings and host-blind. `wouldEvict` compares group +names and nothing else. So a raven entry that declared `evicts: ["standard"]` would stop crow's 35b for no reason. + +**Decision (crow-34, owner): the group convention lives in `settings.localModels`, not in the catalog.** +`group`/`evicts` describe what else is running on a given box, which is instance topology rather than a property of +a model. The same Flash-Next entry would need a different group on crow than on raven, so putting it in a curated +file would bake one lab's host layout into content meant to describe models. + +Convention for this design: +- raven's Flash-Next entry: its own group, `evicts: []`. It never evicts anything. +- crow's existing `standard` group: unchanged, and must not name raven's group. +- the heavy states: group `heavy`, evicting everything on both hosts. This is the only group that may cross hosts. + +**This convention is a workaround and should be recorded as one.** Encoding a *host* distinction inside a global, +host-blind string works only while everyone remembers it, and this section is its own evidence: a raven entry that +declared `evicts: ["standard"]` would silently stop crow's 35b. **The intended end state is a host-aware eviction +relation**, after which the convention becomes unnecessary. Writing that down here is what stops every future +multi-host entry from re-learning the trap the same way. + +### 3.3 The harness can consume this, and cannot manage it + +`startModelNow` runs `docker compose up -d` with `cwd: entry.composeDir` on whichever box runs the harness. There +is no host field, no ssh, no docker context. **The harness cannot start or stop a raven-hosted model.** It can +consume one, because `isRunning` fetches `/models` over the LAN. + +For the standard state this does not matter: the service is persistent, so nothing needs to start it on demand. +This is the main reason this design is cheaper than the alternatives considered. Do not build a `composeDir` +entry for it that implies otherwise, because the first start would fail. + +For the heavy states it does matter, and the window script already ssh-es to raven, so the lifecycle belongs there +rather than in the harness. + +### 3.4 One harness defect this design will hit + +`setRoleModel`'s conflict test is: + +```js +const conflicted = bindings.filter(([role, m]) => role !== modelRole && m && m !== ref && isLocal(m)); +``` + +It fires whenever two roles hold two different local models. It never calls `wouldEvict`, never reads +`group`/`evicts`, and has no host concept. So binding a raven model to one role and a crow model to another raises +"local models share one server slot" and offers to collapse them onto one model. It is a `confirm()`, so Cancel is +safe and nothing is destroyed, and the harness will argue against this configuration every time a binding changes. + +The fix is to ask whether starting one actually evicts the other. `wouldEvict` is already exported and already the +right predicate, and with the convention in 3.2 it returns false for the raven/crow pair. The work is exposing the +eviction relation to the client, which today receives only a boolean `local` flag from `annotate()`. This is +pi-lab's tree and needs its own change with a test. + +## 4. The heavy states + +Both are window modes, not services. Entry conditions, per pi-lab's classes: + +- **GLM-5.3-Flash IQ4_XS two-box**, pipeline parallelism off, `-b` equal to `-ub` at 2048 (#28360 asserts at 512 + at every rung, and at 1024/2048 with PP on). Least exercised of the three. Its top_k quality gate has not run. +- **DSv4-Flash IQ4_XS two-box**, tree `~/llama-hc-sop`, now equivalent to master plus #28571 since #26578 merged + on 2026-09-07 (`7a333e724`). Highest quality of the three at 9/9, and the exact shape that wedged the box three + times. Operator present, explicit override, never a one-tap option. + +Both now require **evicting production on two hosts** and restoring both. Today `dsv4-window.sh` restores crow +prod only. Teaching it raven prod is a prerequisite for this design, and it is the one new capability the heavy +states need. + +Context caps: on the max-stack lineage, keep two-box context below 128k, because the RPC worker aborts at +`ggml-rpc.cpp:1386` on 128k prompts on both cache and no-cache builds. This is lineage-scoped, not universal: +R24 W1b ran a 120k prompt two-box on `llama-hc-sop`, which carries `supports_op`, successfully. For qwen4exp above +64k on the #28571 lineage it is untested, and Thursday's W0 tests exactly that. + +## 4.1 A note on how the constraints here are written + +Where this spec had a choice, it states arithmetic rather than a rule. "Evict raven production before an +R25-lineage arm" is a rule someone forgets at 02:00. "92.6 GiB each on a 124 GiB box" is arithmetic anyone can +redo from scratch and get the same answer. The second form survives the people who wrote it, and it fails loudly +when the hardware changes underneath it, where a remembered rule just quietly stops being true. + +Prefer it wherever a later section adds a constraint. + +## 5. The window contract, which is not negotiable + +Whatever wraps a heavy state must preserve these five properties. Each was written after an incident, and the +orchestrator should **call** `~/pi-lab/scripts/dsv4-window.sh` rather than reimplement it. + +1. The deadman is **out of process**, a detached watchdog with its own wall clock, and restores production even if + the orchestrator dies, hangs or is killed. This is a lab-wide rule. +2. The memory gate reads **live** state, MemAvailable plus GTT residue after eviction, never a static + `min_vram_gb` floor. A static floor cannot see the 86 GiB a dead process is still pinning, which is the exact + condition that produced the wedges. +3. Production restore is **verified**, health check plus DB integrity, before a window is called closed. With this + design that now means both hosts. +4. The box reservation file is written, because that is what makes the gateway refuse a competing on-demand start. +5. The two-strike ownership guard stays. + +**Veto, from pi-lab, carried here as a requirement:** a curated catalog must never be able to offer a known-wedge +shape as a one-tap option. + +### 5.1 Open seam: window expiry during a live turn + +Raised by crow-34, and jointly owned. A window expiring restores production out from under a live conversation, +which is correct for the box and bad for the bot. + +Proposed answer, for review: **drain, never extend.** The out-of-process deadman keeps its absolute wall clock, +untouched, because property 1 above is not negotiable. A separate soft deadline before it stops accepting new +turns and lets the in-flight turn finish. If that turn is still running at the hard deadline, the deadman wins and +the turn dies. The operator sees the soft deadline in the session drawer. + +Extending the deadman on activity is explicitly rejected: that is how a cap stops being a cap. + +Note that in the standard state this seam is milder than it looks, because evicting for a heavy state stops +raven's Flash-Next but leaves crow's 35b as a genuine fallback, which is not true of any design where the heavy +model displaces crow prod. + +## 6. Catalog work, in shippable order + +`registry/model-catalog.json` v1 already carries `qwen3.8-flash-next`, `glm-5.3-flash` and `deepseek-v4-flash`. +Their `launch` blocks carry only `ctx`, `ngl`, `flash_attn`, `no_mmap`, `jinja` (plus `parallel` on Flash-Next). +None of the validated tuning is expressible and there are no topology or rung variants. On the memory fields, to be +exact: `min_vram_mb` is **absent at the entry level** and **present-but-zero on every quant**, while `min_ram_mb` +*is* populated and meaningful (115,068 MB for Flash-Next UD-Q4_K_XL, 157,911 for GLM UD-IQ4_XS). So the catalog +already carries a RAM figure; what it lacks is measured peak GTT per host per config, which is what a gate needs. +`min_runtime_version` says `b10068` on all three while each entry's own notes admit the stock runtime will not load +them. + +crow-34's sequencing, adopted: land this in pieces rather than as one block. + +1. **`serving.class`** first. Independently deployable, needs no build, topology or window support, and protects + the box on day one by letting the orchestrator refuse a known-wedge shape without an explicit override. Classes: + `resident` (single box, no RPC, safe behind a cap), `windowed` (operator present, two-box, evicts), and + `wedge-risk` (above roughly 85 GiB per box, explicit override, never one-tap). + + **Decision (crow-34, owner): `serving.class` is a catalog schema field on the model entry, and must NOT live in + `settings.localModels`.** The reason is the veto itself. A curated safety property held in per-instance settings + is one settings edit, or one bug in a settings writer, away from a `wedge-risk` shape being relabelled `resident` + locally, after which the orchestrator offers it as one tap. `registry/model-catalog.json` lives in git and gets + reviewed, which is exactly the property a veto needs. **Instance settings may narrow what a box will run, never + widen it.** + + **Class is a property of a config, not of a model**, and the catalog has no variant concept yet. Flash-Next + single-box at 262k is `resident`; DSv4 two-box at 86 to 91 GiB per box is `wedge-risk`; those could be one entry. + So the entry-level field is defined as **a ceiling, not a description: the most dangerous supported shape for + that model.** DSv4 is `wedge-risk` outright and can never be one-tap however it is invoked, Flash-Next is + `resident`, GLM is `windowed`. Over-restrictive in principle, correct in every case that exists today, and it + preserves the veto on day one without waiting for step 4. + + When `topology` lands in step 4, class moves onto the variant, with the entry-level value kept as an **enforced + ceiling: a variant may never declare itself safer than its model's class.** The ratchet is one-way by design + rather than by whoever implements step 4. +2. **`gates`**, live state rather than a static floor: max GTT per host, required MemAvailable. The measured peaks + per config are in the serving doc's memory table and belong here. +3. **`build`**, naming the tree or PR set a config needs, since `min_runtime_version` against a stock release is + known-false for all three models. +4. **`topology`**, host roles and link preconditions. Last, and only needed once a two-box state is curated. + +**Throughput numbers need care.** Any figure the catalog publishes per config must come from a run in the +configuration as deployed. The doc's numbers are host-native. If a config is ever containerized, it needs one +confirmation run first, for the Mesa reason in 3.1. + +## 7. Decisions still open + +- **The `crow-dsv4` provider row — RESOLVED 2026-09-10.** It pointed at `http://127.0.0.1:8020/v1`, was + enabled, and validated in `resolveModel` while nothing listened there, so a bot pointed at it failed every + turn on connection refused. Kevin approved retirement. Both instances now carry `disabled = 1` rather than a + `DELETE`: `resolveModel`'s predicate is `WHERE id = ? AND disabled = 0`, so the row stops resolving while + staying reversible if DSv4 ever returns as a windowed serve. + + **The retirement had to be performed twice, once per instance, and the reason generalises.** An earlier draft + of this bullet said "because `providers` syncs, this is a fleet decision". That is false for this row. + Membership in `SYNCED_TABLES` is necessary but not sufficient — `shouldSyncRow` decides, and it carries a + loopback carve-out (`servers/sharing/instance-sync.js:328`): + + ```js + if (host === "localhost" || host === "::1" || /^127\./.test(host)) return false; + ``` + + `http://127.0.0.1:8020/v1` matches, and that gate covers **both emit and apply**. A peer dialing 127.0.0.1 + reaches itself and never the origin's service, so loopback provider rows are per-instance by construction. + Retiring one is a per-instance action each session performs on its own box, and waiting to observe it arrive + from a peer yields a false negative rather than a result. + +- **Loopback and routable provider rows sit on opposite sides of that carve-out, and this spec creates one of + each.** They are not one category and their failure modes are opposites, so the distinction is load-bearing + rather than editorial: + + | | `crow-dsv4` | `raven-flash-next` | + |---|---|---| + | base_url | `http://127.0.0.1:8020/v1` | `http://10.0.0.126:8030/v1` | + | syncs? | no — loopback carve-out | **yes** | + | a stale row costs | a local annoyance each box fixes itself | a fleet-wide pointer every paired instance inherits | + + §3.1 already wants the raven row to propagate, and that remains right. What follows from it is that the row + must not be published before the service behind it is real and reachable from the instances that will inherit + it — otherwise the spec fleet-publishes exactly the class of dead pointer the bullet above spent a day + retiring. Two consequences, both verified live on 2026-09-10: + + - **`host` rides the wire** (it is not in `EXCLUDED_COLUMNS.providers`), so whichever copy of a row wins on + `lamport_ts` imposes its `host` value fleet-wide. A row whose service runs on raven must say `host = 'raven'` + on **every** instance. `host = 'local'` is not merely cosmetic: `resolveWarmableProviderName` + (`gpu-orchestrator.js:619`) returns null only for a non-`local` host, so a `local` mislabel falls through to + the sibling scan and is one same-baseUrl bundle away from an instance trying to warm a service on another + machine. + - **Reachability is per-instance and the firewall decides it.** raven's `ufw` rule scopes 8030 to `10.0.0.237` + (crow) alone. The crow instance is paired with grackle and black-swan as well, neither of which can reach + that endpoint — black-swan, off-LAN, never will. Publishing the row to them costs an entry that probes as + down. The honest fix is the firewall rule, not `gpu_policy.local_only`, which would suppress the crow↔R4 + sync this spec actually wants. +- **Priority of the two harness changes** against the catalog work. Note that remote lifecycle (3.3) is *not* + needed for the standard state and should not be built speculatively. That leaves the two-host window (3.0) and the + swap predicate (3.4). pi-lab's read, which this spec endorses: **the two-host window first**, because it is the + hard prerequisite that makes a two-box arm safe once raven carries production, where the swap predicate is a + confirm dialog. Kevin's call. +- **An isolated per-container GTT measurement**, offered by pi-lab and not yet taken. The window restore path + already brings containers back one at a time, so logging GTT between each restore would yield real per-container + figures instead of the 43 GiB bound this spec uses. It is a log-only change in a script that runs unattended with + a chain armed, so it is a five-minute job that wants a deliberate go rather than a quiet edit. +- **524k as a curated rung.** Deferred. It has exactly one two-box run, no byte-identity check and no quality arm, + where 262k and 1M each have several runs, a full corpus twice and 8/9 twice. It wants an identity check and one + zoo arm before curation. + +## 8. Scheduling + +Raven stops being single-tenant. It is master for every two-box arm, so a heavy state and any two-box benchmark +work are mutually exclusive in time, and Flash-Next production must be evicted before either. + +pi-lab has rewritten the CROW-SCHEDULE standing note accordingly: raven is a production host, the service is native +systemd for the Mesa reason in 3.1, and three consequences follow. Raven is no longer free to borrow. The ordering +constraint in 3.0 applies. And property 3 of the window contract, verified restore, now spans two hosts.