deepseek41 : admit unified host memory before allocation

[ajaxdude]

Overview

Adds the DeepSeek V4.1 Strix Halo host-memory admission and watchdog integration layer for halo-box#48. This PR is stacked on #7 at exact full-graph head 081c549451b92c01fe832ffb0bdd9ad90370e21a and includes the accepted external-watchdog behavior from #1 at exact head 778db6f50eae04e6c232c69b9575bdbd0747962b.

Before expert-cache or model backend allocation, admission reads integer-byte host use from Linux procfs, rejects any configured swap entry, requires an IGPU-only unified-memory topology, measures full-graph state through the no-allocation memory implementation, accounts the conservative graph workspace and all bounded staging/output categories, and auto-fits complete expert slots under the 116 GiB soft ceiling. It enforces the 118 GiB external-watchdog threshold and strict <120 GiB hard requirement without changing the GGUF, quantization, requested context, or explicit ubatch.

The common CLI and server use ubatch 32 for DeepSeek V4.1 only when -ub is not specified. The server resolves implicit auto parallelism to one DeepSeek sequence and aligns no-allocation fitting to the admitted envelope; explicit values are preserved and must fit. Direct libllama defaults resolve the admitted DeepSeek ubatch while other architectures retain an effective default of 512. Expert replacement accounting includes the full incoming routed-expert union plus the largest 4096-aligned direct-I/O bounce read. Runtime-context ownership is acquired immediately after parameter validation, before sampler, output, state, backend, or scheduler allocation, with exception rollback.

The external watchdog remains a composable process-group wrapper rather than duplicated C++ process control. It monitors host-wide use and swap after startup, forwards SIGHUP, SIGINT, and SIGTERM, escalates at the configured grace timeout, and caps overrides at repository policy limits. The independently approved canonical replacement publishes a version 2 self-authenticating lease, per-sample atomic heartbeat, guardian fail-closed lifecycle, and persistent JSONL audit that bind the live watchdog, exact thresholds, procfs root, child process group, and exact child command. Independent exact-SHA review closed all nine recorded watchdog findings.

Measurements

No performance claim is made. This change is a safety gate and runtime accounting layer; real Strix Halo model validation is currently blocked by active unrelated DeepSeek V4 work, an enabled 32 GiB /swapfile, and no active watchdog. I did not interrupt workloads or change swap/ROCm configuration.

Device:     Local Apple M5 validation host; Strix Halo runtime not exercised
Memory:     Strix target is 128 GB unified LPDDR5X; local host differs
Power:      Not measured
BIOS:       Not measured
Kernel:     macOS local build; Linux procfs covered with fixtures
Backend:    Metal/Accelerate local model-free tests
Build:      Release, build-admission
Baseline:   081c549451b92c01fe832ffb0bdd9ad90370e21a
Watchdog:   778db6f50eae04e6c232c69b9575bdbd0747962b
Change:     4f4a47fd703e0d24bf2dcf0bb93dfa6c8635d1b5
Model:      Published DeepSeek V4.1 GGUF geometry in model-free fixtures; no model file loaded

Baseline:

Not applicable: no throughput or latency claim.

After:

Not applicable: no throughput or latency claim.

Correctness:

• Built llama-cli, llama-server, admission/schema/Engram/expert/memory/runtime tests, expert-store, recurrent rollback, save/load state, arg parser, and architecture test targets.
• 14/14 focused and adjacent CTests passed, including deterministic server-default, fit-envelope, public-context-default, and runtime-context acquisition/rollback coverage.
• The canonical 36-test watchdog suite completed normally and with ResourceWarning promoted to errors on macOS: 28 passed and 8 Linux-only tests skipped in each mode. The watchdog script SHA-256 is d2781a25f978dd2bc14fc113079aa2dbf513aa157b44da9d0d51d750daa6c94f, byte-identical to the independently approved artifact. py_compile, flake8 with flake8-no-print, and file-scoped ty passed.
• test-llama-archs --arch qwen4exp and --arch grovemoe passed available backend comparisons; DeepSeek41 compact architecture checks skipped as intended.
• git diff --check and the ASCII-only diff scan passed.

Additional information

Admission diagnostics report current host use, fixed/dense/state/workspace/staging/output bytes, selected and required expert slots, exact cache bytes, replacement bytes, aligned direct-I/O bounce bytes, safety margin, soft/watchdog/hard thresholds, ignored device-reported bytes, and the rejecting category. Context checkpoints are explicit at 32768, 65536, 98304, and 131072; an unsupported or non-fitting request fails instead of being lowered.

The documented validation command uses NVMe model storage and explicitly binds ROCR_VISIBLE_DEVICES=0, HIP_VISIBLE_DEVICES=0, HIP_LAUNCH_BLOCKING=1, -dev ROCm0, -ub 32, -np 1, 192 slots, and the exact 72900 MiB expert budget. Its preflight must confirm that ROCm0 reports gfx1151 and validate the active watchdog lease. Buffered expert/Engram I/O cannot claim boundedness. Dynamic embedding-output requests are rejected and cause the next encode/decode operation to fail explicitly because they are outside the admitted output profile.

Requirements

• I have read and agree with the contributing guidelines
• This change is Strix Halo specific, or justified by measurements on Strix Halo. General llama.cpp improvements belong in halo-box/llama.cpp instead
• AI usage disclosure: AGENT-AUTHORED by GitHub Copilot powered by GPT-5.6 Sol. The agent implemented, reviewed, merged the corrected stacked parents, and ran local validation.
• Commit-trailer disclosure: ancestry-only merge commits b26c0e687a556ca501fb056dbbdc4c6294bf90b9 and d0a161602f96ae7fb90d5e702f010f198ce71c6a were created during agent-coordinated stack propagation and contain no independent semantic change; they do not carry the required Assisted-by/Copilot trailers. History was not rewritten or force-pushed.
• What was NOT verified: Real Ryzen AI Max+ 395 startup, gfx1151 IGPU device classification, zero-swap admission, the external watchdog around the published model, model output equivalence, and 32768/65536/98304/131072 runtime progression remain unverified until the Strix host is idle, swap is disabled with explicit authorization, and the watchdog is active. No model/evaluation I/O was performed on rotational storage and no ROCm packages were changed.

[ajaxdude]

Independent review after publication found three remaining admission blockers in head d0a161602f96ae7fb90d5e702f010f198ce71c6a; this head is superseded pending fixes and should not be treated as final verification:

• common applications still replace the bounded DeepSeek41 default with implicit n_ubatch=512, requiring all 384 expert slots and making normal CLI/server startup inadmissible;
• the single-context reservation occurs after output/state/scheduler allocations, so a second context can exceed the host ceiling before rejection;
• replacement peak accounting omits the incoming per-ubatch expert payload plus one aligned direct-I/O bounce buffer while old residents remain live.

The correction will preserve explicit -ub overrides, acquire the sole context before expensive allocation with rollback on construction failure, and add min(384, 6*n_ubatch) * 9,953,280 + 3,874,816 bytes to replacement peak accounting. Canonical watchdog head 8acdb46f83251b531b896658d5c913ba8edecf4a has also been merged locally. Focused regressions and the complete admission/watchdog suite will be rerun before the replacement head is published.

[ajaxdude]

Published blocker replacement head a127c1999f45ff0937cda1b808958d37f7a5a6d1 on top of full graph 081c549451b92c01fe832ffb0bdd9ad90370e21a and canonical watchdog 8acdb46f83251b531b896658d5c913ba8edecf4a. This closes the three admission-envelope blockers: DeepSeek41-only implicit ubatch 32 with explicit -ub preserved, runtime-context acquisition before expensive allocations with exception rollback, and full incoming expert replacement plus aligned direct-I/O bounce accounting. Local validation: 14/14 focused/adjacent CTests, 20/20 watchdog tests, adjacent qwen4exp/grovemoe architecture comparisons, build targets, diff check, and ASCII scan passed. Strix hardware validation remains blocked by the active workload, configured swap, and absent watchdog; no host state was changed.

[ajaxdude]

CI note for run 34739092629: the Ubuntu server job passed. The Windows job completed with one failure in unit/test_router.py::test_router_delete_model, a requests ReadTimeout after 600 seconds; its summary was 1 failed, 370 passed, and 6 skipped in 839.06 seconds. This matches the previously observed unrelated server-router timeout pattern and does not exercise or touch the admission changes. The failure remains unsuppressed and no code or test configuration was changed.

[ajaxdude]

Follow-up: inspected Windows job 103675593593 failed-step logs directly. The sole failure is unit/test_router.py::test_router_delete_model at 603.73 seconds with requests.exceptions.ReadTimeout to 127.0.0.1:8110 using the 600-second read timeout; summary remains 1 failed, 370 passed, 6 skipped. No admission source or test failed. I requested one rerun of the failed job without suppressing or changing code.

[ajaxdude]

Final admission replacement published at 265f3df3055cbcdbf10a545d8d7c234ddd495fab. It normally merges canonical watchdog PR #1 head 59833018814c0883f995848912cd5a52889c5303 and binds the documented validation launch to ROCm0, required gfx1151 preflight, HIP_LAUNCH_BLOCKING=1, -ub 32, 192 slots, and the exact 72900 MiB expert budget. Validation passed: 14/14 focused and adjacent CTests; watchdog 24/24 normally and 24/24 with ResourceWarning errors; py_compile; flake8 with flake8-no-print; file-scoped ty; full PR diff and ASCII checks. The earlier Windows router ReadTimeout remains documented as unrelated and unsuppressed. Strix execution remains blocked by the active workload, configured 32 GiB swapfile, and no active watchdog; no host state was changed.

[ajaxdude]

Hardware-use freeze: head 265f3df3055cbcdbf10a545d8d7c234ddd495fab includes watchdog 59833018814c0883f995848912cd5a52889c5303, which independent review found has four high-severity fail-closed defects. This admission head is superseded for Strix validation pending a normal merge of the corrected watchdog and complete revalidation. The earlier unrelated Windows router timeout remains unsuppressed and is not the reason for this freeze.

Written by GPT-5.6 Sol.

[ajaxdude]

HARDWARE FREEZE: head 265f3df3055cbcdbf10a545d8d7c234ddd495fab is superseded for hardware use because watchdog parent 59833018814c0883f995848912cd5a52889c5303 has four open HIGH fail-closed defects: forgeable launch/artifact identity, child process group surviving watchdog SIGKILL, emergency handling delayed by artifact fsync, and cleanup/escalation/reap aborted by artifact failure. Do not integrate or run this head on Strix. I will consume a corrected watchdog head only after an independent read-only review explicitly closes all four findings with regression evidence.

[ajaxdude]

Review candidate 8badaf9b581c5f17c7e6affdd88c3d5618795eb9 normally merges independently approved watchdog 778db6f50eae04e6c232c69b9575bdbd0747962b; all nine watchdog findings are closed. Local evidence: 14/14 focused and adjacent CTests; watchdog suite completed in normal and ResourceWarning modes with 29 passed and 8 Linux-only skipped each; py_compile, flake8 with flake8-no-print, file-scoped ty, full PR diff check, and ASCII scan passed. The exact validation envelope remains ROCm0/gfx1151 preflight, HIP_LAUNCH_BLOCKING=1, ubatch 32, 192 slots, 72900 MiB, 116/118 GiB thresholds, strict below 120 GiB, and zero configured swap. Hardware validation is not authorized or claimed.

[ajaxdude]

Corrected review candidate c2c98de7cc0e1876989f90fc49aeb0aafc2d2193: approved watchdog ancestry remains 778db6f50eae04e6c232c69b9575bdbd0747962b, and scripts/strix_memory_watchdog.py is now byte-identical to that artifact with SHA-256 d2781a25f978dd2bc14fc113079aa2dbf513aa157b44da9d0d51d750daa6c94f. The admission-only watchdog threshold test was removed; 116/118/strict-below-120 enforcement remains covered in DeepSeek admission. Validation: 14/14 CTests; canonical watchdog suite in both modes with 28 passed/8 Linux-only skipped; py_compile, flake8+no-print, file-scoped ty, full diff, and ASCII checks passed. Hardware validation remains unauthorized and unperformed.