Fix local inference on Windows: silent CPU fallback, dead CLI flags, and broken engine packaging

[alnavone]

Fix local inference on Windows: silent CPU fallback, dead CLI flags, and broken engine packaging

Four bugs in the sd.cpp local-inference path, found while setting the feature up on a Windows + RTX 3090 machine. Each is independently reproducible; the first three make local generation unusable or silently slow, the fourth ships a non-functional engine in packaged Windows builds.

Tested against leejet/stable-diffusion.cpp master-813-bfbef5b on Windows 11 x64, RTX 3090 (24 GB), no CUDA toolkit installed.

---

1. Windows CUDA builds silently fall back to CPU

downloadBinary() fetches sd-master-*-bin-win-cuda12-x64.zip, which ships ggml-cuda.dll but not the CUDA runtime it links against. cudart64_12.dll, cublas64_12.dll, and cublasLt64_12.dll are published as a separate asset (cudart-sd-bin-win-cu12-x64.zip) in the same release.

Without them, ggml skips the CUDA backend and falls back to CPU. There is no error, no warning, and nothing in the UI to suggest anything is wrong — the app looks like it is working, just slowly.

Before, on a 24 GB RTX 3090:

[INFO ] ggml_extend_backend.cpp:545  - Found 1 backend devices:
[DEBUG] ggml_extend_backend.cpp:548  - #0: CPU

GPU utilisation 0%, VRAM 19 MiB, 2259 s of CPU time, still unfinished after 7 minutes at 512x512 / 20 steps.

After:

[INFO ] ggml_extend_backend.cpp:545  - Found 2 backend devices:
[DEBUG] ggml_extend_backend.cpp:548  - #0: CUDA0
[DEBUG] ggml_extend_backend.cpp:548  - #1: CPU
stable-diffusion.cpp:5675 - sampling completed, taking 4.27s

Fix: new ensureCudaRuntime() runs after the engine extracts. On Windows only, when ggml-cuda.dll is present but the runtime DLLs are not, it pulls the cudart-* asset from the same release, extracts it beside sd-cli.exe, and flattens any nested layout. No-op on macOS/Linux and on CPU/Vulkan/ROCm builds. It re-checks afterwards and warns rather than failing silently if the install comes up short.

Anyone who already installed the engine is unaffected on upgrade — the check is idempotent and skips when the DLLs are present.

2. --sd-version and --flux no longer exist

generate() passes --sd-version sdxl for SDXL models and --sd-version sd2 for SD2. Upstream removed those flags; the architecture is now detected from the checkpoint. --flux is gone the same way.

The result is not a degraded image — sd-cli aborts before doing any work:

[ERROR] common.cpp:325  - error: unknown argument: --sd-version

This makes every SDXL model in the catalog fail 100% of the time on current sd.cpp builds.

Fix: dropped the branch. sd.cpp reports Version: SDXL on its own from the same checkpoint.

3. SDXL's sampler name is invalid

modelCatalog.js sets sampler: 'dpmpp2m' for SDXL Base. The accepted spelling is dpm++2m:

[ERROR] common.cpp:1259 - error: invalid sample method dpmpp2m

Masked by bug #2 — argument parsing failed on --sd-version first — so it only surfaces once that is fixed.

Fix: dpmpp2m -> dpm++2m.

4. Packaged Windows builds bundle a dead engine

stage-local-ai-binary.js copies an allowlist of filenames. On Windows that list is ['sd-cli.exe'], plus an optional 'sd-server' that never matches anything because the Windows binary is sd-server.exe. So npm run electron:build:win bundles exactly one file out of the ~24 the runtime needs.

Reproducing the old staging output — sd-cli.exe alone in a directory:

exit=-1073741515   # 0xC0000135 STATUS_DLL_NOT_FOUND

The process cannot start at all. Not --help, nothing.

The allowlist is wrong in principle, not just in its entries: sd.cpp spreads its runtime across stable-diffusion.dll, the ggml*.dll family (including nine per-ISA CPU variants), codec DLLs, and the CUDA runtime — and that set varies by platform and build flavour. macOS and Linux were under-staged too, just less fatally, since they also ship libggml* alongside libstable-diffusion.*.

Fix: replaced the allowlist with a recursive copyTree() that stages the whole source directory. REQUIRED_FILES is kept as a validation gate rather than a copy manifest, and gains stable-diffusion.dll on Windows for symmetry with the darwin/linux entries. Removed the dead OPTIONAL_FILES.

Verified against a real CUDA build: 24 files, 1147 MB staged, and the staged binary generates on CUDA0 standalone.

---

Tests

New tests/stageLocalAiBinary.test.js (4 tests, node:test, matching the existing suite): full-runtime staging with a realistic CUDA file set, rejection of a source directory missing the core shared library, unsupported-platform rejection, and nested-bin resolution.

node --test "tests/**/*.test.js"
# tests 21 | pass 21 | fail 0

17 pre-existing tests still pass.

End-to-end verification

Driving the app's own local-ai:generate IPC handler against all three model families:

PASS  dreamshaper-8               5.9s   439 KB
PASS  stable-diffusion-xl-base   27.1s  2450 KB
PASS  z-image-turbo              21.2s  1893 KB

Bugs #1 and #4 are Windows-specific. #2 and #3 affect SDXL on every platform.