feat(launch): UX improvements regarding auto device placement, DeepSeek V4 flags and profiles, pass-through container entrypoint

[Graffioh]

Fixes #712.

Why

On an R9700 + Strix Halo host, the :rocm container loaded DeepSeek V4 Flash onto device 0 (the 32 GB R9700) and failed to allocate 93 GB. There was no way to fix this from the container, because the launch system could not express most luce_server setups:

• serve dropped any extra arguments.
• A bare flag such as --target-device was exec'd as a command.
• GPU detection used nvidia-smi only, so ROCm launches got no context sizing.
• The target always went to device 0.
• DeepSeek V4 received --draft/--ddtree, which it ignored; its drafter was env-only (LUCE_DS4_SPEC, LUCE_DS4_DRAFT).
• The qualified R9700 + Strix recipe needed 43 exported variables.

What changes

luce_server

• --target-device auto places the model on a GPU it fits on: discrete before integrated, then the lowest index; if none fits, the largest. LUCE_TARGET_DEVICE supplies the default when no flag or profile names a device.
  • The fit is weights + a 2–4 GiB margin, plus, for Qwen3.5/3.6, the KV cache at --max-ctx, sized from the GGUF header with the backend's own helper and cache types (Qwen3.8-27B: 2.2 GiB at 131072 tokens with q4_0). DeepSeek V4 and other families size their own caches and keep the margin. --list-devices prints kv_mib at the default context.
  • Native launches keep auto:0 unless they opt in.
  • Draft placement has one order everywhere: an explicit --draft-device (auto:N means GPU N), then DeepSeek V4's LUCE_DS4_DRAFT_GPU/_BACKEND, then the target GPU.
• luce_server --list-devices [model.gguf] prints each GPU's backend:N index, arch and memory, the model's arch and size, and the auto choice.
• When a load fails on a device that is too small, the server names the device that would hold the model.
• --draft on DeepSeek V4 loads the DSpark drafter, and --draft-device places it.
  • LUCE_DS4_SPEC / LUCE_DS4_DRAFT still work when --draft is absent.
  • An explicit --draft that fails to load now stops startup, and an empty --draft is rejected; the env spelling keeps its autoregressive fallback.
• --expert-device backend:gpu is the flag form of LUCE_DS4_MOE_TP=1 _INPROC=1 _GPU=<n> _BACKEND=<b>.
• --profile ds4-strix and --profile ds4-r9700-strix carry the qualified DeepSeek V4 recipes (flags plus, for the R9700 profile, 37 tuning variables).
  • Explicit flags replace a profile's value regardless of order.
  • Variables already set in the environment keep their value, and the startup log names them.

Container entrypoint (rewritten; 543 → ~300 lines)

• serve [flags] and bare flags go to luce_server last, so they win. LUCE_ARGS and LUCE_PROFILE cover the same for compose files. A new devices subcommand runs --list-devices.
• The GPU probe uses luce_server --list-devices, so CUDA and ROCm are handled the same way. The entrypoint exports LUCE_TARGET_DEVICE=auto and sizes the context from the chosen GPU. A profile's own --max-ctx is left alone, and an explicit --max-ctx flag is passed once and shown in the startup log. With --target-devices, the log names that placement.
• Drafts are chosen by architecture: DeepSeek V4 gets *dspark*.gguf and no --ddtree, and projector or DSpark files are never used as DFlash drafts.
• Removed:
  • the HOST_INFO writer, which nothing reads;
  • the lucebox subcommand, whose package is not in the image;
  • forwarding of --think-soft-close-min-ratio and --debug-thinking-logits, which luce_server rejects with "unknown option".

Docs

• The recipes in DS4.md, RECOMMENDED_SETUPS.md and image-input.md now use --draft, --expert-device and --profile, and the README covers Docker passthrough.
• Drift fixed along the way:
  • the --max-ctx default (8192, not 131072) and the --model-name default (luce);
  • a failed fused-decode full load stops startup instead of falling back to hybrid placement;
  • the 3090 + Strix recipe never set LUCE_DS4_SPEC, so its drafter never loaded.

Behaviour changes

• Container: the target goes to the auto device instead of device 0. This only differs when the model does not fit device 0 (per the estimate: weights + 10%, 2–4 GiB, + the KV cache for Qwen3.5/3.6).
• Container: DeepSeek V4 now uses its DSpark drafter when one is in models/draft/, except with --max-concurrency > 1 or --paged-attention: paged DS4 serving is autoregressive and rejects a drafter. ROCm launches now get the same context tiers as CUDA; before, ROCm stayed at 16384.
• Server: --draft on DeepSeek V4 is no longer ignored.
• Server: a profile's environment is installed only for the model block that is loaded.

Validation

All runs were on an R9700 (gfx1201) + Strix Halo (gfx1151) host with DeepSeek-V4-Flash-0731-ROCMFPX-MIX-STRIX.gguf and the DSpark Q4RMFP4 drafter. Baseline = unmodified main built for the same architectures.

Check	Result
Issue repro, main, plain launch	Tries a 93,321 MiB allocation on device 0 and fails
Branch, --target-device auto	Picks hip:1, loads the whole model, serves correctly
Branch, plain launch on device 0	Fails as before, then prints hint: ... hip:1 ... is the better fit
Strix: documented env recipe vs --profile ds4-strix --draft	4/4 greedy outputs identical (including a 2,253-token sparse prefill); same kernel defaults logged; decode 36.6/27.0/38.9/40.6 vs 37.0/27.4/39.0/43.3 tok/s
R9700 + Strix: 43-variable recipe vs --profile ds4-r9700-strix --draft	4/4 outputs identical across 2 runs each; identical placement (1720/9288 experts) and acceptance; mean decode 40.6 and 36.1 vs 37.4 and 37.5 tok/s (per-prompt run-to-run spread is about ±15%)
Profile env vs DS4.md recipe (scripted)	Exactly the 37 variables left after the 6 now expressed as flags, with identical values
Entrypoint, run natively on this host with no arguments	Auto-detected target → hip:1, DSpark drafter, no --ddtree, max_ctx=131072, served a request
ctest -R "server_unit|feature_gate" (HIP)	601/601 pass, including the new test_launch_policy and server_unit_entrypoint

Not covered locally:

• A CUDA build. device_select.cpp has a CUDA branch (sm_XY arch string); CI covers it.
• Building the container image. Docker is not available to this user here, so the entrypoint was exercised natively against the real binary.

Out of scope / follow-ups

• Not converted to profiles: serve_ds4_dual_rocm_128k.sh (RX 7900 + Strix) and the qualification harness keep their env recipes, which still work. I can't qualify an RX 7900 profile here.
• Image contents: the :rocm image does not ship backend_ipc_daemon or build with LUCE_ROCMFP2_AFFINE, so the CUDA + Strix target-shard split and the RX 7900 profile still need a native build.
• Harness launchers: harness/clients/common.sh and server/scripts/run.py keep their own launch and draft resolution.

🤖 Generated with Claude Code

[Graffioh]

@cubic-dev-ai review this PR

[cubic-dev-ai]

@cubic-dev-ai review this PR

@Graffioh I have started the AI code review. It will take a few minutes to complete.

[cubic-dev-ai]

All reported issues were addressed across 22 files

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

All reported issues were addressed across 23 files

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>