bugfix: fix qwen3.5 dp empty shard crash across dense and moe.

[Enguikong]

Description

Under Data Parallel (dp_size > 1), request scheduling across DP replicas can leave a replica with zero sequences in a forward step. To keep HCCL collectives in lockstep, worker_impl pads such an empty shard with a single fake token. This works for plain-attention models, but Qwen3.5's hybrid Gated Delta Net (GDN) path does not account for the fake shard — its GDN state tensors (kv_cache_tokens_nums, linear_state_ids, …) are left undefined and then read during forward, crashing the worker.

Crash signature (the rank that got the empty shard dies first; rank 0 cascades):

rank N (empty shard):
  terminate called after throwing 'c10::Error'
  frame #5: at::_ops::gt_Scalar::call(...)      # reads undefined GDN tensor
rank 0 (cascade):
  F batch_input_builder.cpp:688] Check failed: q_seq_len > 0 (0 vs. 0)

Full crash stack (rank with empty shard)

terminate called after throwing an instance of 'c10::Error'
  what():  Expected a proper Tensor but got None (or an undefined Tensor in C++) for argument #0 'self'
Exception raised from checked_cast_variable at /pytorch/torch/csrc/autograd/VariableTypeManual.cpp:66
frame #0: c10::Error::Error(...) in libc10.so
frame #1: c10::detail::torchCheckFail(...) in libc10.so
frame #5: at::_ops::gt_Scalar::call(at::Tensor const&, c10::Scalar const&) in libtorch_cpu.so
frame #6-13: xllm (GDN forward -> worker execute)
frame #14: libhccl.so

The MoE path fails separately: FusedMoEImpl::forward feeds dp_global_token_nums (still 0 for the padded rank) into parallel_state::gather, which fails CHECK_EQ(local_rows, token_num_list[rank]).

The balanced-DP path (all ranks busy) never triggers this, so single-request smoke tests pass and the bug only surfaces under concurrency.

Fix — four places, +49/-8, no behavior change on dp=1 (every guard is gated on the empty-shard / is_dummy condition, which never fires when dp_size == 1):

• worker_impl.cpp: early-return prepare_input_params_for_linear_attention when q_max_seq_len == 0 || num_sequences == 0, before reading undefined GDN tensors. Mirrors AttentionMetadataBuilder::build's is_dummy branch.
• npu_torch/qwen3_gated_delta_net_base.cpp: early-return Qwen3GatedDeltaNetBaseImpl::forward on attn_metadata.is_dummy, returning torch::zeros_like(hidden_states). Placed before the FlashComm1 sequence gather so dummy shards never enter the collective. Mirrors the existing is_dummy early-return in npu_torch/attention.cpp.
• llm/qwen3_next_hybrid_base.h: pass device_ into AttentionMetadataBuilder::build so the is_dummy branch can construct fake slot_mapping / q_seq_lens / kv_seq_lens tensors on the right device. Uses the builder's existing device parameter (default std::nullopt); other callers unaffected.
• npu_torch/fused_moe.cpp: add file-local helper make_moe_dp_token_nums that normalizes zero entries in dp_global_token_nums to one. All ranks apply the same 0→1 normalization so the token list is consistent across the DP group; the fake row is dropped by the existing per-rank slice-back. Only FusedMoEImpl::forward is touched (the ep>1 paths are out of scope).

Relationship to #2024: #2024 (preserve raw dp token counts for empty shards) addresses the same empty-shard root cause but on a different path — it adds raw_dp_global_token_nums for DpEpPadding (EP padding / lm-head output compaction / reduce-scatter). This PR is complementary: it fixes the GDN forward, the linear-attention preprocessing, and the FusedMoEImpl::forward DP gather, none of which #2024 touches. On current main, FusedMoEImpl::forward still gathers with the un-normalized dp_global_token_nums, so this fix is still required.

Verification (Ascend 910C):

• Repro: unfixed base → dp=2 4-concurrent = 4/4 empty-reply + c10::Error; with fix → 4/4 HTTP 200, both ranks alive.
• 128-concurrent, max_tokens=64, warmup: 128/128 HTTP 200, zero crash on 4B/9B/27B (dense) and 35B-A3B (MoE, ep=1), at dp=2 and dp=4, world ∈ {2,4}.
  • Dense DP speedup: 4B dp=2 1.39x / dp=4 1.76x; 9B 1.33x / 1.65x; 27B(tp=2) dp=2 1.41x.
  • Prefix cache compatible: shared 500-token-prefix workload reaches dp=4 3.53x (88% efficiency); zero regression across 9 runs.
  • Same-TP PD disaggregation verified end-to-end (Prefill + Decode via xllm-service): instances link, 4/4 HTTP 200, zero crash.

Scope / limitations:

• Text-only DP (dense/MoE via --backend=llm). Qwen3.5 VL does not route through the DP encoder path — dp=1 recommended for VL until addressed separately.
• Only FusedMoEImpl::forward is patched; expert-parallel (ep>1) paths unchanged.
• 35B-A3B was measured at dp=1/tp=4 vs dp=2/tp=2 (single-card OOM prevents a tp-aligned baseline), so its wall-clock is "correct under DP+MoE", not a clean DP-only speedup.
• Repo x86/ARM NPU smoke CI covers only Qwen2-7B single-card, so it does not exercise Qwen3.5 DP/MoE; the evidence above is from manual 910C runs.

Related Issues

N/A — no existing tracking issue. Complementary to #2024 (see Description).

Change Type

• Bug fix
• New feature
• Performance improvement
• Refactor
• Documentation
• Test
• Build or CI

Pull Request Checklist

PR Title and Commit Messages

• The PR title and each commit message follow the xLLM commit format: <type>: <subject>.

Pre-commit Checks

• I have installed pre-commit by running pip install pre-commit or an equivalent command.
• I have installed the hooks with pre-commit install.
• I have run pre-commit run --all-files and fixed any reported issues.

The pre-commit clang-format hook could not build its virtualenv on this dev machine. Style was verified manually with clang-format 20.1.6 (the exact version pinned in .pre-commit-config.yaml); all four changed files report no formatting diff. Will re-run the full hook once the environment is fixed.

Self Review

• I have self-reviewed the code according to .agents/skills/code-review/references/custom-code-style.md, especially code written or assisted by AI.
• I have rebased this PR onto the latest main branch.

Branched from c2fc480c. Only 1 commit ahead of main's merge-base; main's newer commits touch fused_moe.cpp (L434 ctor) and worker_impl.cpp (a different function) in non-overlapping regions, so this auto-merges cleanly. Will rebase before merge.

Build and Test Coverage

• Tests have been added or updated as needed.
• CUDA: python setup.py build test has passed on a CUDA machine.
• NPU: python setup.py build test has passed on an NPU machine.
• MLU: python setup.py build test has passed on an MLU machine.

Draft PR. Unit tests are a planned follow-up commit (helpers are structured to be testable). Functionally verified on Ascend 910C as described above; the full build test suite and CUDA/MLU machines are not available on this dev box.

Reviewer Notes

• dp=1 production path is unaffected: every guard is gated on the empty-shard / is_dummy condition, which never fires when dp_size == 1.
• The GDN and MoE guards deliberately mirror the existing is_dummy handling in npu_torch/attention.cpp rather than introducing a new pattern.
• Complementary to bugfix: preserve raw dp token counts for empty shards. #2024 (different code paths — see Description) and no conflict with the heterogeneous-TP PD work in feat: support heterogeneous qwen3.5 PD disaggregation. #1995 (which touches worker_impl.cpp:~1623, a different function).
• Follow-ups: (1) unit tests for make_moe_dp_token_nums + the two early-return guards; (2) VL DP is out of scope, documented as dp=1-recommended.

[Copilot]

Pull request overview

This PR fixes crashes when running Qwen3.5 dense + MoE under Data Parallel (dp_size > 1) where scheduling can produce an empty shard on one DP replica. The fix ensures dummy/padded shards don’t read undefined GDN state and don’t violate collective invariants during forward.

Changes:

• Pass device_ into AttentionMetadataBuilder::build() so its is_dummy branch can materialize placeholder tensors on the correct device.
• Add early-return guards for dummy/empty-shard inputs in linear-attention preprocessing and in Qwen3.5 GDN forward.
• Normalize dp_global_token_nums (0→1) for MoE DP gather to keep gather preconditions consistent with the fake-token padding behavior.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File	Description
xllm/models/llm/qwen3_next_hybrid_base.h	Passes device_ into AttentionMetadataBuilder::build() so dummy attention metadata is created on the right device.
xllm/core/runtime/worker_impl.cpp	Skips linear-attention preprocessing for dummy/empty shards to avoid reading undefined GDN tensors.
xllm/core/layers/npu_torch/qwen3_gated_delta_net_base.cpp	Early-returns zeros_like(hidden_states) when attn_metadata.is_dummy to avoid undefined-state reads and skip FlashComm1 gather.
xllm/core/layers/npu_torch/fused_moe.cpp	Normalizes MoE DP token counts (0→1) before parallel_state::gather() to prevent empty-shard collective failures.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.