You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
microsoft/onnxruntime#32697 provides the schema and validation contract, but mixed-width execution currently returns NOT_IMPLEMENTED. This PR qualifies graph emission, initializer binding, failure behavior, and package roundtrip; it does not claim runtime numerical support.
Follow the ORT QMoE projection-specific bit-width contract for Olive checkpoints with INT2 gate/up experts and INT4 down experts. Preserve legacy uniform INT4 emission and add strict checkpoint, package, and rewrite regression coverage.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: titaiwang <titaiwang@microsoft.com>
The reason will be displayed to describe this comment to others. Learn more.
Copilot review overview
🟡 Changes recommended
Mixed and legacy per-layer QMoE plans can diverge between graph construction and weight preprocessing, causing valid layer combinations or dense fallbacks to fail weight binding.
Get a fresh assessment by requesting another Copilot review.
Resolve every routed layer against its effective Olive layout so mixed and legacy uniform QMoE layers can coexist. Fail closed when packed sidecars target dense fallbacks and honor projection-specific packing geometry.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: titaiwang <titaiwang@microsoft.com>
Reject routed-expert exclusions and projection-specific GPTQ/AWQ layouts that cannot bind the current graph, while preserving redundant dense-fallback overrides. Validate every routed layer's Olive sidecars in mixed plans.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: titaiwang <titaiwang@microsoft.com>
Two-layer QMoE test misses per-layer attributes and initializer bindings
src/mobius/models/moe_test.py:526
This two-layer regression validates only the preprocessed dictionary, so the new per-layer qmoe_source_paths graph wiring could assign mixed-width attributes to the wrong layer while this test still passes. The linked acceptance criterion also requires the multi-layer case to inspect node attributes and final initializer bindings. Please build the two-layer package, associate each QMoE node by its FC1 initializer input, assert that only mixed_layer has the FC-specific bit attributes, apply the processed weights, and compare both layers' bound initializer bytes.
Strengthen the two-layer regression to associate each QMoE node with its FC1 initializer, validate projection-specific bit attributes on only the mixed layer, and compare both layers' bound FC1 and FC2 initializer bytes.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: titaiwang <titaiwang@microsoft.com>
Addressed the previously missed multi-layer coverage in 5fdadb6. The two-layer regression now associates each QMoE node with its layer-specific FC1 initializer through the emitted producer chain, asserts that only the selected mixed layer carries fc1=2/fc2=4/fc3=2 attributes, applies the processed weights, and byte-compares both layers’ bound FC1/FC2 initializers. The focused QMoE suite passes (156 tests).
The reason will be displayed to describe this comment to others. Learn more.
Copilot review overview
🔵 Needs a closer look
GPTQ/AWQ parent overrides are resolved using incompatible Olive matching semantics.
Review effort: Balanced Findings: None
Previously missed (1)
In code that hasn't changed since last review
Use generic subtree matching for GPTQ and AWQ overrides
src/mobius/_weight_utils.py:207
This applies Olive's literal override/substring-exclusion rules to GPTQ and AWQ too. Those formats use QuantizationConfig.for_module subtree matching, so a valid parent override such as model.layers.0.mlp is currently ignored for both expert projections; the graph can retain the global geometry and later mismatch the packed checkpoint instead of failing closed. Delegate non-Olive resolution to the generic matcher before applying Olive-specific semantics.
Delegate GPTQ and AWQ module rules to the generic subtree matcher while preserving Olive's literal override and substring exclusion semantics. Add parent, near-match, and exclusion regressions and document the producer-specific contract.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: titaiwang <titaiwang@microsoft.com>
Addressed review 5284088183 in a0a05d0. Non-Olive QMoE resolution now delegates to QuantizationConfig.for_module, restoring generic plain-path subtree matching for GPTQ/AWQ while leaving Olive literal overrides and substring exclusions unchanged. Added GPTQ/AWQ regressions for a matching parent override, a substring-only near miss, and a parent exclusion; updated the design contract. Validation: 162 QMoE tests passed, mypy and lint passed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Contract
Mixed nodes emit the legacy INT4 fallback together with:
fc1_expert_weight_bits=2fc2_expert_weight_bits=4fc3_expert_weight_bits=2for fused SwiGLUCheckpoint preprocessing fails closed for missing, partial, stale, malformed, or unsupported expert sidecars. Plain and regex overrides follow Olive producer semantics.
Validation
python -m pytest src/mobius/models/moe_test.py src/mobius/models/qwen35_test.py src/mobius/components/_moe_test.py src/mobius/rewrite_rules/_qmoe_fusion_test.py src/mobius/_weight_utils_test.py -q --tb=short— 241 passedpython -m mypy src/mobius/_weight_utils.pypassedgit diff --checkpassedRuntime boundary
microsoft/onnxruntime#32697 provides the schema and validation contract, but mixed-width execution currently returns
NOT_IMPLEMENTED. This PR qualifies graph emission, initializer binding, failure behavior, and package roundtrip; it does not claim runtime numerical support.Addresses #735.