Skip to content

Add a CPU runtime contract test for mixed-width QMoE #745

Description

@titaiwangms

Motivation

#744 adds Mobius export support for mixed-width com.microsoft::QMoE, including the target layout:

  • FC1/FC3 routed expert weights: INT2
  • FC2 routed expert weights: INT4

microsoft/onnxruntime#32697 added the corresponding FC-specific schema attributes and packed-layout validation. CPU, CUDA, and WebGPU intentionally return NOT_IMPLEMENTED for mixed-width execution until projection-aware kernel dispatch is implemented.

The current Mobius CI wheel does not contain the new FC-specific attributes, so #744 cannot yet prove that its exported model reaches the intended runtime boundary rather than failing earlier because of an invalid schema, shape, zero point, or initializer binding.

Scope

Add a CPU negative runtime contract test using an ONNX Runtime build that contains microsoft/onnxruntime#32697.

  1. Build or load a tiny Mobius mixed-width QMoE model with FC1/FC3 INT2 and FC2 INT4.
  2. Load it with CPUExecutionProvider.
  3. Verify the model passes schema and packed-layout validation.
  4. Run the model far enough to reach the CPU QMoE execution boundary.
  5. Assert that execution fails with the expected mixed-width NOT_IMPLEMENTED status, not an earlier malformed-model error.
  6. Record or gate on an ORT version/commit containing the FC-specific QMoE attributes.

Acceptance criteria

  • The test exercises the final serialized model and bound initializers produced by Mobius, not only an isolated operator fixture.
  • FC1, FC2, and FC3 effective widths and packed tensor shapes are checked through ORT validation.
  • A missing attribute, invalid shape, invalid zero-point geometry, or unbound initializer causes the test to fail rather than being accepted as the expected result.
  • The only accepted runtime failure is the explicit mixed-width QMoE NOT_IMPLEMENTED boundary from the CPU provider.
  • The test is clearly documented as a negative contract test and makes no numerical-parity or execution-support claim.
  • Once ORT CPU mixed-width execution is implemented, replace or extend this with positive logits/parity coverage.

Dependencies

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions