onnxruntime
cf509d84 - QMoE: fail loudly when weights_prepacked=0 but PrePack did not run (#28965)

Commit
28 days ago
QMoE: fail loudly when weights_prepacked=0 but PrePack did not run (#28965) ### Description When a QMoE model sets `weights_prepacked=0` (raw `[E, N, K/pack]` int weights) and the session has `session.disable_prepacking`, `PrePack()` never runs, so `packed_fc{1,2}_weights_` stay null and `int_weights_consumed_by_prepack` is false. The code then falls through to the raw initializer pointers — but those bytes are not in CUTLASS layout, so the runner consumes them as-if-prepacked and produces silently wrong output with no diagnostic. Changes in `onnxruntime/contrib_ops/cuda/moe/moe_quantization.cc` (`QMoE::ComputeInternal`): - **Int path**: Added a defensive `INVALID_ARGUMENT` guard — when `is_int && !weights_prepacked_` but either prepack buffer is null, return a clear error instead of feeding non-CUTLASS bytes to the runner. - **wfp4afp8 native path**: Same fall-through (`packed_fp4_fc{1,2}_weights_ ? ... : raw`) replaced with an explicit guard that errors when the repacked FP4 buffers were not produced. Also added a focused regression test in `onnxruntime/test/contrib_ops/moe_test.cc` covering `quant_type='int'` with `weights_prepacked=0` and `session.disable_prepacking=1`, asserting that QMoE fails with an actionable error instead of producing output. Merged the branch with the latest `main`. ### Motivation and Context A prior fix removed the null-pointer crash on this path but left a misleading-success outcome that is newly user-reachable via the `weights_prepacked=0` contract — the exact silent-failure mode the offline-path work set out to eliminate. These guards convert that into a loud, actionable error. The wfp4afp8 branch shares the same fall-through and is hardened for consistency. The added regression test ensures this fail-loudly behavior remains covered going forward, especially when prepacking is disabled at the session level. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Author
Parents
Loading