Fix MoE expert decompression for non-32-divisible bit widths (#47315)
`DecompressExperts.convert` rebuilt each expert's unpacked in-dim as
`packed_cols * (32 // num_bits)`. compressed-tensors uses dense bit-packing,
so this is only exact when `num_bits` divides 32. For 3/5/6/7-bit weights it
under-counts: e.g. a 3-bit `down_proj` packed as (2048, 48) reconstructs an
in-dim of `48 * 10 = 480` instead of the true 512, producing wrongly shaped
expert weights and a `torch._grouped_mm` "contraction dimension of mat_a and
mat_b must match" crash at load. 4-bit and 8-bit are unaffected.
The checkpoint already stores an exact per-expert `weight_shape`, and it is
routed into `convert` via the `.weight_shape$` source pattern. Prefer it, and
fall back to the packed-tensor reconstruction only when it is missing/empty
(e.g. left empty on most ranks under TP/EP sharding).
Verified on a 3-bit (W3A16) AWQ GlmMoeDsa checkpoint: loading goes from a hard
crash to perplexity 1.0002 vs. 1.0000 for the bf16 base on the same text.
Signed-off-by: Karthik Kothuri <karthikkothuri2009@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
transformers
d0db27f8 - Fix MoE expert decompression for non-32-divisible bit widths (#47315)
