vllm
4f57aa65 - Fix EPLB + NVFP4: exclude broadcast scales and fix stale quant config

Commit
200 days ago
Fix EPLB + NVFP4: exclude broadcast scales and fix stale quant config Cherry-pick of PR #34646 (elvircrn/fix-eplb-nvfp4-contiguous). Pre-compute g1/g2 alphas as registered parameters so EPLB rearranges them alongside expert weights. Without this, the quant config caches g1_alphas = a_scale * w_scale_2 once at init, and EPLB's in-place rearrangement of w_scale_2 leaves the cached product stale. Also excludes broadcast activation scales (w13_input_scale, w2_input_scale) from EPLB to prevent contiguity assertion crash (these are expanded stride-0 tensors from .max().expand()). Co-Authored-By: Elvir Crncevic <elvircrn@users.noreply.github.com> Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Author
Parents
Loading