vllm
4fb4e753 - [Model][Perf] Fuse Beam's TP all-reduce with its sandwich norms and MoE finalize

Commit
33 days ago
[Model][Perf] Fuse Beam's TP all-reduce with its sandwich norms and MoE finalize MNNVL one-shot/two-shot all-reduce built on FlashInfer's Lamport protocol, with Beam's FP32 norm-residual-norm epilogue and an optional top-k MoE finalize before the reduction. FusedMoEConfig gains defer_moe_finalize(allow_ep=True) so the finalize can also be deferred under replicated-token EP (TP=1, EP>1). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Author
Committer
Parents
Loading