onnxruntime
d65da43f - Add opt-in post-SiLU clipping to MoE operators (#33954)

Commit
2 days ago
Add opt-in post-SiLU clipping to MoE operators (#33954) ### Description Add opt-in `swiglu_limit_after_activation` to MoE/QMoE for models that clip the gate after SiLU. Existing clipping behavior remains the default. The change covers supported floating, integer and block-FP8 CUDA paths, rejects unsupported combinations explicitly, and adds numerical/default-behavior tests and operator documentation. Validation: 18 clipping tests / 55 subtests passed; the full FP8 suite passed 63 tests / 66 subtests with one existing opt-in memory skip. Forced dense, GEMV and WMMA cases were verified with CUDA kernel traces. ### Motivation and Context Ling-3.0-flash-VL requires post-SiLU clipping rather than the existing pre-SiLU order. Stacked on microsoft/onnxruntime#33817. This PR adds only the clipping follow-up and its tests. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a32500b6-be93-4206-a257-11406b00a3f3
Author
Parents
Loading