[CUDA] Size QMoE split-K SwiGLU GEMV partials by GEMV rows, not prefill rows (#34168)
### Description
CutlassMoeFCRunner reserves an FP32 split-K partials buffer for the
fused SwiGLU MoE GEMV (FP16 activations, INT4 weights, swiglu_fusion ==
1, see MoeGemvSplitK2SwiGLUSupported) and sized it for every expanded
row of the call. Its only reader is the split-K two-pass GEMV. That path
runs only when is_moe_gemv_supported() accepts the problem, which needs
expanded_num_rows <= moe_gemv::kMaxProfiledExpandedRows (80), and only
with FP32 GEMV accumulation (ORT_MOE_GEMV_FP32_ACCUM=1). Above 80
expanded rows the grouped GEMM runs and the buffer is never read, so at
prefill sizes the workspace carried a buffer that grows with the prompt
and is never used.
This change caps the partials at 80 expanded rows. Calls with up to 80
expanded rows get exactly the same size as before; larger calls keep a
fixed 2 * 80 * 2 * inter_size * 4 bytes (0.94 MiB for inter_size 768).
Tests:
- QMoEWorkspaceTest.SplitKSwiGluPartialsDoNotScaleWithPrefillRows fails
on main (per-token growth at GEMV sizes 176,640 B, needs at least
225,857 B) and passes with this change.
- QMoEWorkspaceTest.SplitKSwiGluPartialsCoverGemvRows checks that
GEMV-sized calls still get the full buffer.
Validation on A100 80GB (SM80), using layer 0 of
Qwen3-30B-A3B-Instruct-2507 (INT4, block 128, top-8, 128 experts) as a
standalone QMoE model, main vs this branch:
- Outputs are bitwise identical at 8, 80, 88 and 1024 expanded rows,
both with ORT_MOE_GEMV_FP32_ACCUM=1 and with the default.
- With ORT_MOE_GEMV_FP32_ACCUM=1, Nsight Systems shows
moe_gemv_splitk_partials_kernel and moe_gemv_splitk_reduce_swiglu_kernel
launching at 8 and 80 expanded rows, and no GEMV kernels at 88 and 1024.
- compute-sanitizer memcheck reports 0 errors on those runs. memcheck
alone would not catch an undersized buffer, because the partials share
one workspace allocation with the other buffers. As a control, halving
the cap changes the output at 80 expanded rows while memcheck still
reports 0 errors, so the parity check is the meaningful one.
- End to end with the full model through ORT GenAI at a 32,640-token
prompt and ORT_MOE_GEMV_FP32_ACCUM=1 (one process per build): process
peak 47,248 MiB on main vs 43,152 MiB with this change, identical
generated tokens, decode 126.2-126.6 vs 126.3-126.6 tokens/s. The buffer
is reserved the same way with the default accumulation. An earlier run
of the same change on main at e4234e5617 measured a peak of 47,254 vs
43,158 MiB in that mode (two processes per build, identical in each).
### Motivation and Context
For Qwen3-30B-A3B at a 32,640-token prompt the partials buffer was 2 *
261,120 * 1,536 * 4 B = 3,059 MiB of workspace that was never read.
Limits:
- Only the FP16 + INT4 + fused-SwiGLU runner allocates this buffer, so
other dtypes and activations are unaffected.
- The small capped buffer is still reserved when FP32 accumulation is
off. Dropping it entirely would tie the workspace layout to that
environment variable, so this PR leaves it.
- Tested on SM80 Linux only. CI covers SM86 and Windows.
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>