onnxruntime
[CUDA] Optimize packed QSA scoring and multi-token verification
#33328
Merged

[CUDA] Optimize packed QSA scoring and multi-token verification #33328

kunal-vaishnavi
apsonawane Add CUDA QMoE block-scaled FP8 support with routed-expert compaction
8a332695
apsonawane Fix CUDA QMoE registration for independent FP8 scale types
926f6863
apsonawane Merge main into feature/cuda-qmoe-block-fp8
4b5b7b5d
apsonawane Fix FP8 QMoE layout regression test and documentation
1eb16c89
apsonawane Fix QMoE CI expectations and older PyTorch test handling
bd24e9f6
kunal-vaishnavi Extend routed-expert compaction to NVFP4 QMoE fallback
38560baa
apsonawane Address QMoE block FP8 review
6a6df332
apsonawane Merge origin/main into feature/cuda-qmoe-block-fp8
5457831e
apsonawane Fix QMoE documentation whitespace
3388d912
kunal-vaishnavi Fix FP4 GEMV expert-map fallback for unsupported fused prologues
53a34c29
apsonawane Format QMoE expert-map fallback
0fa899ca
apsonawane Fix QMoE dequant lambda return type
082668fb
kunal-vaishnavi Fix QMoE packed-prefill tiling and CI review regressions
20aa6ffb
kunal-vaishnavi Support offline NVFP4 QMoE weights through weights_prepacked
504238c6
kunal-vaishnavi Fuse block-scaled FP8 QMoE GEMV and GEMM without weight scratch
8d0cc54b
kunal-vaishnavi Preserve FP8 fused layouts and validate CUDA graph replay
ca6ed6fc
kunal-vaishnavi Use native contiguous K-packed NVFP4 GEMV loads
e7e3de4b
kunal-vaishnavi Support 80 expanded rows in native K-packed NVFP4 GEMV
c956a0cb
kunal-vaishnavi Merge origin/main into kvaishnavi/qwen38-nvfp4-offline-packing
8445c4d3
kunal-vaishnavi Optimize packed QSA scoring and short-K native NVFP4 GEMV
d56c2f66
kunal-vaishnavi Reuse verification keys and compact QSA candidates; add opt-in FC1 ex…
15f63727
apsonawane Optimize Qwen sparse attention, prefill convolution and quantized MoE
bd2a9c90
kunal-vaishnavi Optimize grouped sparse attention and CUDA weight-only decode
3aad2c65
kunal-vaishnavi Fix Qwen NVFP4 CI test APIs and review formatting
19cd01c2
kunal-vaishnavi Revert the last three Qwen optimization commits
84d17393
kunal-vaishnavi kunal-vaishnavi marked this pull request as ready for review 34 days ago
kunal-vaishnavi Optimize native NVFP4 GEMV and add parity probes
6ee0c274
kunal-vaishnavi Merge lower NVFP4 GEMV changes into the QSA stack
c00e7085
kunal-vaishnavi kunal-vaishnavi changed the title [CUDA] Optimize packed QSA verification and native NVFP4 GEMV [CUDA] Optimize packed QSA scoring and multi-token verification 34 days ago
kunal-vaishnavi Merge main and address QMoE layout and dispatch review feedback
4a0e901c
kunal-vaishnavi Fix NVFP4 benchmark trailing newline for Ruff
0fa4cc9d
kunal-vaishnavi Fix QMoE C++ and Python formatting reported by CI
81af1c59
sayanshaw24 Merge latest NVFP4 dependency into CUDA optimization stack
c3a17573
sayanshaw24 Merge current main into refreshed CUDA dependency stack
5a850ad5
sayanshaw24 Merge current main into offline NVFP4 dependency
54cad1ff
sayanshaw24 Align packed QSA stack with refreshed offline NVFP4 parent
a5f1db60
kunal-vaishnavi kunal-vaishnavi requested a review from copilot-pull-request-reviewer copilot-pull-request-reviewer 32 days ago
sayanshaw24
copilot-pull-request-reviewer
copilot-pull-request-reviewer commented on 2026-10-15
github-advanced-security
github-advanced-security commented on 2026-10-15
Copilot Merge current main into offline NVFP4 dependency
997b6f60
Copilot Merge cleaned offline NVFP4 target into packed QSA
7c6c19e9
Copilot Fix packed QSA replay grids and causal boundary coverage
c8347f5d
kunal-vaishnavi kunal-vaishnavi force pushed from ea4f74cc to c8347f5d 31 days ago
Copilot Fix consistent QSA probe launch grid tuple shape
f98b7deb
Base automatically changed from kvaishnavi/qwen38-nvfp4-offline-packing to main 31 days ago
Copilot Merge main into packed QSA optimization branch
b2dc5782
tianleiwu
tianleiwu approved these changes on 2026-10-16
kunal-vaishnavi kunal-vaishnavi merged b016c164 into main 30 days ago
kunal-vaishnavi kunal-vaishnavi deleted the kvaishnavi/optimize-packed-qsa branch 30 days ago

Login to write a write a comment.

Login via GitHub

Assignees
No one assigned
Labels
Milestone