onnxruntime
[CUDA] Add fast INT3 QMoE kernels and offline SM80 INT2/INT4 prepack support
#33569

Merged

[CUDA] Add fast INT3 QMoE kernels and offline SM80 INT2/INT4 prepack support #33569

jiafatom merged 6 commits into main from tlwu/20261023/qmoe-int3-int2-prepack-cuda

[CUDA] Add fast INT3 QMoE kernels and offline SM80 INT2/INT4 prepack …

a8cc7d36

tianleiwu requested a review from

copilot-pull-request-reviewer 19 days ago

copilot-pull-request-reviewer commented on 2026-10-23

tianleiwu marked this pull request as draft 19 days ago

fix(cuda): address QMoE INT3/INT2 prepack review feedback

06107452

perf(cuda): widen QMoE decode GEMV to 80 expanded rows and fused rout…

665afae3

Merge remote-tracking branch 'origin/main' into pr33569-work

4ba6c383

github-actions commented on 2026-10-24

Optimize moe_gemv CtaM=1 FP32 dot product and Int4 M=8 small-N dispatch

26efaff5

fix(cuda): address PR review feedback for QMoE INT3/INT2 prepack

0e864e08

tianleiwu marked this pull request as ready for review 18 days ago

jiafatom approved these changes on 2026-10-25

jiafatom merged bd6010c0 into main 18 days ago

jiafatom deleted the tlwu/20261023/qmoe-int3-int2-prepack-cuda branch 18 days ago

Reviewers

jiafatom

github-actions

Assignees

No one assigned

Labels

None yet

Milestone

No milestone