PR #29038 [CUDA] QMoE GEMV fast path for batch-1 decode

[CUDA] QMoE GEMV fast path for batch-1 decode #29038

tianleiwu merged 8 commits into main from tlwu/20260613/qmoe_gemv

QMoE GEMV kernel

a4474774

tianleiwu requested a review from

copilot-pull-request-reviewer 39 days ago

copilot-pull-request-reviewer commented on 2026-06-13

address feedbacks

a37ac011

gemv for group wise quantization

c51e7b4a

support bf16

9f701737

support int8 per-column

b9c35807

tianleiwu requested a review from

copilot-pull-request-reviewer 38 days ago

tianleiwu changed the title ~~Add INT4 per-channel MoE GEMV fast path for batch-1 decode~~ QMoE GEMV fast path for batch-1 decode 38 days ago

tianleiwu changed the title ~~QMoE GEMV fast path for batch-1 decode~~ [CUDA] QMoE GEMV fast path for batch-1 decode 38 days ago

copilot-pull-request-reviewer commented on 2026-06-14

support block size 32

a8088b9d

tianleiwu requested a review from

hariharans29 37 days ago

tianleiwu requested a review from

apsonawane 37 days ago

tianleiwu requested a review from

kunal-vaishnavi 37 days ago

tianleiwu added release:1.27.0

Fix buffer aliasing and Use fp32 accumulation

f7b900e8

github-actions commented on 2026-06-15

github-advanced-security commented on 2026-06-15

lintrunner

79cb66ad

sanaa-hamel-microsoft approved these changes on 2026-06-15

tianleiwu merged 56f6fee6 into main 37 days ago

tianleiwu deleted the tlwu/20260613/qmoe_gemv branch 37 days ago

tianleiwu removed release:1.27.0

tianleiwu added release:1.27.1

Reviewers

sanaa-hamel-microsoft

github-actions

github-advanced-security

copilot-pull-request-reviewer

hariharans29

apsonawane

kunal-vaishnavi

Assignees

No one assigned

Labels

release:1.27.1

Milestone

No milestone

onnxruntime [CUDA] QMoE GEMV fast path for batch-1 decode #29038 Merged

[CUDA] QMoE GEMV fast path for batch-1 decode #29038

onnxruntime
[CUDA] QMoE GEMV fast path for batch-1 decode
#29038

Merged