PR #29038 [CUDA] QMoE GEMV fast path for batch-1 decode

QMoE GEMV kernel

tianleiwu committed 43 days ago

address feedbacks

tianleiwu committed 43 days ago

gemv for group wise quantization

tianleiwu committed 43 days ago

support bf16

tianleiwu committed 43 days ago

support int8 per-column

tianleiwu committed 43 days ago

support block size 32

tianleiwu committed 42 days ago

Fix buffer aliasing and Use fp32 accumulation

tianleiwu committed 41 days ago

lintrunner

tianleiwu committed 41 days ago

onnxruntime [CUDA] QMoE GEMV fast path for batch-1 decode #29038 Merged