onnxruntime
[CUDA] Batched small-M GEMV for MatMulNBits (4-bit/8-bit)
#29451

Merged

[CUDA] Batched small-M GEMV for MatMulNBits (4-bit/8-bit) #29451

jambayk merged 6 commits into main from jambayk/mnb-small-m-gemv

Add standard-layout small-M batched GEMV for CUDA MatMulNBits

8d0bbbba

Add MatMulNBits small-M profiling script and experiment log

c1a7a07c

github-actions commented on 2026-06-30

Add small-M batched GEMV for 8-bit MatMulNBits

0278001d

Fix bf16 scales in MatMulNBits profiler

23a4c562

jambayk requested a review from

copilot-pull-request-reviewer 12 days ago

copilot-pull-request-reviewer commented on 2026-06-30

Correct MatMulNBits small-M comments and doc intro

9ef33405

jambayk marked this pull request as ready for review 12 days ago

jambayk requested a review from

tianleiwu 12 days ago

tianleiwu commented on 2026-07-01

Loosen small-M shared-memory gate and add batched-tile tests

c43d3764

jambayk requested a review from

tianleiwu 11 days ago

tianleiwu approved these changes on 2026-07-01

jambayk merged 0b5580da into main 11 days ago

jambayk deleted the jambayk/mnb-small-m-gemv branch 11 days ago

Reviewers

tianleiwu

github-actions

Assignees

No one assigned

Labels

None yet

Milestone

No milestone