PR #27364 QMoE CPU Performance Update (Up to 4x on 4-bit)

QMoE CPU Performance Update (Up to 4x on 4-bit) #27364

tianleiwu merged 12 commits into main from tlwu/20260216/qmoe_cpu_q4_perf

Fix QMoE CPU

ed0478ad

prepack and cache to improve QMoE perf

2c838293

move cache to prepack; allow q4gemm for block_size=32

2b256017

Turn on q4 gemm by default if there is no accuracy loss

e9dcd411

refine tests

dec88677

tianleiwu requested a review from

apsonawane 170 days ago

tianleiwu requested a review from

copilot-pull-request-reviewer 170 days ago

copilot-pull-request-reviewer commented on 2026-02-17

tianleiwu added release:1.24.2

tianleiwu removed release:1.24.2

refactor CheckInputs

4af13153

fix build

23cdf2d9

robust shape storage

6d351fed

Merge remote-tracking branch 'origin/main' into tlwu/20260216/qmoe_cp…

100738b8

refactoring

0b4c783a

apsonawane commented on 2026-02-20

review feedback

abef62be

tianleiwu requested a review from

apsonawane 166 days ago

tianleiwu requested a review from

copilot-pull-request-reviewer 166 days ago

copilot-pull-request-reviewer commented on 2026-02-20

address AI feedback

11d30ce0

apsonawane approved these changes on 2026-02-20

tianleiwu enabled auto-merge (squash) 166 days ago

tianleiwu merged b2a6e69e into main 165 days ago

tianleiwu deleted the tlwu/20260216/qmoe_cpu_q4_perf branch 165 days ago

tianleiwu added release:1.24.3

tianleiwu removed release:1.24.3

Reviewers

apsonawane

copilot-pull-request-reviewer

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

onnxruntime QMoE CPU Performance Update (Up to 4x on 4-bit) #27364 Merged

QMoE CPU Performance Update (Up to 4x on 4-bit) #27364

onnxruntime
QMoE CPU Performance Update (Up to 4x on 4-bit)
#27364

Merged