PR #19272 [MLAS AArch64] SQNBitGemm optimization

[MLAS AArch64] SQNBitGemm optimization #19272

edgchen1 merged 12 commits into main from edgchen1/sqnbitgemm_subblklen_32

add impl for block len >= 32 that processes 32 block elements at once

5b1a37bb

pack b 32 elements at a time only for compint8

0be671ac

Merge remote-tracking branch 'origin/main' into edgchen1/sqnbitgemm_s…

1893302a

move mask constants out of loop

90efbcfc

revert unneeded changes

8203815b

edgchen1 requested a review 2 years ago

edgchen1 requested a review from

chenfucn 2 years ago

edgchen1 requested a review from

yufenglee 2 years ago

edgchen1 requested a review from

yihonglyu 2 years ago

fix return

f90c6816

yufenglee commented on 2024-01-26

yufenglee dismissed these changes on 2024-01-26

add template argument HasZeroPoint

ccc54440

add single threaded benchmark

38de6c37

Merge remote-tracking branch 'origin/main' into edgchen1/sqnbitgemm_s…

adc91ca3

Revert "add template argument HasZeroPoint"

47bcd5fb

Reapply "add template argument HasZeroPoint"

a4b197c2

don't inline impl functions

1a076647

edgchen1 dismissed their stale review via 1a076647 2 years ago

edgchen1 changed the title ~~[MLAS AArch64] SQNBitGemm - Add support for packing 4-bit values 32 at a time for CompInt8~~ [MLAS AArch64] SQNBitGemm optimization 2 years ago

edgchen1 requested a review from

yufenglee 2 years ago

yufenglee approved these changes on 2024-01-30

edgchen1 merged c379a89b into main 2 years ago

edgchen1 deleted the edgchen1/sqnbitgemm_subblklen_32 branch 2 years ago

Reviewers

yufenglee

chenfucn

yihonglyu

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

onnxruntime [MLAS AArch64] SQNBitGemm optimization #19272 Merged

[MLAS AArch64] SQNBitGemm optimization #19272

onnxruntime
[MLAS AArch64] SQNBitGemm optimization
#19272

Merged