Quantized MatMul - Modular MLAS API Changes for KleidiAI
#25187

Merged

KleidiAI SGEMM/IGEMM/Quantized MatMul - Modular MLAS API Changes for KleidiAI #25187

hariharans29 merged 53 commits into microsoft:main from damdoo01-arm:kai_sgemm_igemm_quant_gemv

damdoo01-arm marked this pull request as draft 360 days ago

edgchen1 commented on 2025-06-27

CLNTFRAME-376: Add initial pipeline setup

b8d540d9

Integrate initial KFI changes

d69d3f5f

updated build and test to have mac stages

9e566649

Sync with latest from old repo

5c03bcd1

Added Dynamic-Quantized Matmuls and GEMV

48b09e36

Fixed copyright attribution

972eef5e

KFI-51 Requires target "kleidiai" error building ONNX RT on aarch64.

baa63df1

damdoo01-arm force pushed from 0b628b90 to ac8b6730 352 days ago

damdoo01-arm marked this pull request as ready for review 352 days ago

damdoo01-arm marked this pull request as draft 352 days ago

damdoo01-arm force pushed from ac8b6730 to baa63df1 352 days ago

MLAS API updates, mlas test fixes and ORT test fixes

a4068c12

damdoo01-arm marked this pull request as ready for review 351 days ago

Remove Arm CI internal directory inadvertently pushed previously

3b347669

Fix to iOS build

44199a57

2nd attempt to fix ios build by force disabling KAI

459acf80

Wrap preprocessor ifdefs around dedicated KAI lib

c675ccdf

Lint fixes

8b8e6a03

Added Android/Linux CI build fixes plus fixed a layer parser fix

91008e9f

Fix to 2 more CI failures. 1. kleidiai dir not visible in some builds…

f984e816

Remove badly named directory

25e98157

Renamed kleidiai dir in lower case

3b9fd9aa

Merge branch 'main' into kai_sgemm_igemm_quant_gemv

fa558b3a

QGemm call fixes that resolve the failing tests in CPU_U8S8_Precision…

db144a03

RESTRICT define created to overcome syntactical differences between c…

38083e65

Resolve conflicts in mlasi.h

8285fcf8

Added support to restrict keyword for msvc in KAI convolve function

967e3376

Attempt to fix MSVC build by correctly linking kleidiai static library

ff5c2ea5

Syntax fix on prev

8e6f4ead

damdoo01-arm force pushed from 0156a8e1 to 939e012c 341 days ago

Reverting back to where we were before MSVC work

d38d63d1

damdoo01-arm force pushed from 5016e718 to d38d63d1 341 days ago

Staged for MSVC changes (now Linux/MacOS compatible)

dead330b

Workaround pending KleidiAI release

13a1c2bc

Merge branch 'microsoft:main' into kai_sgemm_igemm_quant_gemv

8471aa4c

Merge branch 'microsoft:main' into kai_sgemm_igemm_quant_gemv

6da3217d

Reverted to KleidiAI 1.9 removing SME support for MSVC. Fixed duktape…

34ec660e

jywu-msft requested a review from

hariharans29 337 days ago

jywu-msft added KleidiAI

KleidiAI build in Android

be0b6bfd

edgchen1 commented on 2025-07-21

github-advanced-security commented on 2025-07-21

hariharans29 commented on 2025-07-21

edgchen1 commented on 2025-07-21

hariharans29 commented on 2025-07-21

Checkpointing lint fixes after android enablement and addressing comm…

b82f64b6

hariharans29 commented on 2025-07-22

Added MSVC 1944 flag and fixed syntax issue on quantize matmul c++ ca…

810ee40b

Reverted suggestion for 1944 gate, not possible without KAI release

6ef54d92

edgchen1 commented on 2025-07-22

hariharans29 commented on 2025-07-22

Most of the responses to comments (small number outstanding)

1169ccc2

hariharans29 commented on 2025-07-22

github-advanced-security commented on 2025-07-23

hariharans29 commented on 2025-07-23

Unit test updates and lint fixes (more to follow)

db2feff6

DynQuantMatMul enhancements

1aa31f9f

Corrected typo

62175e82

damdoo01-arm commented on 2025-07-22

Added further comments

8e277e34

edgchen1 commented on 2025-07-23

Lint fix

72dd8770

edgchen1 commented on 2025-07-23

Temporary commit to check changes

795ed285

Refactor overrides for kleidi func calls and included fix for matmul …

baa2eb1b

Merge branch 'microsoft:main' into kai_sgemm_igemm_quant_gemv

9c7360e4

Reinstated SME checks where still required

00fda4a6

Added Dynamic Quantization Unit Test

43799128

Fixed lint issues

74e8bf11

Further updates to comments, dyn quant matmul changes only pending

e9c72916

edgchen1 commented on 2025-07-24

jywu-msft added release:1.23.0

Dynamic Quant Matmul changes based on feed back

b201e5ed

Tidy-up based on MS feedback

dba3f0b7

Lint fixes

52961eab

Added Jonathan's guard conditions for DynQuantMatMul

8523708c

Fixed unused variable error after guard include

d45c6bb1

Removed global variable and fixed transA override (removed inadverten…

47535127

edgchen1 commented on 2025-07-24

edgchen1 approved these changes on 2025-07-25

hariharans29 approved these changes on 2025-07-25

hariharans29 merged cd450d15 into main 331 days ago

snnn removed release:1.23.0

Reviewers

hariharans29

edgchen1

JonathanC-ARM

github-advanced-security

Assignees

No one assigned

Labels

KleidiAI

Milestone

No milestone

onnxruntime KleidiAI SGEMM/IGEMM/Quantized MatMul - Modular MLAS API Changes for KleidiAI #25187 Merged

KleidiAI SGEMM/IGEMM/Quantized MatMul - Modular MLAS API Changes for KleidiAI #25187

onnxruntime
KleidiAI SGEMM/IGEMM/Quantized MatMul - Modular MLAS API Changes for KleidiAI
#25187

Merged