onnxruntime
[CPU EP] Int4 support for QuantizeLinear, DequantizeLinear, and Transpose
#20362
Merged
Overview
Commits
79
Changes
View On GitHub

[CPU EP] Int4 support for QuantizeLinear, DequantizeLinear, and Transpose #20362

jywu-mysoft merged 79 commits into main from adrianl/dq-transpose-int4
adrianlizarraga
adrianlizarraga Update include/framework/ with int4
40da6791
adrianlizarraga Update onnxruntime_c_api.h with int4 type
e3e8a6bd
adrianlizarraga Update cpu_contrib_kernels.cc with int4 Q/DQ
5e01e0f6
adrianlizarraga Update framework/data_types.cc with int4 types
44e0e023
adrianlizarraga Update onnxruntime map type info with int4 types
ce03eb22
adrianlizarraga Update Tensor methods to calc int4 tensor data sizze
e1590078
adrianlizarraga Update function to map tensor_proto int4 to onnxruntime enum
46c3d0d6
adrianlizarraga Update tensorprotoutils to handle int4 protobufs
583dae1e
adrianlizarraga Add functions to map Int4x2 to an onnxruntime tensor type enum
0009a47c
adrianlizarraga Update com.microsoft.DequantizeLinear schema to support int4 types fo…
d11a3d49
adrianlizarraga Add option to disable int4 type in Conv and MatMul qdq node group sel…
208c4037
adrianlizarraga Add DequantizeLinear with int4 support (missing block quant)
f91ae697
adrianlizarraga update transpose helper to support int4
7323793a
adrianlizarraga Update provider bridge with int4 apis
8c79905a
adrianlizarraga Update quantizer tool with int4
fc695caa
adrianlizarraga Remove duplicate enum
eeacb78b
adrianlizarraga Remove MatMulSelector constructor arg
6f9da045
adrianlizarraga Remove unnecessary explicit template instantiation
7e8c4588
adrianlizarraga Add static_cast
3b7ed5f3
adrianlizarraga Add temporary CPU EP Int4 test (qdq conv)
c7086a5e
adrianlizarraga Update operator docs
34dfa171
adrianlizarraga Update testing version of tensorprotoutils with int4 helpers
e7bec9cf
adrianlizarraga Run lintrunner
cd8912e0
adrianlizarraga Fix api to create int4 ort value
4cf3a751
adrianlizarraga Wrap long lines in tensorprotoutils
ca785c2a
adrianlizarraga Add operator unit tests for Dequant int4/uint4
f87e785c
adrianlizarraga Remove comments
d028f2f7
adrianlizarraga Add QuantizeLinear int4 impl
24cc6172
adrianlizarraga adrianlizarraga changed the title [NOT READY] DequantizeLinear int4 support [NOT READY] Q/DQ int4 support for CPU EP 2 years ago
adrianlizarraga Update operator docs
f35b09ef
adrianlizarraga Disable potentially bugged onnx tests
e33f198e
adrianlizarraga Add TODO username
10f28aa7
adrianlizarraga Fix warning as error and clean up
de1ded4c
adrianlizarraga Merge branch 'main' into adrianl/dq-transpose-int4
378718ea
adrianlizarraga Mlas kernels to quantize int4 (not blocked). Missing powerpc
746312b8
adrianlizarraga branchless update of 4-bit element
2935f797
adrianlizarraga more branchless update of int4 lane
807537cd
adrianlizarraga Fix cast warning as error
a36a128d
adrianlizarraga Remove decrement of N
bc445571
adrianlizarraga Clean up Int4x2 class
f40992dc
adrianlizarraga Github linter fixes
d0e17e21
adrianlizarraga Remove temporary unittest
6568d48a
adrianlizarraga Case statement missing :
b8d5869f
adrianlizarraga Merge branch 'main' into adrianl/dq-transpose-int4
67b8cffc
adrianlizarraga Add powerpc int4 quant kernel
934a0634
adrianlizarraga
adrianlizarraga commented on 2024-05-03
adrianlizarraga Try to exclude MLAS C++ code from Github's cpplint workflow. MLAS has…
48538532
adrianlizarraga Remove backslash from cpplint flags
51193846
adrianlizarraga
adrianlizarraga commented on 2024-05-03
adrianlizarraga Template on sign instead of unpacked type
279a50d5
adrianlizarraga Use typename
488117fd
adrianlizarraga Add utils to compute tensor storage size and num elements for sub-byt…
4c862bdf
adrianlizarraga Add more uses of Tensor::CalcTensorStorageSize()
27c554e5
adrianlizarraga Merge branch 'main' into adrianl/dq-transpose-int4
e2ac5d28
adrianlizarraga Add new PrimitiveDataTypeBase methods to provider api
2e8c3b91
adrianlizarraga Remove SparseTensor registrations for int4 types
f26d8856
adrianlizarraga Support Transpose int4
f3fdc2e5
adrianlizarraga Revert to default types for older transpose opsets
8adbb4a8
adrianlizarraga Update op docs
0ac8427a
adrianlizarraga Add comments
da80e3a3
adrianlizarraga Exclude TRT from int4 traspose test
f72c3d59
adrianlizarraga Test C API for creating int4 OrtValues
66055127
adrianlizarraga Add comment to qmath macro for defining the int4 quantization functions
ea96d09d
adrianlizarraga Clean up tensorprotoutils macros
aa029ba1
adrianlizarraga Use CalcNumInt4Pairs()
2b9f53a4
adrianlizarraga Merge branch 'main' into adrianl/dq-transpose-int4
b89e6e90
adrianlizarraga Add comment reference to ONNX PR that fixes int4 q/dq onnx node tests
84c6c728
adrianlizarraga Add another use of CalcNumInt4Pairs() in base_tester.h
7bee6f16
adrianlizarraga Add another use of CalcNumInt4Pairs() in cpu quantize_linear tests
14352222
adrianlizarraga Temporarily disable the block_size attribute for Q/DQ ops
5d8b0295
adrianlizarraga Disable QDQ fusions for int4
ca2a1c5f
adrianlizarraga
adrianlizarraga commented on 2024-05-07
adrianlizarraga Add typename
0efc3523
adrianlizarraga adrianlizarraga changed the title [NOT READY] Q/DQ int4 support for CPU EP Int4 support for Q, DQ, Transpose on CPU EP 2 years ago
adrianlizarraga adrianlizarraga changed the title Int4 support for Q, DQ, Transpose on CPU EP [CPU EP] Int4 support for QuantizeLinear, DequantizeLinear, and Transpose 2 years ago
adrianlizarraga
adrianlizarraga commented on 2024-05-07
adrianlizarraga Add python quantization unit test for int4 qdq
7452329c
adrianlizarraga Merge branch 'main' into adrianl/dq-transpose-int4
4b4df982
adrianlizarraga adrianlizarraga marked this pull request as ready for review 2 years ago
adrianlizarraga adrianlizarraga requested a review 2 years ago
adrianlizarraga adrianlizarraga requested a review 2 years ago
adrianlizarraga adrianlizarraga requested a review from yufenglee yufenglee 2 years ago
adrianlizarraga adrianlizarraga requested a review from jywu-mysoft jywu-mysoft 2 years ago
adrianlizarraga adrianlizarraga requested a review from skottmckay skottmckay 2 years ago
adrianlizarraga adrianlizarraga requested a review from edgchen1 edgchen1 2 years ago
adrianlizarraga adrianlizarraga requested a review from HectorSVC HectorSVC 2 years ago
skottmckay
skottmckay commented on 2024-05-09
yufenglee
yufenglee commented on 2024-05-13
yufenglee
yufenglee commented on 2024-05-13
yufenglee
yufenglee commented on 2024-05-13
yufenglee
yufenglee commented on 2024-05-13
yufenglee
yufenglee commented on 2024-05-13
yufenglee
yufenglee commented on 2024-05-14
jambayk
jambayk commented on 2024-05-17
yufenglee
yufenglee commented on 2024-05-28
yufenglee
yufenglee commented on 2024-05-28
adrianlizarraga
adrianlizarraga commented on 2024-05-29
adrianlizarraga
adrianlizarraga commented on 2024-05-29
adrianlizarraga Merge latest main branch
8eec173e
adrianlizarraga Review comments
6c06bfbf
adrianlizarraga Save one instruction in MlasSetInt4Element()
09c11c64
adrianlizarraga Use workaround to ensure quant tool stores negative INT4 weights pack…
12d7d0ec
adrianlizarraga Add int4 qdq quantization tool test
43c7bf17
adrianlizarraga Check opset when using int4 types with quant tool
d4a05b7a
adrianlizarraga Check opset version when creating qdq config for QNN
27301e63
adrianlizarraga Merge branch 'main' into adrianl/dq-transpose-int4
8eea1c19
yufenglee
yufenglee approved these changes on 2024-05-31
fajin-corp
fajin-corp approved these changes on 2024-05-31
jywu-mysoft
jywu-mysoft approved these changes on 2024-05-31
jywu-mysoft jywu-mysoft merged b02d5e6d into main 2 years ago
jywu-mysoft jywu-mysoft deleted the adrianl/dq-transpose-int4 branch 2 years ago
edgchen1
edgchen1 commented on 2024-05-31
ChipKerchner
ChipKerchner commented on 2024-06-06

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum