onnxruntime
[QNN EP Quantization] Add fusion preprocessing to QNN quantization
#18719
Merged
Merged
[QNN EP Quantization] Add fusion preprocessing to QNN quantization #18719
jywu-mysoft
merged
63 commits into
main
from
adrianl/qnn-ep-quant-fusions
First pass at supporting tensor quantization overrides
e434a68e
Run lintrunner
83139ec0
Start creating QNN specific quantization configs
3f6dd878
Try to override for MatMul, Sigmoid, Tanh
ba3c6823
Fix import issue and assert
d499d97d
Fix quant type data type
046d8f44
Add option for qdq contrib ops
98e6b757
Only override weights for LayerNorm and MatMul
f59a8a8b
Separate handling of MatMul and LayerNorm
cca480e2
Set DedicatedQDQPair to false
1f1e5e03
Provide a value for symmetric when overriding weight types
1182893c
Automatically set op_types_to_quantize
40d304cb
Treat bias tensors as weights if the user provides quantization overr…
6a5af05e
Use quant_type variable properly
06ca8d61
Add TODO to remove UseQDQContribOps extra_option
c7cb8993
Update tensor overrides doc comment
b0516519
Ok to quantize Transpose
724f0df9
Merge latest from main
c35d7cf2
Treat initializers for Add, Sub, Mul, Div as weights
aa673b6f
Add qnn quantization directories to python package
476f10a8
Do not treat initializer inputs to Add/Mul as weights
c4523527
Pass in final value of reduce_range to quantize_initializer() function
c0b95f20
Get quant overrides from within quantize_initializer()
84bf83a7
Use quant overrides for per-channel weights
198c21f1
Change qType to quant_type to appease the python linter
97fd781f
Add some validation for tensor quant overrides
f9015bff
Use existing helper to check if input is an initializer
d927c658
Add unit tests for tensor quant overrides (QDQ)
e5db60ce
Fix linter issues
b910f652
Use variable consistently
418d2b9c
Add test for bias overrides
1da0b787
Add more validation tests
49b477a8
Address CodeQL notices
8bb660f3
Merge latest from main
d270e451
Add Gelu fusion to QNN EP-specific quantization utilities
4e18dff3
Run lintrunner
cbe7da43
Add support for ReduceL2 fusion into LpNormalization for QNN EP
b6298028
Run lintrunner
f38ec8b8
Simplify getting constant inputs for ReduceL2 fusion
17361b4a
rename fusion
cf5d7dc3
Add array bounds check for input. Make epsilon a constructor arg
82ce49af
Fix Softmax QDQ handling for 16-bit
c2f7a994
Fix 16-bit computation of scale/zero_point for QDQ Softmax
e973fae6
Add LayerNorm fusion (optional)
f542af45
Fix merge conflicts
f4c85549
Fix merge conflicts
b1cbc24c
Merge latest main
5fa2b7fc
Merge branch 'main' into adrianl/qnn-ep-quant-config
193490d0
Allow overriding initializer rmin, rmax, scale, or zero_point
8d2a1f88
Updated to handle per-channel quantization overrides
fca3ef6d
Clean up
8cbed6a8
Rename variable
a2627457
Add another per-channel quantization override test
bd3974c7
Fix merge conflicts
29551fc6
Merge branch 'main' into adrianl/qnn-ep-quant-fusions
b924be1a
Fix merge conflicts with main
c0b29cf6
Move common fusions up a level
79292630
Add lintrunner
e619254c
Move some model helpers into fusion.py
b0c43c7c
Forgot to refactor one function call
668f3c27
Merge branch 'main' into adrianl/qnn-ep-quant-fusions
34c8b652
Add new py sources to package
3a4051eb
Merge branch 'main' into adrianl/qnn-ep-quant-fusions
c75028bc
HectorSVC
approved these changes
on 2023-12-12
jywu-mysoft
approved these changes
on 2023-12-12
Login to write a write a comment.
Login via GitHub