PR #5595 Quantize LSTM - SemanticDiff

Quantize LSTM #5595

yufenglee merged 21 commits into master from roli/rnn_quant

yufenglee requested a review 5 years ago

yufenglee changed the title ~~(WIP) Quantize LSTM~~ Quantize LSTM 5 years ago

yufenglee force pushed to 24caca41 5 years ago

yufenglee force pushed from 24caca41 to a3f1a12d 5 years ago

tracysh commented on 2020-11-17

zhanghuanrong requested a review from

zhanghuanrong 5 years ago

zhanghuanrong dismissed these changes on 2020-11-18

yufenglee dismissed their stale review 5 years ago

yufenglee force pushed from ad93cfa2 5 years ago

zhanghuanrong dismissed these changes on 2020-11-18

add definition for DynamicQuantizeLSTM

dc9ac369

refactor lstm code for quantization

0161e58d

refine the parallelism logic

87292e59

implement DynamicQuantizeLSTM

4bcbc6fc

add lstm quant in quantization tool

25ca9308

add check

20d18de2

fix quant lstm dimension

882b9097

add a unit test

36ea7f2b

refactor Compute function

1bf2c8fe

remove redudant code

44a76dba

Refactor unit test

4703348e

choose better mean and stddev

8565fe4e

refine scale and zero point

609932cf

fix u8s8 overflow in unit test

b056b0c4

fix build break in Linux

943a096b

merge master to leverage post processor

a9a82569

Add per-column support

cc3b1732

add test for per-channel

be705f28

use batch parallel for quant lstm

0285507b

fix a bug

c0a69634

resolve conflict

bd02a82c

yufenglee dismissed their stale review via bd02a82c 5 years ago

yufenglee force pushed to bd02a82c 5 years ago

zhanghuanrong approved these changes on 2020-11-18

yufenglee merged 6f86c4db into master 5 years ago

yufenglee deleted the roli/rnn_quant branch 5 years ago

Reviewers

zhanghuanrong

tracysh

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

onnxruntime Quantize LSTM #5595 Merged

Quantize LSTM #5595

onnxruntime
Quantize LSTM
#5595

Merged