PR #4156 Optimize LayernormGrad

Optimize LayernormGrad #4156

weixingzhang merged 6 commits into master from layernormgrad_opt

weixingzhang added training

weixingzhang requested a review from

SherlockNoMad 6 years ago

weixingzhang requested a review 6 years ago

yuslepukhin commented on 2020-06-08

weixingzhang force pushed from 832fca23 to b9d57342 6 years ago

SherlockNoMad requested a review from

aishwarya56 6 years ago

SherlockNoMad requested a review from

aishwaryabh 6 years ago

SherlockNoMad removed review request from

aishwarya56 6 years ago

ashbhandare removed review request from

aishwaryabh 6 years ago

ashbhandare requested a review from

ashbhandare 6 years ago

ashbhandare commented on 2020-06-24

weixingzhang force pushed from b9d57342 6 years ago

Draft for LayerNorm Optimization

e2ce3070

Modify LayernormGrad kernel based on new backward graph.

17186318

keep two LayernormGrad implementations.

8b723b6b

expose use_invertible_layernorm_grad to frontend.

b6928a67

fix frontend.

99ca4c31

weixingzhang force pushed to 99ca4c31 6 years ago

add fp16 tests.

3c04d986

ashbhandare approved these changes on 2020-07-02

weixingzhang merged bd11ab68 into master 6 years ago

weixingzhang deleted the layernormgrad_opt branch 6 years ago

Reviewers

ashbhandare

tianleiwu

yuslepukhin

SherlockNoMad

Assignees

No one assigned

Labels

training

Milestone

No milestone

onnxruntime Optimize LayernormGrad #4156 Merged

Optimize LayernormGrad #4156

onnxruntime
Optimize LayernormGrad
#4156

Merged