PR #6289 Add max_norm for gradient clipping.

Add max_norm for gradient clipping. #6289

pengwa merged 19 commits into master from pengwa/clip_norm

add max_norm as user option for gradient clipping

961a59a3

formatting

7bf78359

pengwa requested a review from

SherlockNoMad 5 years ago

pengwa requested a review from

BowenBao 5 years ago

pengwa requested a review from

liqunfu 5 years ago

pengwa requested a review from

spandantiwari 5 years ago

pengwa requested a review from

thiagocrepaldi 5 years ago

pengwa requested a review 5 years ago

pengwa requested a review from

wschin 5 years ago

pengwa commented on 2021-01-08

fix amd gpu build

830a05b1

pengwa requested a review from

nbcsm 5 years ago

pengwa added training

pengwa added component:training-frontend

SherlockNoMad commented on 2021-01-09

SherlockNoMad closed this 5 years ago

SherlockNoMad reopened this 5 years ago

wschin dismissed these changes on 2021-01-11

Merge branch 'master' of https://github.com/microsoft/onnxruntime int…

923fb681

Refine the code a bit, fix build

5c382a07

pengwa dismissed their stale review via 5c382a07 5 years ago

fix tests. add 1e-6 aligned with torch clip_grad

db4e6656

fix tests

32dd6a52

fix ut

1ca83046

fix lamb test due to wrong order of parameter

f22ec656

add clip norm test for adam

faeccf0a

ignore clip norm cases for amd gpu following exiting test cases

9e80404d

fix windows build

b4a6dda3

wschin commented on 2021-01-15

Add more lamb test cases for clip norm

0588a1aa

Merge branch 'master' of https://github.com/microsoft/onnxruntime int…

fd16e27d

refine the gradientL2Norm calculation

69e9b9b2

fix cpu build

4e902228

fix amd traing ut

fd1d9418

wschin dismissed these changes on 2021-01-18

SherlockNoMad dismissed these changes on 2021-01-19

liqunfu dismissed these changes on 2021-01-19

thiagocrepaldi commented on 2021-01-19

add frontend tests

4180f21b

Merge branch 'master' of https://github.com/microsoft/onnxruntime int…

cb2942ab

pengwa dismissed their stale review via cb2942ab 5 years ago

thiagocrepaldi approved these changes on 2021-01-20

pengwa merged 453431f7 into master 5 years ago

pengwa deleted the pengwa/clip_norm branch 5 years ago

Reviewers

thiagocrepaldi

liqunfu

SherlockNoMad

wschin

BowenBao

spandantiwari

nbcsm

Assignees

No one assigned

Labels

training

Milestone

No milestone

onnxruntime Add max_norm for gradient clipping. #6289 Merged

Add max_norm for gradient clipping. #6289

onnxruntime
Add max_norm for gradient clipping.
#6289

Merged