onnxruntime
Compute global gradient norm according to 'enable_grad_norm_clip'
#5728
Merged
Merged
Compute global gradient norm according to 'enable_grad_norm_clip' #5728
SherlockNoMad
merged
9 commits into
master
from
bahuang/grad_norm_fix
Introduce PassThrough op to wait for all gradient ready before weigh…
4393fc10
Compute gradient norm for fp32 runs
fbdda5a1
Skip AddGradientNorm for BF16 run for now
32f2e85c
SherlockNoMad
changed the title 
wschin
commented
on 2020-11-06
wschin
commented
on 2020-11-06
wschin
dismissed these changes
on 2020-11-06
Address comment
8d401fa7
Lafi7e
commented
on 2020-11-07
Fix UT
70cf08fa
Update FE UT expected value
8fbb74cf
Merge branch 'bahuang/gradient_barrier' into bahuang/grad_norm_fix
45db780c
Respect enable_grad_norm_clip
7fc44699
SherlockNoMad
changed the title
wschin
commented
on 2020-11-10
wschin
commented
on 2020-11-10
wschin
commented
on 2020-11-10
wschin
commented
on 2020-11-10
wschin
commented
on 2020-11-10
address comments
bfd22b5a
wschin
approved these changes
on 2020-11-11
liqunfu
approved these changes
on 2020-11-11
Reviewers
Assignees
No one assigned
Labels
training
Milestone
No milestone
Login to write a write a comment.
Login via GitHub