onnxruntime
Make FusedAdam mathematically equivalent to Transformers AdamW
#9343

Merged

Make FusedAdam mathematically equivalent to Transformers AdamW #9343

baijumeswani merged 3 commits into master from bmeswani/fused_adam_transformers_adamw

Make FusedAdam mathematically equivalent to Transformers AdamW

ac3097d7

baijumeswani added training

baijumeswani requested a review from

BowenBao 4 years ago

baijumeswani requested a review from

liqunfu 4 years ago

baijumeswani requested a review from

SherlockNoMad 4 years ago

baijumeswani requested a review from

thiagocrepaldi 4 years ago

baijumeswani requested a review from

tlh20 4 years ago

SherlockNoMad commented on 2021-10-13

Add comment about implementation being equivalent to transformers AdamW

0f8c17c6

SherlockNoMad commented on 2021-10-13

Update unit test as per code review comment

bf550b95

SherlockNoMad approved these changes on 2021-10-18

baijumeswani merged 5da4e07d into master 4 years ago

baijumeswani deleted the bmeswani/fused_adam_transformers_adamw branch 4 years ago

Reviewers

SherlockNoMad

BowenBao

liqunfu

thiagocrepaldi

tlh20

Assignees

No one assigned

Labels

training

Milestone

No milestone