transformers
[trainer] add `--optim adamw_torch_fused` for pt-2.0+
#22144

Merged

[trainer] add `--optim adamw_torch_fused` for pt-2.0+ #22144

stas00 merged 6 commits into main from adamw_fused_torch

[trainer] add --optim adamw_torch_fused

048fa2f0

stas00 changed the title ~~[trainer] add --optim adamw_torch_fused~~ [trainer] add `--optim adamw_torch_fused` for pt-2.0+ 2 years ago

change optim default

37c10f9c

deal with non-torch

83dc1cf8

revert default change; prep; add fp16/amp assert

c3a353c3

stas00 requested a review from

sgugger 2 years ago

stas00 marked this pull request as ready for review 2 years ago

typo

1ff82b15

typo

9f30c252

sgugger approved these changes on 2023-03-14

stas00 merged 085bf5c1 into main 2 years ago

stas00 deleted the adamw_fused_torch branch 2 years ago

Reviewers

sgugger

Assignees

No one assigned

Labels

None yet

Milestone

No milestone