PR #286 Support amp deepspeed backend

Support amp deepspeed backend #286

jeffra merged 5 commits into master from jeffra/enable-amp-backend

add amp support for deepspeed (non-ZeRO)

dcb53bfb

tests for amp mode

efab92b6

jeffra requested a review from

ShadenSmith 5 years ago

jeffra requested a review from

samyam 5 years ago

jeffra requested a review from

tjruwase 5 years ago

update docs around amp constants

b280ee2f

add comment about zero and legacy fp16 mode being similar to O2

c76f0564

tjruwase approved these changes on 2020-07-10

samyam approved these changes on 2020-07-11

fix error in mpu codepath

2bbcab58

jeffra merged f5453124 into master 5 years ago

jeffra deleted the jeffra/enable-amp-backend branch 5 years ago

Reviewers

tjruwase

samyam

ShadenSmith

Assignees

No one assigned

Labels

None yet

Milestone

No milestone