DeepSpeed
Samyamr/full precision for ZeRO Stage2 and Stage3
#1004

Merged

Commits

Adding tf32 and fp32 support for ZeRO Stage 3

samyam committed 4 years ago
Changing to location of self.dtype assignment

samyam committed 4 years ago
Exhaustive setting of self.dtype

samyam committed 4 years ago
Adding fp32 and tf32 support for ZeRO Stage 2

samyam committed 4 years ago
fix loss scale value for static loss scale

samyam committed 4 years ago
Adding tf32 and fp32 support for ZeRO Stage 3

samyam committed 4 years ago
Changing to location of self.dtype assignment

samyam committed 4 years ago
Exhaustive setting of self.dtype

samyam committed 4 years ago
Adding fp32 and tf32 support for ZeRO Stage 2

samyam committed 4 years ago
fix loss scale value for static loss scale

samyam committed 4 years ago
adding documentation for dtype in zero init

samyam committed 4 years ago
fixing conflict

samyam committed 4 years ago
Merge branch 'master' into samyamr/full-precision-for-stage3

jeffra committed 4 years ago
Merge branch 'master' into samyamr/full-precision-for-stage3

tjruwase committed 4 years ago
Disable cpu-adam update_copy api for fp32

tjruwase committed 4 years ago
Disable gradient clipping in engine for ZeRO

tjruwase committed 4 years ago
fp16 mode init required for ZeRO-3

tjruwase committed 4 years ago
Update engine.py

samyam committed 4 years ago
Formatting fix

tjruwase committed 4 years ago
bump DSE

jeffra committed 4 years ago
Merge branch 'master' into samyamr/full-precision-for-stage3

jeffra committed 4 years ago