Megatron-DeepSpeed
d576775c
- Sync torch_rng_state (#277)
Go
Login via GitHub
Home
Pricing
FAQ
Install
Login
via GitHub
Commit
View On
GitHub
Commit
3 years ago
Sync torch_rng_state (#277)
References
#274 - Sync 4 layer norms - bf16, fp32, optimizer states on restart
#277 - Thomas/olruwase/sync layer norms
Author
thomasw21
Parents
2ac141b1
Loading