DeepSpeed
2ffaecbb
- Fix mis-aligned-grad
Go
Login via GitHub
Home
Pricing
FAQ
Install
Login
via GitHub
Commit
View On
GitHub
Commit
4 years ago
Fix mis-aligned-grad When a parameter is not divisible by world size, the partitioned gradients are mis-aligned due to incorrect padding handling. This PR should fix for that.
References
#851 - Samyamr/inference hook fix
Author
samyam
Parents
2e6692c8
Loading