DeepSpeed
b288cf1b
- Enable contiguous gradients with Z1+MoE (#2250)
Go
Login via GitHub
Home
Pricing
FAQ
Install
Login
via GitHub
Commit
View On
GitHub
Hide Minimap (CTRL+M)
Commit
2 years ago
Enable contiguous gradients with Z1+MoE (#2250) MoE training with zero stage 1 only works with `contiguous gradients=True`.
References
#2250 - Enable contiguous gradients with Z1+MoE
Author
siddharth9820
Parents
ebed51df
Files
1
deepspeed/runtime
engine.py
Loading