DeepSpeed
0bb0cc80 - Use zero-tensors for missing gradients to avoid size mismatch

Commit

5 years ago

Use zero-tensors for missing gradients to avoid size mismatch

References

#545 - Fix unbalanced gradients bug in ZeRO-2 gradient accumulation

Author

tjruwase

tjruwase

Parents

Loading