DeepSpeed
91def84e - (1) Size temp grad buffer to largest param, not largest partition.

Commit

4 years ago

(1) Size temp grad buffer to largest param, not largest partition. (2) Handle all args in activation checkpointing.

Author

tjruwase

Parents