DeepSpeed
21c28029 - Adding Gradient Accumulation Data Type Config (#2512)

Commit
3 years ago
Adding Gradient Accumulation Data Type Config (#2512) * Adding gradient accumulation dtype config. * Switching to new DtypeEnum * Adding standalone check function, and unit tests * Variable disambiguation * Adding checks for unsupported states. * Updating for PR comments. * Reorganizing unit test. Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>
Author
Parents
Loading