DeepSpeed

8d87c89e - BF16 optimizer for BF16+ZeRO Stage 1 (#2706)

BF16 optimizer for BF16+ZeRO Stage 1 (#2706) * BF16 optimizer only with ZeRO stage 1. * Updating to grad accum of fp32 for BF16 ZeRO1 case. Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>