DeepSpeed

dcb4a7d6 - Add ZeRO 1 support to PP for BF16. (#3399)

Add ZeRO 1 support to PP for BF16. (#3399) * Add ZeRO 1 support to PP for BF16. * Switching enum. --------- Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>