DeepSpeed

5fe9d610 - Tensor parallelism for Mixture of Experts (#2074)

Tensor parallelism for Mixture of Experts (#2074) * tensor parallelism for mixture of experts Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com> Co-authored-by: Ammar Ahmad Awan <ammar.awan@microsoft.com>