DeepSpeed

59d62d61 - Move tensors to device if mp is not enabled (#4461)

Move tensors to device if mp is not enabled (#4461) Tensors are not moved to device in gather_partitioned_activations in case model parallel is not enabled. Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>