DeepSpeed

f74ee318 - don't gather partitioned activations for mp size 1 (#2454)

don't gather partitioned activations for mp size 1 (#2454) * don't gather partitioned activations for mp size 1 * add inline comment for the change Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>