Megatron-DeepSpeed

391ed488 - Force synchronize the layer norms parameters across all TP