pytorch
b2eaba6b - [spmd] by default average gradients for nccl backend (#99964)

Commit
1 year ago
[spmd] by default average gradients for nccl backend (#99964) This PR by default average gradient for NCCL backend, this allows SPMD's data parallel match with DDP/FSDP results. Pull Request resolved: https://github.com/pytorch/pytorch/pull/99964 Approved by: https://github.com/mrshenli
Author
Committer
Parents
Loading