pytorch
be80b72a - [FSDP] Remove unneeded stream sync from `clip_grad_norm_()` (#89308)

Commit
2 years ago
[FSDP] Remove unneeded stream sync from `clip_grad_norm_()` (#89308) We do not need to have the pre-unshard and unshard streams wait for the computation stream because we are not using the pre-unshard or unshard streams in `clip_grad_norm_()`. The other change is simply avoiding a loop to get `grads`. Pull Request resolved: https://github.com/pytorch/pytorch/pull/89308 Approved by: https://github.com/mrshenli
Author
Committer
Parents
Loading