pytorch
78e2cc6d - Fix wrong handling of `grad_scale` & `found_inf` in fused optimizers (#95847) (#97885)

Commit
2 years ago
Fix wrong handling of `grad_scale` & `found_inf` in fused optimizers (#95847) (#97885) Fixes #95781. The cause seems to be that the current implementation doesn't correctly pass `found_inf` when `grad_scale` is `None`. Therefore parameters can get mistakenly updated by gradients whose some elements are invalid, i.e. nan or inf. Related #94060 I forgot about this wrong handling after #94344 Pull Request resolved: https://github.com/pytorch/pytorch/pull/95847 Approved by: https://github.com/janeyx99
Author
Parents
Loading