pytorch
5f623f5c - Correctly handle duplicate arguments to AOTAutograd (#85301)

Commit
2 years ago
Correctly handle duplicate arguments to AOTAutograd (#85301) If we do not deduplicate them, our custom autograd function will double-count the gradient computed for the variable (since the same x.grad field will be embedded into the graph twice.) The alternative is to destroy aliasing relationships the inputs and trace accumulating individual gradients for each of the inputs into separate grad fields, but this prevents resizing of inputs inside the traced graph from working correctly. In principle, you could detach the inputs, allow metadata changes on them, and then reflect metadata changes to the originals as necessary (in fact, we should do this for other reasons), but AOTAutograd doesn't do this yet. Another alternative is to have dynamo guarantee not to give duplicate tensor inputs, but because AOTAutograd is public API, we are obligated to still handle it correctly here in case a direct user passes duplicate inputs. Signed-off-by: Edward Z. Yang <ezyang@fb.com> Differential Revision: [D39662821](https://our.internmc.facebook.com/intern/diff/D39662821) Pull Request resolved: https://github.com/pytorch/pytorch/pull/85301 Approved by: https://github.com/Chillee, https://github.com/albanD
Author
Committer
Parents
Loading