Avoid computing AutogradKey if not needed. (#46252)
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/46252
Test Plan: CI
Reviewed By: ngimel
Differential Revision: D24272744
fbshipit-source-id: 6cb66d13e6c910df1ad1a8badd43f990e7b55368
