Minor gradcheck update to reduce computations (#45757)
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/45757
Test Plan: Imported from OSS
Reviewed By: glaringlee
Differential Revision: D24137143
Pulled By: anjali411
fbshipit-source-id: e0174ec03d93b1fedf27baa72c3542dac0b70058
