remove pointless test (#52609)
Summary:
Fixes T81870118
Pull Request resolved: https://github.com/pytorch/pytorch/pull/52609
Reviewed By: mruberry
Differential Revision: D26584288
Pulled By: ngimel
fbshipit-source-id: 7cec37db46cfe5b5b2fd21fe7c3e3fcbb8aba049
