Removes flaky check (#33146)
Summary:
Addresses https://github.com/pytorch/pytorch/issues/32949.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/33146
Differential Revision: D19836001
Pulled By: mruberry
fbshipit-source-id: 773069ae0c181e1a050b65b888c87590c1dddb32
