Skips spectral tests to prevent ROCm build from timing out (#42667)
Summary:
Per title.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/42667
Reviewed By: ailzhang
Differential Revision: D22978531
Pulled By: mruberry
fbshipit-source-id: 0c3ba116836ed6c433e2c6a0e1a0f2e3c94c7803
