Move slow-grad checks to CUDA-11.6 (#84313)
Mitigates #84192 by skipping two tests
Please note: We tried to increase the tolerance for test_fn_gradgrad_linalg_det_singular_cuda_float64 but this did not help.
Ref:
Increase `test_fn_gradgrad_linalg_det_singular_cuda_float64` error tolerance to 1e-4 as suggested in https://github.com/pytorch/pytorch/issues/84192#issuecomment-1230644574
Pull Request resolved: https://github.com/pytorch/pytorch/pull/84313
Approved by: https://github.com/malfet, https://github.com/huydhn, https://github.com/Lezcano
