Revert "Test torch._refs with aten and nvfuser executors (#78926)"
This reverts commit d4eebca7bc14b28688914577f690b68313cb846f.
Reverted https://github.com/pytorch/pytorch/pull/78926 on behalf of https://github.com/malfet due to breaks rocms, see https://hud.pytorch.org/pytorch/pytorch/commit/d4eebca7bc14b28688914577f690b68313cb846f
