Revert "Merge torchdim into functorch build (#82454)"
This reverts commit 23b90044dac04d23f43c1a0f518bdbf95efd3b47.
Reverted https://github.com/pytorch/pytorch/pull/82454 on behalf of https://github.com/zengk95 due to this is breaking mac jobs on trunk https://hud.pytorch.org/pytorch/pytorch/commit/23b90044dac04d23f43c1a0f518bdbf95efd3b47
