Revert "[pt2] add `SymInt` support for `linalg.pinv` (#102367)"
This reverts commit 0d5b74da0cab798fbfdb9caa53fad816999c8386.
Reverted https://github.com/pytorch/pytorch/pull/102367 on behalf of https://github.com/kit1980 due to Broke slow tests https://github.com/pytorch/pytorch/actions/runs/5095190248/jobs/9160028124 ([comment](https://github.com/pytorch/pytorch/pull/102367#issuecomment-1565104562))
