Revert "adding a custom caster for c10::SymInt (#82692)"
This reverts commit dee63f4f7bb35559fc34539422229d9ab375dfb9.
Reverted https://github.com/pytorch/pytorch/pull/82692 on behalf of https://github.com/seemethere due to Broke internal builds, see [logs](https://www.internalfb.com/intern/sandcastle/job/4503600373141339/insights)
