Revert "Transmute refined SymInt into int (#104828)"
This reverts commit 0f322a300efe588a4f7ae61dabcfd4fe0aa9e225.
Reverted https://github.com/pytorch/pytorch/pull/104828 on behalf of https://github.com/ezyang due to executorch failure ([comment](https://github.com/pytorch/pytorch/pull/104828#issuecomment-1635997559))
