Don't use Proxy torch function in the sym size calls (#121981)
Fixes #ISSUE_NUMBER
Changes from https://github.com/pytorch/pytorch/pull/121938 + adds test
@bypass-github-pytorch-ci-checks
Pull Request resolved: https://github.com/pytorch/pytorch/pull/121981
Approved by: https://github.com/davidberard98
