[PyTorch] Fix extra refcount bumps in matchTypeVariables (#66719)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/66719
Some cast that could be castRaw. Parameters did not need to force a refcount bump.
ghstack-source-id: 140756356
Test Plan: CI
Reviewed By: suo
Differential Revision: D31697455
fbshipit-source-id: 87a8cba221a7ae53f2a485acafd31622e9328ff0
