Fix unstable CI related to dynamo tests (#105797)
this PR fix the current unstable CI. The test failure comes from a bad
revert in https://github.com/pytorch/pytorch/pull/105581 where it does
not revert the intended PR correctly (there were some merge conflicts
and some logic got deleted during this revert)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/105797
Approved by: https://github.com/ezyang
