Don't guard if there are unbacked SymInts (#119312)
Fixes https://github.com/pytorch/pytorch/issues/119309
Not sure how to write the test.
Signed-off-by: Edward Z. Yang <ezyang@meta.com>
Pull Request resolved: https://github.com/pytorch/pytorch/pull/119312
Approved by: https://github.com/lezcano
