Simplify guards using info from previous guards (#121463)
Let me see what CI thinks about this one. Will add tests tomorrow.
Fixes https://github.com/pytorch/pytorch/issues/119917
Pull Request resolved: https://github.com/pytorch/pytorch/pull/121463
Approved by: https://github.com/ezyang
