Fix backwards compat test by building twice (#80969)
fixes #80361
Test plan:
https://github.com/pytorch/pytorch/runs/7308328975?check_suite_focus=true -- this is from [5eb2f4b](https://github.com/pytorch/pytorch/pull/80969/commits/5eb2f4bcf1cc31a2d8e43124ef67f2a5c8e54e2f), the only difference is I added GHA grouping to hide the build in the logs + cleaned up chill code
Pull Request resolved: https://github.com/pytorch/pytorch/pull/80969
Approved by: https://github.com/malfet
