Fix CI build (#44068)
Summary:
Some of our machines have only 1 device.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/44068
Reviewed By: wanchaol
Differential Revision: D23485730
Pulled By: izdeby
fbshipit-source-id: df6bc0aba18feefc50c56a8f376103352fa2a2ea
