Revert D24452660: [pytorch][PR] Add CUDA 11.1 CI
Test Plan: revert-hammer
Differential Revision:
D24452660 (https://github.com/pytorch/pytorch/commit/1479ed91be56a10e0014c8b1646fbcbf8d741dd9)
Original commit changeset: 3480a2533214
fbshipit-source-id: 1e720c5d6fe1a377f6decd3ecc4f412c53fb293c
