Do not install nvidia docker for non-NVIDIA configs (#41144)
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/41144
Differential Revision: D22457124
Pulled By: malfet
fbshipit-source-id: e615199cb78b315aa700efcc7332ebf4299212bf
