Turn off non-default stream testing. (#21793)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/21793
ghimport-source-id: 5264fa90ca77fbc79898cfa2f0ee02f47dec27d4
Test Plan: Imported from OSS
Differential Revision: D15874814
Pulled By: ezyang
fbshipit-source-id: 5c51ab9ae431faf2db549b88b07ba00783acab25
