Don't switch streams when running Caffe2 ops from c10. (#55121)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/55121
This is done by allow -1 as a stream ID, meaning "don't change
the stream", in SwitchToDevice
Fixes #54830
Signed-off-by: Edward Z. Yang <ezyang@fb.com>
Test Plan: Imported from OSS
Reviewed By: malfet
Differential Revision: D27527544
Pulled By: ezyang
fbshipit-source-id: c54983d6fc79a8fa1c65a71559a57425e40ba717
