make test context managers exception safe
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/22502
Test Plan: Imported from OSS
Differential Revision: D16121783
Pulled By: suo
fbshipit-source-id: e5f991b189261f596355541cc331ef92667bd1a5
