in caching allocator, ignore and clear the error if not ready
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/18584
Differential Revision: D14675041
Pulled By: bddppq
fbshipit-source-id: c1fab797e0d224e0a481a0395a3f9975c4265ff6
