Ensure initializedContextIds_ map is cleaned up appropriately in DistEngine. (#29787)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/29787
The initializedContextIds_ map was never cleaned up in DistEngine and
kept on growing as we continue to run backward passes. To fix this, in this PR
we ensure that the context id is cleaned up from this map once we are done with
the backward pass.
Closes #29083
ghstack-source-id: 94161770
Test Plan: waitforbuildbot
Differential Revision: D18498937
fbshipit-source-id: 8d31fc066f6994627766f2b6ca36efa1bef89840
