Resolve #25605 cyclic reference in _LRScheduler (#25776)
Summary:
Cyclic reference was introduced in a previous version due to runtime overwriting of the bound method `optimizer.step`. This is now avoided by keeping a weak reference to the optimizer instance.
Credit: https://stackoverflow.com/questions/26157952/why-set-a-bound-method-to-python-object-create-a-circular-reference
Pull Request resolved: https://github.com/pytorch/pytorch/pull/25776
Differential Revision: D17420770
Pulled By: ezyang
fbshipit-source-id: 546ec94cf725ebfddb310b24e6a2e146ddecd1f6
