[JIT] Specialize autograd zero: actually remove the original graph after we created its versioned copy. (#43900)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/43900
The original code assumed that the versioning if was inserted in the
beginning of the graph while in fact it was inserted in the end. We're
now also not removing `profile_optional` nodes and rely on DCE to clean
it up later (the reason we're not doing it is that deletion could
invalidate the insertion point being used).
Test Plan: Imported from OSS
Reviewed By: eellison
Differential Revision: D23432175
Pulled By: ZolotukhinM
fbshipit-source-id: 1bf55affaa3f17af1bf71bad3ef64edf71a3e3fb
Author
Mikhail Zolotukhin
