use make_shared and make_unique and clean unneeded code (#51829)
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/51829
Reviewed By: izdeby
Differential Revision: D26306098
Pulled By: smessmer
fbshipit-source-id: 4f6c0469c68f044c0bfe0925fcf7b030a25d15e2
