Save unintended memory usage
Summary: We have places in torchbench that doesn't release the memory properly and therefore w/ dynamo, the memory reporting will be much higher but they're really false positive.
Reviewed By: bertmaher, jiawenliu64
Differential Revision: D44567443
fbshipit-source-id: a610b2ee5c7c473830349382c9b127ce26597dd9
