Fixes to make trt timing_cache really work (#67524)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/67524
We have some loose ends to tie to make timing cache really work. This diff fixes them.
Reviewed By: wushirong
Differential Revision: D32012021
fbshipit-source-id: 1e93c76d48a3740a02613e1f19222ed92cca9deb
