[PyTorch] Store Tensor explicitly in IValue (#48824)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/48824
Enables following diff, which will make toTensor() return
`const Tensor&` and allow callers to avoid refcounting overhead.
ghstack-source-id: 119327370
Test Plan:
ivalue_test
Internal benchmark to ensure perf parity. Some interesting steps
during the debugging process:
- First version was about a 5% regression
- Directly implementing move construction instead of using swap
lowered the regression to 2-3%
- Directly implementing move assign was maybe an 0.5% improvement
- Adding C10_ALWAYS_INLINE on move assign got our regression to
negligible
- Fixing toTensor() to actually be correct regressed us again, but
omitting the explicit dtor call as exhaustively spelled out in a
comment fixed it.
Reviewed By: bwasti
Differential Revision: D25324617
fbshipit-source-id: 7518c1c67f6f2661f151b43310aaddf4fb6e511a
