[PyTorch] Don't force refcount bump when accessing DictEntryRef key/value (#68158)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/68158
to() sometimes returns a reference; let's forward that through.
ghstack-source-id: 143424916
Test Plan: Combined with following diff, seeing a huge drop in dict_unpack self time in ctr_mobile_feed local_ro net. Following diff by itself didn't work.
Reviewed By: suo
Differential Revision: D32347391
fbshipit-source-id: da96295bf83ea30867a2e3fceedc9b4e0a33ffa3
