Use TP Tensor.metadata to carry device info (#44396)
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/44396
Test Plan: Imported from OSS
Reviewed By: lw
Differential Revision: D23602576
Pulled By: mrshenli
fbshipit-source-id: c639789979b2b71fc165efbcf70f37b4c39469df
