pytorch

07de11c2 - [torch.Package/TorchScript] TS serialization importer to handle unified format (#54891)

[torch.Package/TorchScript] TS serialization importer to handle unified format (#54891) Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/54891 Changed TorchScript's jit/serialization importer logic to handle both original TS serialization format and new unified TS format Original TS file format: ``` resnet.pt ├── data # tensor data │ ├── 94286146172688 │ ├── 94286146172784 │ └── ... ├── code/ # TorchScript code │ ├── __torch__ │ │ ├── torch │ │ │ └── nn ... │ │ └── torchvision ... │ ├── __torch__.py │ └── __torch__.py.debug_pkl ├── data.pkl # the ScriptModule object, pickled by the TS pickler ├── version # version metadata ├── constants.pkl # any tensor constants present in the TS code └── extra ├── name_of_file └── foo ``` Unified file format: ``` ─── package_name.pt ├── .data │ ├── ts_code # code shared between models │ │ ├── 0 │ │ │ ├── constants.pkl │ │ │ └── data.pkl │ │ ├── 1 │ │ │ ├── constants.pkl │ │ │ └── data.pkl │ │ └── code │ │ ├── __torch__ │ │ │ ├── torch │ │ │ │ └── nn ... │ │ │ └── torchvision ... │ │ ├── __torch__.py │ │ └── __torch__.py.debug_pkl │ ├── 0.storage │ ├── 1.storage │ ├── <many more storages> │ ├── 201.storage │ ├── extern_modules │ └── version └── res ├── mod.pkl # maps to ts_id 0 and .data/ts_code/0 └── mod2.pkl # maps to ts_id 1 and .data/ts_code/1 ``` Test Plan: Imported from OSS Reviewed By: suo Differential Revision: D27832548 Pulled By: Lilyjjo fbshipit-source-id: 4a6e84c3a9bac8eed6a4e4afc2ac76dd691858b0