pytorch
a1216061 - [DataPipe] Fix deepcopy filehandle for Mapper and in-place modification for IterableWrapper (#65220)

Commit
3 years ago
[DataPipe] Fix deepcopy filehandle for Mapper and in-place modification for IterableWrapper (#65220) Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/65220 Fixes #65221 - Remove deepcopy from Mapper to support file handles - Convert `IterableWrapper` to deepcopy iterable instance within each iterator to prevent in-place modification (different data per epoch) - Convert `IDP` to `IterableWrapper` in test_datapipe.py - Refine the variable names (prevent using `dp` that is module reference) Test Plan: Imported from OSS Reviewed By: malfet Differential Revision: D31021886 Pulled By: ejguan fbshipit-source-id: 72a9eee66c758e2717d591cd0942892bddedc223
Author
Parents
Loading