pytorch
a9970460 - [DataPipe] Update `mux` data pipe (#76384)

Commit
2 years ago
[DataPipe] Update `mux` data pipe (#76384) Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/76384 OSS issue discussion: https://github.com/pytorch/data/issues/346 This diff updates `mux` and `mux_longest` data pipe. `mux`: Yields one element at a time from each of the input Iterable DataPipes (functional name: ``mux``). As in, one element from the 1st input DataPipe, then one element from the 2nd DataPipe in the next iteration, and so on. It ends when the shortest input DataPipe is exhausted. `mux` example: ``` >>> from torchdata.datapipes.iter import IterableWrapper >>> dp1, dp2, dp3 = IterableWrapper(range(3)), IterableWrapper(range(10, 15)), IterableWrapper(range(20, 25)) >>> list(dp1.mux(dp2, dp3)) [0, 10, 20, 1, 11, 21, 2, 12, 22] ``` Test Plan: buck test mode/dev //pytorch/data/test:tests -- --exact 'pytorch/data/test:tests - test_mux_longest_iterdatapipe (test_datapipe.TestDataPipe)' https://www.internalfb.com/intern/testinfra/testrun/3096224791148107 Reviewed By: ejguan Differential Revision: D35799965 fbshipit-source-id: 320e71a342ec27e6e9200624aad42f4b99f97c3a (cherry picked from commit 741ed595275df6c05026ed6f0e78d7052328fb7d)
Committer
Parents
Loading