pytorch
3c109876 - don't add extra shuffle in DataLoader2 if one is present

Commit
3 years ago
don't add extra shuffle in DataLoader2 if one is present Without this, `DataLoader2` will just add an `Shuffler` to the end of the datapipe if `shuffle=True`: ```py from torch.utils.data.dataloader_experimental import DataLoader2 from torchdata.datapipes.iter import IterableWrapper, IterDataPipe, Shuffler class Sorter(IterDataPipe): def __init__(self, datapipe): self.datapipe = datapipe def __iter__(self): return iter(sorted(self.datapipe)) data = list(range(1000)) dp = IterableWrapper(data) dp = Shuffler(dp).set_shuffle(False) dp = Sorter(dp) dl2 = DataLoader2(dp, shuffle=True, batch_size=None) assert list(dl2) == data # fails unless you hit a lucky random seed ``` This example is somewhat non-sensical, but demonstrates we cannot simply add a `Shuffler`. Pull Request resolved: https://github.com/pytorch/pytorch/pull/75014 Approved by: https://github.com/ejguan
Author
Committer
Parents
Loading