pytorch
33bb8ae3 - Set shuffle to DataPipes with set_shuffle API (#83741)

Commit
2 years ago
Set shuffle to DataPipes with set_shuffle API (#83741) This PR requires PR is landed: https://github.com/pytorch/pytorch/pull/83202 ## changes - For `apply_shuffle_setting` and `apply_shuffle_seed`, it makes sure it will apply shuffle setting to each of DataPipe that contains a method called `set_shuffle` or `set_seed`. - Change the API from `apply_shuffle_seed` to `apply_random_seed`. - Fix a bug that `apply_shuffle_seed` only accepts DataPipe that is hashable. After the PR, this function uses `id` to prevent seeding the same DataPipe multiple times per epoch. - Fix another bug from `shuffler` that `reset` with `_enable=False` would also reset `_seed`. Pull Request resolved: https://github.com/pytorch/pytorch/pull/83741 Approved by: https://github.com/NivekT
Author
Committer
Parents
Loading