Megatron-DeepSpeed
Add step to download dataset before running the preprocess_data_dist test
#81

Merged

Add step to download dataset before running the preprocess_data_dist test #81

stas00 merged 3 commits into bigscience-workshop:main from adammoody:disttest

tests: add step to predownload dataset for preprocess_data_dist

12758e2b

load dataset directly in pytest

cf73cffd

adammoody force pushed from 24d6cee2 to cf73cffd 4 years ago

stas00 requested changes on 2021-08-26

cache load_dataset result

4c7af933

stas00 merged fa9f54c9 into main 4 years ago

adammoody deleted the disttest branch 4 years ago

Reviewers

stas00

Assignees

No one assigned

Labels

None yet

Milestone

No milestone