PR #298 MTF optimize dataloading

MTF optimize dataloading #298

thomasw21 merged 7 commits into thomas/mtf_train_script from thomas/mtf_optimize_dataloading

- Remove unecessary code from MTFDataset

cd1c7293

thomasw21 changed the base branch from main to thomas/mtf_train_script 3 years ago

Remove unecessary test

4e7a2996

Woops

f899927b

Let's go even faster

81c98d62

Woops

a0654312

Commit new index dataset

8afa8a7a

thomasw21 marked this pull request as ready for review 3 years ago

thomasw21 requested a review from

Muennighoff 3 years ago

Hopefully len isn't that slow

b8b7ff1b

Muennighoff approved these changes on 2022-07-04

thomasw21 merged f2df7715 into thomas/mtf_train_script 3 years ago

thomasw21 deleted the thomas/mtf_optimize_dataloading branch 3 years ago

Reviewers

Muennighoff

Assignees

No one assigned

Labels

None yet

Milestone

No milestone