PR #264 Preprocessing from arrow file to load an HF dataset

Preprocessing from arrow file to load an HF dataset #264

TevenLeScao wants to merge 6 commits into main from preprocess_from_HF_dataset

preprocessing from arrow file to load an HF dataset

20f582fe

TevenLeScao requested a review from

thomasw21 3 years ago

TevenLeScao requested a review from

stas00 3 years ago

making sure iterator actually advances

f30f97cb

fixed confusion between row format and column format

c868bc6c

Adding faster hack version

505e8cac

Adding faster hack version

ea9051e0

added dirty hack to have a maximum size

cb7648b0

Reviewers

thomasw21

stas00

Assignees

No one assigned

Labels

None yet

Milestone

No milestone