transformers
In group_texts function, drop last block if smaller than block_size
#17908

Merged

In group_texts function, drop last block if smaller than block_size #17908

sgugger merged 1 commit into huggingface:main from billray0259:bug_fix_language_modeling_task

In `group_texts` function, drop last block if smaller than `block_size`

b6beb267

SaulLu requested a review from

stevhliu 3 years ago

SaulLu requested a review from

sgugger 3 years ago

SaulLu approved these changes on 2022-06-28

sgugger approved these changes on 2022-06-28

sgugger merged bfcd5743 into main 3 years ago

billray0259 deleted the bug_fix_language_modeling_task branch 3 years ago

Reviewers

sgugger

SaulLu

stevhliu

Assignees

No one assigned

Labels

None yet

Milestone

No milestone