transformers
c10decf7
- [Bart: example] drop columns that are exclusively pad_token_id… (#3400)
Go
Login via GitHub
Home
Pricing
FAQ
Install
Login
via GitHub
Commit
View On
GitHub
Commit
6 years ago
[Bart: example] drop columns that are exclusively pad_token_id… (#3400) * trim seq_len below 1024 if there are columns full of pad_token_id * Centralize trim_batch so SummarizationDataset can use it too
References
#3400 - [Bart: example] drop columns that are exclusively pad_token_id from input_ids
Author
sshleifer
Parents
63f4d8ca
Loading