TF/Numpy variants for all DataCollator classes #13105
Rocketknight1
merged
36 commits into
master
from
tf_data_collators
sgugger
approved these changes
on 2021-08-12
sgugger
approved these changes
on 2021-08-12
Rocketknight1
changed the title
sgugger
commented
on 2021-08-30
Adding a TF variant of the DataCollatorForTokenClassification to get …
6775d6d3
Added a Numpy variant and a post_init check to fail early if a missin…
0ae2587b
Fixed call to Numpy variant
d577ac9f
Added a couple more of the collators
346a6461
Update src/transformers/data/data_collator.py
33dbb29d
Fixes, style pass, finished DataCollatorForSeqToSeq
70168b57
Added all the LanguageModeling DataCollators, except SOP and Permutat…
050208b1
Adding DataCollatorForPermutationLanguageModeling
ce8dcc32
Style pass
dda89060
Add missing `__call__` for PLM
f81068b0
Remove `post_init` checks for frameworks because the imports inside t…
abf83257
Remove unused imports
52cf8286
First attempt at some TF tests
4b823f92
A second attempt to make any of those tests actually work
6cf757a7
TF tests, round three
6db05c6f
TF tests, round four
212b6e0f
TF tests, round five
dbb38488
TF tests, all enabled!
2d59e3dc
Style pass
eed3c889
Merging tests into `test_data_collator.py`
3616216f
Merging tests into `test_data_collator.py`
f4137170
Fixing up test imports
71cb01c9
Fixing up test imports
984496d4
Trying shuffling the conditionals around
d9b16f7d
Commenting out non-functional old tests
e1975bd2
Completed all tests for all three frameworks
1616c6f4
Style pass
e57a75b5
Fixed test typo
9b654d9d
Style pass
95b0ddc0
Move standard `__call__` method to mixin
472f74ba
Rearranged imports for `test_data_collator`
57910285
Fix data collator typo "torch" -> "pt"
582a95cb
Fixed the most embarrassingly obvious bug
4b9cfb52
Update src/transformers/data/data_collator.py
dc58c697
Renaming mixin
375c3162
Updating docs
28606ece
LysandreJik
approved these changes
on 2021-08-31
Login to write a write a comment.
Login via GitHub