transformers
Add magic method to our TF models to convert datasets with column inference
#17160
Merged
Overview
Commits
35
Changes
View On GitHub

Add magic method to our TF models to convert datasets with column inference #17160

Rocketknight1 merged 35 commits into main from magic_tf_dataset_method
Rocketknight1
HuggingFaceDocBuilderDev
Rocketknight1 Rocketknight1 force pushed from f76056b8 to 037e8481 4 years ago
Rocketknight1 Rocketknight1 marked this pull request as ready for review 4 years ago
Rocketknight1 Rocketknight1 requested a review from sgugger sgugger 4 years ago
Rocketknight1 Rocketknight1 requested a review from gante gante 4 years ago
Rocketknight1 Rocketknight1 requested a review from LysandreJik LysandreJik 4 years ago
Rocketknight1
gante
gante commented on 2022-05-19
gante
sgugger
sgugger commented on 2022-05-19
Rocketknight1 Add method to call to_tf_dataset() with column inference
1ccd7260
Rocketknight1 Add test for dataset creation
2c71f846
Rocketknight1 Add a default arg for data collator
919cf82d
Rocketknight1 Fix test
0066598e
Rocketknight1 Fix call with non-dev version of datasets
b40fa6eb
Rocketknight1 Test correct column removal too
f5f667d0
Rocketknight1 make fixup
258392be
Rocketknight1 More tests to make sure we remove unwanted columns
ae4be4af
Rocketknight1 Fix test to avoid predicting on unbuilt models
673e23d6
Rocketknight1 Fix test to avoid predicting on unbuilt models
0ee6e1da
Rocketknight1 Fix test to remove unwanted head mask columns from inputs
2313b3ad
Rocketknight1 Stop pushing your debug breakpoints to the main repo of the $2bn comp…
221ae783
Rocketknight1 Skip the test in convnext because no grouped conv support
15061825
Rocketknight1 Drop bools from the dataset dict
a9010b10
Rocketknight1 Make style
0a29747e
Rocketknight1 Skip the training test for models whose input dicts don't give us labels
5b35ff4a
Rocketknight1 Skip transformerXL in the test because it doesn't return a simple loss
a1b6e924
Rocketknight1 Skip TFTapas because of some odd NaN losses
33812eaa
Rocketknight1 make style
fb19ea9f
Rocketknight1 make fixup
a7f6a859
Rocketknight1 Add docstring
0787a458
Rocketknight1 fixup
24c0a66c
Rocketknight1 Update src/transformers/modeling_tf_utils.py
fc89e11d
Rocketknight1 Update src/transformers/modeling_tf_utils.py
0d4553dd
Rocketknight1 Update src/transformers/modeling_tf_utils.py
03111729
Rocketknight1 Update src/transformers/modeling_tf_utils.py
a7b1f601
Rocketknight1 Update src/transformers/modeling_tf_utils.py
f01f8c21
Rocketknight1 Remove breakpoint from tests
24cc6a6c
Rocketknight1 Fix assert, add requires_backends
e381daff
Rocketknight1 Rocketknight1 force pushed to e381daff 4 years ago
Rocketknight1 Protect tokenizer import with if TYPE_CHECKING
0f473db8
Rocketknight1 make fixup
753c0c50
Rocketknight1 Add noqa, more fixup
b23efa1e
Rocketknight1 More rearranging for ~* aesthetics *~
163b4f72
Rocketknight1 Adding defaults for shuffle and batch_size to match to_tf_dataset()
e29f98fe
sgugger
sgugger commented on 2022-05-19
Rocketknight1 Update src/transformers/modeling_tf_utils.py
726cb39b
gante
gante approved these changes on 2022-05-19
sgugger
sgugger approved these changes on 2022-05-19
LysandreJik
LysandreJik approved these changes on 2022-05-19
Rocketknight1
Rocketknight1
Rocketknight1 Rocketknight1 merged 19a8a303 into main 4 years ago
Rocketknight1 Rocketknight1 deleted the magic_tf_dataset_method branch 4 years ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum