transformers
`tokenizer` train from iterator without pre_tokenizers
#35396
Merged
Overview
Commits
6
Changes
View On GitHub

`tokenizer` train from iterator without pre_tokenizers #35396

ArthurZucker merged 6 commits into main from fix-tokenizer
ArthurZucker
ArthurZucker fix if else issues
a1352bdf
ArthurZucker ArthurZucker changed the title fix if else issues `tokenizer` train from iterator without pre_tokenizers 1 year ago
HuggingFaceDocBuilderDev
ArthurZucker add a test
78b71b61
ArthurZucker ArthurZucker requested a review from Rocketknight1 Rocketknight1 1 year ago
ArthurZucker Merge branch 'main' into fix-tokenizer
e0ac2160
ArthurZucker fix the test
0b155870
ArthurZucker Merge branch 'fix-tokenizer' of github.com:huggingface/transformers i…
c483c6a7
ArthurZucker style
7ba3e76d
ArthurZucker ArthurZucker merged eb4579cf into main 1 year ago
ArthurZucker ArthurZucker deleted the fix-tokenizer branch 1 year ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum