transformers
Fix AutoTokenizer when no fast tokenizer is available
#13336
Merged
Overview
Commits
2
Changes
View On GitHub

Fix AutoTokenizer when no fast tokenizer is available #13336

sgugger merged 2 commits into master from fix_auto_tokenizer
sgugger
sgugger Fix AutoTokenizer when a tokenizer has no fast version
dbad09f3
sgugger sgugger requested a review from LysandreJik LysandreJik 4 years ago
sgugger Add test
4525edc4
LysandreJik
LysandreJik approved these changes on 2021-08-30
sgugger sgugger merged c4ecd234 into master 4 years ago
sgugger sgugger deleted the fix_auto_tokenizer branch 4 years ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum