transformers
5787e4c1
- Various tokenizers fixes (#5558)
Go
Login via GitHub
Home
Pricing
FAQ
Install
Login
via GitHub
Commit
View On
GitHub
Commit
5 years ago
Various tokenizers fixes (#5558) * BertTokenizerFast - Do not specify strip_accents by default * Bump tokenizers to new version * Add test for AddedToken serialization
References
#5558 - Various tokenizers fixes
Author
n1t0
Parents
21f28c34
Loading