transformers
5787e4c1 - Various tokenizers fixes (#5558)

Commit
5 years ago
Various tokenizers fixes (#5558) * BertTokenizerFast - Do not specify strip_accents by default * Bump tokenizers to new version * Add test for AddedToken serialization
Author
Parents
Loading