transformers
ebc36108
- [tokenizers] Fix #5081 and improve backward compatibility (#5125)
Go
Login via GitHub
Home
Pricing
FAQ
Install
Login
via GitHub
Commit
View On
GitHub
Commit
5 years ago
[tokenizers] Fix #5081 and improve backward compatibility (#5125) * fix #5081 and improve backward compatibility (slightly) * add nlp to setup.cfg - style and quality * align default to previous default * remove test that doesn't generalize
References
#5125 - [tokenizers] Fix #5081 and improve backward compatibility
Author
thomwolf
Parents
d2a7c86d
Loading