transformers
1240be3e
- Merge pull request #2312 from vitaliyradchenko/fix_special_and_add_tokens_loading
Go
Login via GitHub
Home
Pricing
FAQ
Install
Login
via GitHub
Commit
View On
GitHub
Commit
6 years ago
Merge pull request #2312 from vitaliyradchenko/fix_special_and_add_tokens_loading Correct tokenization for special and added tokens
References
#2312 - Correct tokenization for special and added tokens
Author
thomwolf
Parents
cea04a24
b262577d
Loading