transformers
68bb33d7
- Fixing OPT fast tokenizer option. (#18753)
Go
Login via GitHub
Home
Pricing
FAQ
Install
Login
via GitHub
Commit
View On
GitHub
Commit
3 years ago
Fixing OPT fast tokenizer option. (#18753) * Fixing OPT fast tokenizer option. * Remove dependency on `pt`. * Move it to GPT2 tokenization tests. * Added a few tests.
References
#18753 - Fixing OPT fast tokenizer option.
Author
Narsil
Parents
578e18e0
Loading