transformers
7b0c99ad
- Merge pull request #1174 from huggingface/fix_byte_level_added_tokens
Go
Login via GitHub
Home
Pricing
FAQ
Install
Login
via GitHub
Commit
View On
GitHub
Commit
6 years ago
Merge pull request #1174 from huggingface/fix_byte_level_added_tokens Fix byte-level BPE decoding error when using added tokens
References
#1174 - Fix byte-level BPE decoding error when using added tokens
Author
thomwolf
Parents
b6cd856b
31d3373b
Loading