transformers
dced2624 - Update tokenization_bertweet.py (#16941)

Commit
3 years ago
Update tokenization_bertweet.py (#16941) The emoji version must be either 0.5.4 or 0.6.0. Newer emoji versions have been updated to newer versions of the Emoji Charts, thus not consistent with the one used for pre-processing the pre-training Tweet corpus (i.e. not consistent with the vocab).
Author
Parents
Loading