transformers
1c7c34bc - Improve `PreTrainedTokenizerFast` loading time when there are many added tokens (#31404)

Commit
2 years ago
Improve `PreTrainedTokenizerFast` loading time when there are many added tokens (#31404) * use hash * use hash * update --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Author
Parents
Loading