transformers
6793f0cf - Fix bug in slow tokenizer conversion, make it a lot faster (#24266)
Commit
View On GitHub
Commit
2 years ago
Fix bug in slow tokenizer conversion, make it a lot faster (#24266) * Make conversion faster, fix None vs 0 bug * Add second sort for consistency * Update src/transformers/convert_slow_tokenizer.py Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com> --------- Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
Author
stephantul stephantul
Parents
1609a436
FAQ Terms Privacy Refunds Impressum
Loading