transformers
Fix bug in slow tokenizer conversion, make it a lot faster
#24266

Merged

Commits

Make conversion faster, fix None vs 0 bug

stephantul committed 2 years ago
Add second sort for consistency

stephantul committed 2 years ago
Update src/transformers/convert_slow_tokenizer.py

stephantul committed 2 years ago

Loading