transformers
9b6e5e20 - fix(tokenizers): stop treating a missing transformers_version as Mistral (#49733)

Commit
8 hours ago
fix(tokenizers): stop treating a missing transformers_version as Mistral (#49733) * fix(tokenizers): stop treating a missing transformers_version as Mistral * Delete tests/tokenization/test_tokenization_fast.py * Update src/transformers/tokenization_utils_tokenizers.py * restore the file * delete changes * nit * handle empty transformers_version, add regression test --------- Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com> Co-authored-by: Arthur <arthur.zucker@gmail.com> Co-authored-by: Ita Zaporozhets <31893021+itazap@users.noreply.github.com>
Author
Parents
Loading