transformers
Fix: use `TokenizersBackend` for Olmo3 to preserve custom `pre_tokenizer`
#44294
Merged
Merged
Fix: use `TokenizersBackend` for Olmo3 to preserve custom `pre_tokenizer` #44294
ArthurZucker
merged
1 commit into
huggingface:main
from
mario-sanz:fix-olmo3-tokenizer-class
fix: use TokenizersBackend for olmo3 to preserve custom pre_tokenizer
2b957f2d
ArthurZucker
approved these changes
on 2026-02-26
Login to write a write a comment.
Login via GitHub