PR #42389 [`Mistral Tokenizers`] Fix tokenizer detection

[`Mistral Tokenizers`] Fix tokenizer detection #42389

ArthurZucker merged 21 commits into huggingface:main from vasqu:fix-mistral-detection

fix

4c147c9c

vasqu requested a review from

ArthurZucker 182 days ago

vasqu commented on 2025-11-25

sanity check

5606702c

vasqu commented on 2025-11-25

style

bb9a4617

comments

4bdb0df3

make it v5 explicit

7ffbe21b

make explicit fixes possible in local tokenizers

0e509d35

ArthurZucker commented on 2025-11-25

remove hub usage on local

bc0c2f04

fix

9aaf2c04

extend test for no config case

3857739a

vasqu added for patch

ArthurZucker approved these changes on 2025-11-25

move mistral patch outside to separate fn

f56961fe

fix local path only

1bebb90a

add a tes

15bf24f3

make sure test does not pass before this PR

d3970fdf

styling

2a1c8e3b

make sure it exists

ee332ad6

fix

82cd35df

fix

5d8546f4

rename

ede0e3f2

1f41bc50

Merge branch 'fix-mistral-detection' of github.com:vasqu/transformers…

a1cabf3f

last nit i hope lord

4dda987b

ArthurZucker merged b6055550 into main 181 days ago

vasqu deleted the fix-mistral-detection branch 159 days ago

Reviewers

ArthurZucker

Assignees

No one assigned

Labels

for patch

Milestone

No milestone

transformers [`Mistral Tokenizers`] Fix tokenizer detection #42389 Merged

[`Mistral Tokenizers`] Fix tokenizer detection #42389

transformers
[`Mistral Tokenizers`] Fix tokenizer detection
#42389

Merged