PR #27494 [`tokenizers`] update `tokenizers` version pin

[`tokenizers`] update `tokenizers` version pin #27494

ArthurZucker merged 5 commits into main from pin-tokenizers

update `tokenizers` version pin

1a8ef58f

force tokenizers>=0.15

6b1efdaf

Merge branch 'main' of github.com:huggingface/transformers into pin-t…

3cc8a129

Merge branch 'main' of github.com:huggingface/transformers into pin-t…

2de05601

ArthurZucker requested a review from

LysandreJik 2 years ago

LysandreJik approved these changes on 2023-11-15

LysandreJik commented on 2023-11-15

use 0.14

a580473f

ArthurZucker merged 3d1a7bf4 into main 2 years ago

ArthurZucker deleted the pin-tokenizers branch 2 years ago

Reviewers

LysandreJik

Assignees

No one assigned

Labels

None yet

Milestone

No milestone