PR #34972 [`tokenizers`] bump to 0.21

[`tokenizers`] bump to 0.21 #34972

ArthurZucker merged 2 commits into main from bump-tokenizerr

bump to 0.21

933f1536

ArthurZucker changed the title ~~bump to 0.21~~ [`tokenizers`] bump to 0.21 1 year ago

ArthurZucker requested a review from

LysandreJik 1 year ago

LysandreJik approved these changes on 2024-11-27

ruidazeng approved these changes on 2024-11-28

ruidazeng approved these changes on 2024-11-30

Merge branch 'main' into bump-tokenizerr

9db8b268

ArthurZucker merged 93f87d3c into main 1 year ago

ArthurZucker deleted the bump-tokenizerr branch 1 year ago

Reviewers

LysandreJik

ruidazeng

Assignees

No one assigned

Labels

None yet

Milestone

No milestone