transformers
Migration guide: tokenization
#42477
Merged

Migration guide: tokenization #42477

LysandreJik
LysandreJik Adjust tokenization part
b8eaecdf
LysandreJik LysandreJik requested a review from itazap itazap 199 days ago
LysandreJik LysandreJik requested a review from ArthurZucker ArthurZucker 199 days ago
ArthurZucker
ArthurZucker approved these changes on 2025-11-28
itazap
itazap commented on 2025-11-28
LysandreJik Apply suggestions from code review
c0e86588
LysandreJik Ita's changes
f4f775c1
LysandreJik LysandreJik merged fa3cf839 into main 196 days ago
LysandreJik LysandreJik deleted the adjust-tokenization-migration-guide branch 196 days ago
HuggingFaceDocBuilderDev

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
Milestone