transformers
Replace `Tokenizer` with `PreTrainedTokenizerFast` in `ContinuousBatchProcessor`
#39858
Merged
Overview
Commits
2
Changes
View On GitHub

Replace `Tokenizer` with `PreTrainedTokenizerFast` in `ContinuousBatchProcessor` #39858

ArthurZucker merged 2 commits into main from fix-cb
qgallouedec
qgallouedec Replace Tokenizer with PreTrainedTokenizerFast in ContinuousBatchProc…
b281af82
qgallouedec qgallouedec requested a review from ArthurZucker ArthurZucker 248 days ago
HuggingFaceDocBuilderDev
qgallouedec Merge branch 'main' into fix-cb
42a1d437
ArthurZucker
ArthurZucker approved these changes on 2025-08-04
ArthurZucker ArthurZucker merged e3505cd4 into main 246 days ago
ArthurZucker ArthurZucker deleted the fix-cb branch 246 days ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum