transformers
Replace `Tokenizer` with `PreTrainedTokenizerFast` in `ContinuousBatchProcessor`
#39858
Merged
Merged