transformers
e3505cd4 - Replace `Tokenizer` with `PreTrainedTokenizerFast` in `ContinuousBatchProcessor` (#39858)

Commit

1 year ago

Replace `Tokenizer` with `PreTrainedTokenizerFast` in `ContinuousBatchProcessor` (#39858) Replace Tokenizer with PreTrainedTokenizerFast in ContinuousBatchProcessor

References

#39858 - Replace `Tokenizer` with `PreTrainedTokenizerFast` in `ContinuousBatchProcessor`

Author

qgallouedec

Parents

380b2a03

transformers e3505cd4 - Replace `Tokenizer` with `PreTrainedTokenizerFast` in `ContinuousBatchProcessor` (#39858)

transformers
e3505cd4 - Replace `Tokenizer` with `PreTrainedTokenizerFast` in `ContinuousBatchProcessor` (#39858)