onnxruntime
2ea4ea43 - Fix how Whisper tokenizer is loaded and saved (#25037)

Commit
210 days ago
Fix how Whisper tokenizer is loaded and saved (#25037) ### Description This PR changes the class from which the Whisper tokenizer is loaded. ### Motivation and Context By using `AutoTokenizer` instead of `WhisperTokenizer`, the `save_pretrained` method will also save the `tokenizer.json` file. This file is used in ONNX Runtime GenAI's tokenizers.
Parents
Loading