transformers
b7698930 - Use centralized get_text_with_replacements for token expansion

Commit
157 days ago
Use centralized get_text_with_replacements for token expansion Migrate the image/video/audio token expansion of these processors to the shared ProcessorMixin helper introduced in #45493 instead of bespoke logic: each processor now defines `replace_image_token`/`replace_video_token`/ `replace_audio_token` and routes expansion through `get_text_with_replacements`. Models: deepseek_ocr2, colqwen2, ernie4_5_vl_moe (image+video), glm_image, llava_next (+ granite4_vision via modular), llava_onevision, granite_speech. Note: `get_text_with_replacements` mutates the text list in place, so callers that must not edit the caller's input pass a copy.
Author
Parents
Loading