transformers
c2c78f79 - Migrate remaining multimodal processors to get_text_with_replacements

Commit
155 days ago
Migrate remaining multimodal processors to get_text_with_replacements Route image/video/audio placeholder-token expansion through the shared ProcessorMixin.get_text_with_replacements helper (#45493) for the processors that still used bespoke `.replace`/`re.sub`/sentinel logic, each now defining replace_image_token/replace_video_token/replace_audio_token (or building the replacement list inline where the expansion is context-dependent): pixtral, internvl, qianfan_ocr, lighton_ocr, qwen2_audio, csm, video_llava, smolvlm, higgs_audio_v2, emu3, cohere2_vision, kosmos2, florence2, deepseek_vl, deepseek_vl_hybrid, gemma3, minicpmv4_6, paddleocr_vl, phi4_multimodal, qwen2_5_omni, qwen3_omni_moe. Output is byte-for-byte identical (validated against each model's existing token-count tests, or byte-equivalence harnesses where tests are gated by torchaudio/unreleased checkpoints). Mllama is intentionally left as-is (one image token per image + cross-attention, nothing to expand). gemma3n is deferred: its Hub chat template emits soft tokens, so it needs a soft-token anchored migration rather than a boi/eoi one.
Author
Parents
Loading