transformers
d09f53a8 - Remove stale and redundant _no_split_modules entries (#47645)

Commit
90 days ago
Remove stale and redundant _no_split_modules entries (#47645) * Remove stale and redundant _no_split_modules entries Some models list `_no_split_modules` entries naming classes that do not exist anywhere in transformers, so `device_map` never matches them and the modules they were meant to keep together get split across devices: - idefics2: Idefics2DecoderLayer - idefics3: Idefics3DecoderLayer - video_llava: VideoLlavaVisionAttention Others hardcode classes owned by a submodel that is loaded through the Auto* classes. `post_init` already collects `_no_split_modules` from child submodels, so these entries are redundant, and in blip_2's case actively wrong: it declared both T5Block and OPTDecoderLayer regardless of which language model the checkpoint actually uses. - blip_2: T5Block, OPTDecoderLayer - depth_anything: DPTViTEmbeddings (the backbone is an AutoBackbone, so this never matched; it now correctly reports the backbone's Dinov2Layer) - llava_next: LlamaDecoderLayer - moshi: MimiTransformerLayer The resulting runtime sets are more accurate than the hardcoded lists were. Generated modeling files updated with the modular converter, which propagates the removals to smolvlm, llava_next_video, llava_onevision, kyutai_speech_to_text and prompt_depth_anything. deepseek_ocr2 and granite4_vision only shift the position of their own override. * update modular _no_split_modules * switch to attributeerror * fix dac
Author
Parents
Loading