transformers
a8f150d3 - Add meta-init test for default configs and fix models that fail it

Commit
164 days ago
Add meta-init test for default configs and fix models that fail it Add `test_can_be_initialized_on_meta_with_default_config` to `ModelTesterMixin`: every architecture must build from its *default* (release-scale) config on the `meta` device. The existing meta test only used the hand-tuned tiny config, which hides config-default bugs. Fixes for models that failed the new test: - `lazy_load_kernel`: degrade to the Python path on *any* hub-resolution failure (was only `FileNotFoundError`/`AssertionError`), so model init never needs the network. Fixes the whole mamba/SSM-hybrid family on `meta`/offline. - ~35 config-default fixes: `num_key_value_heads`/`head_dim` defaulting, `None` MoE sizes, `vocab_size`, special-token vs vocab size, `list`+`tuple` concat, `None` sub-config guards, `prediction_length`, default backbones, layer-type schedules, missing config fields/attributes, etc. - Two test overrides for irreducible shared-config conflicts (one config serving mutually-exclusive heads): t5gemma (enc-dec vs encoder-only) and beit (backbone contract vs semantic-segmentation `out_indices`).
Author
Parents
Loading