transformers
e487eaf4 - Fix `_init_weights` gaps in 7 audio models and enable `test_can_init_all_missing_weights` for them (#49386)

Commit
14 days ago
Fix `_init_weights` gaps in 7 audio models and enable `test_can_init_all_missing_weights` for them (#49386) * Initialize masked_spec_embed in _init_weights for 7 audio models Wav2Vec2, Wav2Vec2-Conformer, Data2VecAudio, WavLM, UniSpeech, UniSpeechSat, and SEW create `masked_spec_embed` with `uniform_()` in `__init__`, but `from_pretrained` skips that and initializes missing weights through `_init_weights`, which had no branch for it. Checkpoints that lack the tensor (e.g. facebook/wav2vec2-base-960h) therefore got uninitialized memory, so training with SpecAugment read NaN or garbage. Add the same branch HuBERT already has, and a regression test covering all seven models. Fixes #49385 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Merge the masked_spec_embed check into the elif condition Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * remove redundant test * Test masked_spec_embed init in each model's own test file * Enable test_can_init_all_missing_weights for 7 audio models Replace the per-model masked_spec_embed tests with the common test_can_init_all_missing_weights, which skips models added before 2023. A new test_can_init_all_missing_weights_before_2023 attribute opts these models in. Running it surfaced three more weights that _init_weights did not handle: AMSoftmaxLoss.weight (XVector head), WavLM's gru_rel_pos_const, and UniSpeechSat's label_embeddings_concat. --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Author
Parents
Loading