Fix `_init_weights` gaps in 7 audio models and enable `test_can_init_all_missing_weights` for them (#49386)
* Initialize masked_spec_embed in _init_weights for 7 audio models
Wav2Vec2, Wav2Vec2-Conformer, Data2VecAudio, WavLM, UniSpeech, UniSpeechSat,
and SEW create `masked_spec_embed` with `uniform_()` in `__init__`, but
`from_pretrained` skips that and initializes missing weights through
`_init_weights`, which had no branch for it. Checkpoints that lack the
tensor (e.g. facebook/wav2vec2-base-960h) therefore got uninitialized
memory, so training with SpecAugment read NaN or garbage. Add the same
branch HuBERT already has, and a regression test covering all seven models.
Fixes #49385
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Merge the masked_spec_embed check into the elif condition
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* remove redundant test
* Test masked_spec_embed init in each model's own test file
* Enable test_can_init_all_missing_weights for 7 audio models
Replace the per-model masked_spec_embed tests with the common
test_can_init_all_missing_weights, which skips models added before 2023.
A new test_can_init_all_missing_weights_before_2023 attribute opts these
models in. Running it surfaced three more weights that _init_weights did
not handle: AMSoftmaxLoss.weight (XVector head), WavLM's
gru_rel_pos_const, and UniSpeechSat's label_embeddings_concat.
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>