transformers
d8c23549 - Fix Moonshine training-loss double-shift (train against labels, not labels[..., 1:]) (#46784)

Commit
15 days ago
Fix Moonshine training-loss double-shift (train against labels, not labels[..., 1:]) (#46784) * Fix Moonshine training-loss double-shift Moonshine right-shifts labels into decoder_input_ids, then computes the loss via ForCausalLMLoss, which shifts again, so it trains against labels[..., 1:]. Use a plain CrossEntropyLoss instead, matching Whisper/Bart and the VisionEncoderDecoder fix in #40863. Adds a regression test (with and without -100 padding). * Regenerate Moonshine streaming model * Mirror the training-loss regression test in moonshine_streaming make fix-repo propagated the loss fix to the streaming model, so the same no-double-shift test should guard it there too.
Author
Parents
Loading