transformers
51f4f905 - [Whisper] Fix decoder_position_ids for left-padded batches in longform generation

Commit
26 days ago
[Whisper] Fix decoder_position_ids for left-padded batches in longform generation PR #44130 refactored prepare_inputs_for_generation to stop using cache_position for input slicing, but inadvertently removed the cumsum-based decoder_position_ids computation from decoder_attention_mask. Without it, WhisperDecoder falls back to past_kv.get_seq_length() for all batch items — a single value that ignores per-item padding — giving wrong position embeddings for left-padded sequences. With condition_on_prev_tokens=True and heterogeneous batch sizes, shorter sequences get left-padded, so each item has a different real token length. The restored cumsum correctly computes per-item positions from the decoder_attention_mask regardless of padding. Fixes test_whisper_longform_multi_batch_hard_prev_cond and test_whisper_longform_no_speech_detection (both failing since Feb 2026). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Author
Committer
Parents
Loading