transformers
e0aeab4c - [GPT2] Fix encoder_attention_mask being silently discarded in cross-attention (#47946)

Commit
2 days ago
[GPT2] Fix encoder_attention_mask being silently discarded in cross-attention (#47946) * [Bug] Fix encoder_attention_mask being silently discarded in GPT2 cross-attention The new attn mask interface (#42848) rewired GPT2Model and its decision_transformer copy to build the cross-attention mask via create_bidirectional_mask, but added an encoder_attention_mask = None line directly above the call that overwrote the incoming mask before it was passed in. As a result, encoder padding was ignored in cross-attention. Remove the line and add a regression test. * Update tests/models/gpt2/test_modeling_gpt2.py --------- Co-authored-by: Matt <Rocketknight1@users.noreply.github.com>
Parents
Loading