[Attention Mask] Refactor all encoder-decoder attention mask (#27086)
* [FA2 Bart] Add FA2 to all Bart-like
* better
* Refactor attention mask
* remove all customized atteniton logic
* format
* mass rename
* replace _expand_mask
* replace _expand_mask
* mass rename
* add pt files
* mass replace & rename
* mass replace & rename
* mass replace & rename
* mass replace & rename
* Update src/transformers/models/idefics/modeling_idefics.py
* fix more
* clean more
* fix more
* make style
* fix again
* finish
* finish
* finish
* finish
* finish
* finish
* finish
* finish
* finish
* finish
* Apply suggestions from code review
* Apply suggestions from code review
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* small fix mistral
* finish
* finish
* finish
* finish
---------
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>