Remove ambiguous `padding_mask` and instead use a 2D->4D Attn Mask Mapper (#26792)
* [Attn Mask Converter] refactor attn mask
* up
* Apply suggestions from code review
Co-authored-by: fxmarty <9808326+fxmarty@users.noreply.github.com>
* improve
* rename
* better cache
* renaming
* improve more
* improve
* fix bug
* finalize
* make style & make fix-copies
* correct more
* start moving attention_mask
* fix llama
* improve falcon
* up
* improve more
* improve more
* Update src/transformers/models/owlv2/modeling_owlv2.py
* make style
* make style
* rename to converter
* Apply suggestions from code review
---------
Co-authored-by: fxmarty <9808326+fxmarty@users.noreply.github.com>