🚨 Refactor ViT to updated standards (#41693)
* refactor attention
* use mlp instead of intermediate and output
* use modular for deit
* nit
* fix deit (to check) and refactor beit
* use modular for beit
* style
* refactor vit deit beit segformer
* add pos_embed_utils, refactor swin
* Fix after merge + refactor pixio
* refactor most drop_path in transformers
* refactor vit_msn
* fix modular
* refactor vivit and audio_spectrogram_transformer, remove obsolete copied from and add todos
* Fix repo
* fix nit
* Fix drop_path_schedule
* nit
* use modular for pos embed
* reverse DropPath as a util
* add conversion ijepa + fix offload vit_mae
* fix modular
* fix qkv bias in output proj + swin attention per stage
* address review comments
* fix swinn
* improve swin and beit
* last fixes after review
* standardize backbone beit
* fix beit
* fix beit
* fix repo
* changes after review
* Fix beit
* fix after review
* fix unexpected key swin
* Address review
* fix repo
* Fix after review
* fix modular
* fix depth pro
* Fix conversion mappings
* fix style
* Fix failing tests
* fix ci
* Fix two slow failing tests
