[`FA-2`] Fix fa-2 issue when passing `config` to `from_pretrained` (#28043)
* fix fa-2 issue
* fix test
* Update src/transformers/modeling_utils.py
Co-authored-by: fxmarty <9808326+fxmarty@users.noreply.github.com>
* clenaer fix
* up
* add more robust tests
* Update src/transformers/modeling_utils.py
Co-authored-by: fxmarty <9808326+fxmarty@users.noreply.github.com>
* fixup
* Update src/transformers/modeling_utils.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* pop
* add test
---------
Co-authored-by: fxmarty <9808326+fxmarty@users.noreply.github.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>