Fix dtype casting in swinv2 and swinv2sr to allow non-FP32 inference (#31589)
* Fix dtype casting in modeling_swin2sr to allow non-FP32 inference
* Fix formattting
* Fix for swinv2 too
* Update src/transformers/models/swin2sr/modeling_swin2sr.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/swinv2/modeling_swinv2.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Add FP16 tests for swin2sr and swinv2
* [run_slow] swin2sr, swinv2
* [run_slow] swin2sr, swinv2
---------
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>