Keep image processor backends in sync on keys and dtypes (#48739)
* Keep image processor backends in sync on keys and dtypes
The torchvision backends of Idefics2, Idefics3 and SmolVLM allocated the
batch pixel_attention_mask buffer without a dtype, so the int64 masks built
by pad() were stored as float32 while the PIL backend returned int64. The
Fuyu PIL backend computed the original image sizes but never returned them,
so its output was missing the image_sizes key the torchvision backend has.
test_backends_equivalence only compared pixel_values, and the Idefics2,
Idefics3 and SmolVLM overrides compared the masks as
pixel_attention_mask.float(), which hid the dtype difference. Compare every
returned key instead: same keys, same dtype and shape, exact equality for
integral outputs and the existing tolerances for float ones.
* Compare non-tensor outputs and drop the mask-only helper
_assert_encodings_equivalence now walks lists and tuples and compares
everything else exactly, so rows, cols and other list outputs are checked
instead of skipped. The model overrides keep only their inputs and use it
for every key.
* Drop the SmolVLM and Idefics3 backend equivalence overrides
With torchvision >= 0.27 the torchvision backend supports LANCZOS on CPU,
so the base tests pass for both models and the BICUBIC workaround is not
needed anymore.
---------
Co-authored-by: Raushan Turganbay <raushan@huggingface.co>