transformers
a7d9d949 - Keep image processor backends in sync on keys and dtypes (#48739)

Commit
6 days ago
Keep image processor backends in sync on keys and dtypes (#48739) * Keep image processor backends in sync on keys and dtypes The torchvision backends of Idefics2, Idefics3 and SmolVLM allocated the batch pixel_attention_mask buffer without a dtype, so the int64 masks built by pad() were stored as float32 while the PIL backend returned int64. The Fuyu PIL backend computed the original image sizes but never returned them, so its output was missing the image_sizes key the torchvision backend has. test_backends_equivalence only compared pixel_values, and the Idefics2, Idefics3 and SmolVLM overrides compared the masks as pixel_attention_mask.float(), which hid the dtype difference. Compare every returned key instead: same keys, same dtype and shape, exact equality for integral outputs and the existing tolerances for float ones. * Compare non-tensor outputs and drop the mask-only helper _assert_encodings_equivalence now walks lists and tuples and compares everything else exactly, so rows, cols and other list outputs are checked instead of skipped. The model overrides keep only their inputs and use it for every key. * Drop the SmolVLM and Idefics3 backend equivalence overrides With torchvision >= 0.27 the torchvision backend supports LANCZOS on CPU, so the base tests pass for both models and the BICUBIC workaround is not needed anymore. --------- Co-authored-by: Raushan Turganbay <raushan@huggingface.co>
Author
Parents
Loading