diffusers

[Paint by example] Fix cpu offload for paint by example

#2062

Merged