diffusers

[IP Adapter] Fix `cache_dir` and `local_files_only` for image encoder

#9272

Merged