diffusers

[LoRA] refactor lora loading at the model-level

#11719

Merged