Support Kohya-ss style LoRA file format (in a limited capacity) (#3437)
* add _convert_kohya_lora_to_diffusers
* make style
* add scaffold
* match result: unet attention only
* fix monkey-patch for text_encoder
* with CLIPAttention
While the terrible images are no longer produced,
the results do not match those from the hook ver.
This may be due to not setting the network_alpha value.
* add to support network_alpha
* generate diff image
* fix monkey-patch for text_encoder
* add test_text_encoder_lora_monkey_patch()
* verify that it's okay to release the attn_procs
* fix closure version
* add comment
* Revert "fix monkey-patch for text_encoder"
This reverts commit bb9c61e6faecc1935c9c4319c77065837655d616.
* Fix to reuse utility functions
* make LoRAAttnProcessor targets to self_attn
* fix LoRAAttnProcessor target
* make style
* fix split key
* Update src/diffusers/loaders.py
* remove TEXT_ENCODER_TARGET_MODULES loop
* add print memory usage
* remove test_kohya_loras_scaffold.py
* add: doc on LoRA civitai
* remove print statement and refactor in the doc.
* fix state_dict test for kohya-ss style lora
* Apply suggestions from code review
Co-authored-by: Takuma Mori <takuma104@gmail.com>
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>