[Feat] Support SDXL Kohya-style LoRA (#4287)
* sdxl lora changes.
* better name replacement.
* better replacement.
* debugging
* debugging
* debugging
* debugging
* debugging
* remove print.
* print state dict keys.
* print
* distingisuih better
* debuggable.
* fxi: tyests
* fix: arg from training script.
* access from class.
* run style
* debug
* save intermediate
* some simplifications for SDXL LoRA
* styling
* unet config is not needed in diffusers format.
* fix: dynamic SGM block mapping for SDXL kohya loras (#4322)
* Use lora compatible layers for linear proj_in/proj_out (#4323)
* improve condition for using the sgm_diffusers mapping
* informative comment.
* load compatible keys and embedding layer maaping.
* Get SDXL 1.0 example lora to load
* simplify
* specif ranks and hidden sizes.
* better handling of k rank and hidden
* debug
* debug
* debug
* debug
* debug
* fix: alpha keys
* add check for handling LoRAAttnAddedKVProcessor
* sanity comment
* modifications for text encoder SDXL
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* denugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* up
* up
* up
* up
* up
* up
* unneeded comments.
* unneeded comments.
* kwargs for the other attention processors.
* kwargs for the other attention processors.
* debugging
* debugging
* debugging
* debugging
* improve
* debugging
* debugging
* more print
* Fix alphas
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* clean up
* clean up.
* debugging
* fix: text
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Batuhan Taskaya <batuhan@python.org>
