diffusers
493f9529 - [`PEFT` / `LoRA`] PEFT integration - text encoder (#5058)

Commit

1 year ago

[`PEFT` / `LoRA`] PEFT integration - text encoder (#5058) * more fixes * up * up * style * add in setup * oops * more changes * v1 rzfactor CI * Apply suggestions from code review Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com> * few todos * protect torch import * style * fix fuse text encoder * Update src/diffusers/loaders.py Co-authored-by: Sayak Paul <spsayakpaul@gmail.com> * replace with `recurse_replace_peft_layers` * keep old modules for BC * adjustments on `adjust_lora_scale_text_encoder` * nit * move tests * add conversion utils * remove unneeded methods * use class method instead * oops * use `base_version` * fix examples * fix CI * fix weird error with python 3.8 * fix * better fix * style * Apply suggestions from code review Co-authored-by: Sayak Paul <spsayakpaul@gmail.com> Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com> * Apply suggestions from code review Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com> * add comment * Apply suggestions from code review Co-authored-by: Sayak Paul <spsayakpaul@gmail.com> * conv2d support for recurse remove * added docstrings * more docstring * add deprecate * revert * try to fix merge conflicts * v1 tests * add new decorator * add saving utilities test * adapt tests a bit * add save / from_pretrained tests * add saving tests * add scale tests * fix deps tests * fix lora CI * fix tests * add comment * fix * style * add slow tests * slow tests pass * style * Update src/diffusers/utils/import_utils.py Co-authored-by: Benjamin Bossan <BenjaminBossan@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Benjamin Bossan <BenjaminBossan@users.noreply.github.com> * circumvents pattern finding issue * left a todo * Apply suggestions from code review Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com> * update hub path * add lora workflow * fix --------- Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com> Co-authored-by: Sayak Paul <spsayakpaul@gmail.com> Co-authored-by: Benjamin Bossan <BenjaminBossan@users.noreply.github.com>

References

#5058 - [`PEFT` / `LoRA`] PEFT integration - text encoder

Author

younesbelkada

Parents

b32555a2

Files46

.github/workflows
- pr_test_peft_backend.yml
src/diffusers
- loaders.py
- models
  - lora.py
- pipelines
  - alt_diffusion
    - pipeline_alt_diffusion.py
    - pipeline_alt_diffusion_img2img.py
  - controlnet
    - pipeline_controlnet.py
    - pipeline_controlnet_img2img.py
    - pipeline_controlnet_inpaint.py
    - pipeline_controlnet_inpaint_sd_xl.py
    - pipeline_controlnet_sd_xl.py
    - pipeline_controlnet_sd_xl_img2img.py
  - stable_diffusion
    - pipeline_cycle_diffusion.py
    - pipeline_stable_diffusion.py
    - pipeline_stable_diffusion_attend_and_excite.py
    - pipeline_stable_diffusion_depth2img.py
    - pipeline_stable_diffusion_diffedit.py
    - pipeline_stable_diffusion_gligen.py
    - pipeline_stable_diffusion_gligen_text_image.py
    - pipeline_stable_diffusion_img2img.py
    - pipeline_stable_diffusion_inpaint.py
    - pipeline_stable_diffusion_inpaint_legacy.py
    - pipeline_stable_diffusion_k_diffusion.py
    - pipeline_stable_diffusion_ldm3d.py
    - pipeline_stable_diffusion_model_editing.py
    - pipeline_stable_diffusion_panorama.py
    - pipeline_stable_diffusion_paradigms.py
    - pipeline_stable_diffusion_pix2pix_zero.py
    - pipeline_stable_diffusion_sag.py
    - pipeline_stable_diffusion_upscale.py
    - pipeline_stable_unclip.py
    - pipeline_stable_unclip_img2img.py
  - stable_diffusion_xl
    - pipeline_stable_diffusion_xl.py
    - pipeline_stable_diffusion_xl_img2img.py
    - pipeline_stable_diffusion_xl_inpaint.py
    - pipeline_stable_diffusion_xl_instruct_pix2pix.py
  - t2i_adapter
    - pipeline_stable_diffusion_adapter.py
    - pipeline_stable_diffusion_xl_adapter.py
  - text_to_video_synthesis
    - pipeline_text_to_video_synth.py
    - pipeline_text_to_video_synth_img2img.py
- utils
  - __init__.py
  - import_utils.py
  - peft_utils.py
  - state_dict_utils.py
  - testing_utils.py
tests/lora
- test_lora_layers_old_backend.py
- test_lora_layers_peft.py

diffusers 493f9529 - [`PEFT` / `LoRA`] PEFT integration - text encoder (#5058)

diffusers
493f9529 - [`PEFT` / `LoRA`] PEFT integration - text encoder (#5058)