diffusers

[Textual inversion] Refactor textual inversion to make it cleaner

#5076

Merged