[Flux Dreambooth lora] add latent caching (#9160)
* add ostris trainer to README & add cache latents of vae
* add ostris trainer to README & add cache latents of vae
* style
* readme
* add test for latent caching
* add ostris noise scheduler
https://github.com/ostris/ai-toolkit/blob/9ee1ef2a0a2a9a02b92d114a95f21312e5906e54/toolkit/samplers/custom_flowmatch_sampler.py#L95
* style
* fix import
* style
* fix tests
* style
* --change upcasting of transformer?
* update readme according to main
* keep only latent caching
* add configurable param for final saving of trained layers- --upcast_before_saving
* style
* Update examples/dreambooth/README_flux.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
* Update examples/dreambooth/README_flux.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
* use clear_objs_and_retain_memory from utilities
* style
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
