Speedup model loading by 4-5x ⚡ (#11904)
* update
* update
* update
* pin accelerate version
* add comment explanations
* update docstring
* make style
* non_blocking does not matter for dtype cast
* _empty_cache -> clear_cache
* update
* Update src/diffusers/models/model_loading_utils.py
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com>
* Update src/diffusers/models/model_loading_utils.py
---------
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com>
