[Imports] Fix many import bugs and make sure that doc builder CI test works correctly (#5176)
* [Doc builder] Ensure slow import for doc builder
* Apply suggestions from code review
* env for doc builder
* fix more
* [Diffusers] Set import to slow as env variable
* fix docs
* fix docs
* Apply suggestions from code review
* Apply suggestions from code review
* fix docs
* fix docs