[Core] Fix/pipeline without text encoders for SDXL (#5301)
* fix: sdxl pipeline when unet is not available.
* fix moe
* account for text
* ifx more
* don't make unet optional.
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* split conditionals.
* add optional components to sdxl pipeline
* propagate changes to the rest of the pipelines.
* add: test
* add to all
* fix: rest of the pipelines.
* use pipeline_class variable
* separate pipeline mixin
* use safe_serialization
* fix: test
* access actual output.
* add: optional test to adapter and ip2p sdxl pipeline tests/
* add optional test to controlnet sdxl.
* fix tests
* fix ip2p tests
* fix more
* fifx more.
* use np output type.
* fix for StableDiffusionXLMultiControlNetPipelineFastTests.
* fix: SDXLOptionalComponentsTesterMixin
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* fix tests
* Empty-Commit
* revert previous
* quality
* fix: test
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
