[LoRA] make `set_adapters()` robust on silent failures. (#9618)
* make set_adapters() robust on silent failures.
* fixes to tests
* flaky decorator.
* fix
* flaky to sd3.
* remove warning.
* sort
* quality
* skip test_simple_inference_with_text_denoiser_multi_adapter_block_lora
* skip testing unsupported features.
* raise warning instead of error.