diffusers

fix a bug in 2nd order schedulers when using in ensemble of experts config

#5511

Merged