diffusers

[core] fix group offloading when using torchao

#13276

Merged