diffusers

Fix group offloading synchronization bug for parameter-only GroupModule's

#12077

Merged