diffusers

Fix unique memory address when doing group-offloading with disk

#11767

Merged