fix #14063 for Kandinsky5 pipeline load with device_map=balanced (#14050)
fix bug for Kandinsky5 pipeline load with device_map=balanced
Signed-off-by: Liu, Kaixuan <kaixuan.liu@intel.com>
Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com>