Fix model parallel issue for altclip model and ChineseClip model (#45487)
* fix model parallel device mismatch issue for altclip model
Signed-off-by: Liu, Kaixuan <kaixuan.liu@intel.com>
* fix model parallel issue for ChineseClip model
Signed-off-by: Liu, Kaixuan <kaixuan.liu@intel.com>
* refine code
Signed-off-by: Liu, Kaixuan <kaixuan.liu@intel.com>
---------
Signed-off-by: Liu, Kaixuan <kaixuan.liu@intel.com>