Remove unnecessary device synchronization for stage 2 (#2500)
* Remove unnecessary device synchronization for stage 2
* Remove unnecessary device synchronization for stage 2
Co-authored-by: liyidong.lyd <liyidong.lyd@alibaba-inc.com>
Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: Joe Mayer <114769929+jomayeri@users.noreply.github.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
