DeepSpeed
b6e24adb - fixes in _partition_param_sec function (#5613)

Commit
1 year ago
fixes in _partition_param_sec function (#5613) There are few fixes: - When param.ds_secondary_tensor is not None and the param has not been updated we don't need to update the param.ds_secondary_tensor. - In HPU the 2nd tensor partition will always be completed before the all-gather, so we don't need to add synchronize().
Author
Parents
Loading