DeepSpeed
b2145896 - Fix: ZenFlow Adam integration for updated PyTorch backward flow (#7759) (#7771)

Commit
62 days ago
Fix: ZenFlow Adam integration for updated PyTorch backward flow (#7759) (#7771) This PR refactors ZenFlow’s integration with DeepSpeed Adam/AdamW optimizers to adapt to recent changes in PyTorch’s backward execution model. It is a follow-up to #7759 and adapts ZenFlow to recent PyTorch backward changes. It is a fast patch to restore correct behavior under the new loss.backward() flow. Signed-off-by: Tingfeng Lan <erc8gx@virginia.edu>
Author
Parents
Loading