DeepSpeed
bd52453e - Fix BF16 owned-fragment gradient reconstruction on current master

Commit
18 days ago
Fix BF16 owned-fragment gradient reconstruction on current master Contribute only the owned slice of each replicated BF16 gradient buffer and index BF16 gradients by the full parameter list. Remove the obsolete graph-harvesting access while preserving per-gradient correction cleanup and the existing default. Cover split and unsplit parameters across two optimizer steps, GAS one/two, BF16 Stage 1 and ZeRO Stage 2 with independent gradient and clipping references. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: yh0903 <helloyu0903@gmail.com>
Author
Parents
Loading