DeepSpeed
set the default to use set_to_none for clearing gradients in BF16 optimizer.
#5434
Merged
Merged
set the default to use set_to_none for clearing gradients in BF16 optimizer. #5434
loadams
merged
8 commits into
deepspeedai:master
from
inkcherry:fix_5175_
set clear_grad default to None, use foreach_zero
678a6e9e
fix typo
6c568acf
Merge branch 'master' into fix_5175_
d1ec5444
tjruwase
commented
on 2024-04-19
fix condition&add detach
6b77946b
Merge branch 'fix_5175_' of https://github.com/inkcherry/DeepSpeed in…
7c9c8a92
tjruwase
commented
on 2024-04-20
Merge branch 'master' into fix_5175_
85f295dc
tjruwase
approved these changes
on 2024-04-22
Merge branch 'master' into fix_5175_
32fc1ced
Merge branch 'master' into fix_5175_
617e28aa
Login to write a write a comment.
Login via GitHub