DeepSpeed
Low-precision master params/grads/optimizer states
#7700
Merged

Low-precision master params/grads/optimizer states #7700

tohtana
add options for pure bf16
a4c63626
add low precision master weight mode
b5dcaa84
allow fp16+z1
dbd39ae2
test combination with autocast
88e0bbdf
update document
31369929
tohtana tohtana marked this pull request as ready for review 1 year ago
tohtana tohtana requested a review from tjruwase tjruwase 1 year ago
tohtana tohtana requested a review from loadams loadams 1 year ago
tohtana Merge branch 'master' into tohtana/pure_bf16_config
54316e49
fix formatting
a2f69418
sfc-gh-truwase Merge branch 'master' into tohtana/pure_bf16_config
dc1f4a79
sfc-gh-truwase
sfc-gh-truwase commented on 2025-12-02
sfc-gh-truwase
sfc-gh-truwase commented on 2025-12-02
sfc-gh-truwase
sfc-gh-truwase approved these changes on 2025-12-02
tohtana Merge branch 'master' into tohtana/pure_bf16_config
886b1fc0
add comment
71ac2af3
simplify description in config json
5400979e
fix sphinx format issue (subsections not rendered)
c70e15b1
tohtana Merge branch 'master' into tohtana/pure_bf16_config
769339ca
tohtana tohtana enabled auto-merge (squash) 353 days ago
tohtana tohtana merged 39a682d7 into master 353 days ago
bigyanghimire

Login to write a write a comment.

Login via GitHub

Assignees
No one assigned
Labels
Milestone