DeepSpeed
DeepSpeedZeroOptimizer: refactor bit16 flattening to support more accelerators
#4833
Merged
Merged
DeepSpeedZeroOptimizer: refactor bit16 flattening to support more accelerators #4833
tjruwase
merged
11 commits into
deepspeedai:master
from
nelyahu:zeroOptParamsFlatenning
ZeroOptimizer: avoid storage sharing when flatenning params
45d7a034
Merge branch 'microsoft:master' into zeroOptParamsFlatenning
324e33f9
nelyahu
changed the title 
tjruwase
commented
on 2023-12-18
tjruwase
commented
on 2023-12-18
tjruwase
commented
on 2023-12-18
tjruwase
commented
on 2023-12-18
tjruwase
commented
on 2023-12-18
Merge branch 'master' into zeroOptParamsFlatenning
2c794564
Use meta tensor instead of reconstructing CPU tensors
cbd6c70b
Merge branch 'master' into zeroOptParamsFlatenning
067e079f
tjruwase
commented
on 2024-01-02
fix orig_group_numel: missing accumulation
8b0d4cee
Merge branch 'master' into zeroOptParamsFlatenning
2e41d9ef
tjruwase
approved these changes
on 2024-01-03
Merge branch 'master' into zeroOptParamsFlatenning
890ae610
Merge branch 'master' into zeroOptParamsFlatenning
5ade6863
fix orig_group_numel: move accumulation two lines above
c10b626f
Merge branch 'master' into zeroOptParamsFlatenning
57b01129
Login to write a write a comment.
Login via GitHub