MiCS implementation #2964
samadejacobs
merged
32 commits into
deepspeedai:master
from
dmlc:mics
include mics config and optimizer
5985e2ef
Merge branch 'master' of https://github.com/microsoft/DeepSpeed into …
e5c82845
Merge branch 'master' of https://github.com/microsoft/DeepSpeed into …
cb42e7c7
change private vars to public vars
8978909d
Merge branch 'master' of https://github.com/microsoft/DeepSpeed into …
81923722
Port the init function from stage3
d9dd5584
Merge branch 'master' of https://github.com/microsoft/DeepSpeed into …
ade09d5f
Merge branch 'master' into clean-mics
fccc3ede
adding a model test file for mics
7fcea79d
Merge branch 'master' into clean-mics
6bf13428
adopt to get_acceleartor api and fp16 group defrag
e3e375f9
WIP: porting mics modification to ms master
b865f20b
Merge remote-tracking branch 'ms/master' into clean-mics
a5f8e60a
WIP: included gradient all-reduce among replication groups
28aed966
Merge branch 'master' into clean-mics
8366eb22
WIP: ported hierarchical all gather part
4f7d1563
[Bug fix] using the comm group attached on the param
fdb8706a
Merge branch 'master' into mics
1666f0a9
torch2.0 support
66d44bdb
jeffra
commented
on 2023-03-31
Merge branch 'master' into mics
dbf539b9
remove print
f360f0b7
Merge branch 'master' into mics
05afdbd0
delegate wait op
5367dc8a
Merge branch 'master' into mics
2be1dc6e
[Bug] fix naming
7a713332
adding doc string
2bb43cad
samadejacobs
approved these changes
on 2023-04-24
Merge branch 'master' into mics
068325a0
disabled auto-merge 2 years ago
Manually disabled by user
Manually disabled by user
resolving recursive import
0102dcd2
fix formating, typo and license
29e786cb
Merge branch 'master' into mics
e45836a1
fix license and unit test error
2f9be953
Merge branch 'master' into mics
9e06d7e9
Assignees
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub