DeepSpeed
parallelize writing of layer checkpoint files across data parallel instances
#1419
Merged
Merged
parallelize writing of layer checkpoint files across data parallel instances #1419
tjruwase
merged
13 commits into
deepspeedai:master
from
adammoody:layerckpt
adammoody
changed the title
tjruwase
commented
on 2022-09-20
parallelize layer checkpoints across data parallel groups
8fef9f6c
adammoody
commented
on 2022-09-20
use partition_uniform to determine start/end index values
c64a7d4e
formatting fix
6f8c9d1e
Merge branch 'master' into layerckpt
fa99397f
Merge branch 'master' into layerckpt
f05dc913
config: add option for parallel write of layer checkpoints in pipelin…
ed8bc48e
yapf fixes
92f6a840
enable parallel layer write according to config param
2dbf0a4f
avoid extraneous makedir when rank 0 writes all layers
2f311e99
Merge branch 'master' into layerckpt
27002cfa
tjruwase
approved these changes
on 2022-10-10
Merge branch 'master' into layerckpt
f54324a5
Merge branch 'master' into layerckpt
6d5518b3
Merge branch 'master' into layerckpt
0e4d92b7
Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub