DeepSpeed
[AffineIR] Affine storage maps for live ZeRO parameters and optimizer states
#8885

Merged

[AffineIR] Affine storage maps for live ZeRO parameters and optimizer states #8885

delock merged 12 commits into deepspeedai:master from jinyouzhi:zero-storage-affine-map

Add affine storage maps for ZeRO parameters and optimizer states

95934d8c

Compose layout maps with their storage maps

d82532e4

Describe AutoTP parameter storage under ZeRO

f5f9015c

Specify live storage maps for phase 2

f32a995e

Describe nested storage with N-D boxes

17143322

Gather optimizer fragment ranges in a gloo-compatible shape

ca87f28b

Fix stat size lookup in checkpoint conversion test

683217d2

jinyouzhi changed the title ~~Affine storage maps for live ZeRO / AutoTP parameters and optimizer states~~ [AffineIR] Affine storage maps for live ZeRO / AutoTP parameters and optimizer states 7 days ago

jinyouzhi changed the title ~~[AffineIR] Affine storage maps for live ZeRO / AutoTP parameters and optimizer states~~ [AffineIR] Affine storage maps for live ZeRO parameters and optimizer states 7 days ago

Merge branch 'master' into zero-storage-affine-map

33125856

Keep nested boxes dense in every replica and support scalars

2d8d873e

jinyouzhi marked this pull request as ready for review 7 days ago

jinyouzhi requested a review from