PR #2284 Universal checkpoint for zero stage 1

Refactor universal checkpointing and tensor fragments

4b87f300

Merge branch 'master' into olruwase/refactor_universal_checkpoint

4317b846

Formatting

dfc816df

Merge branch 'master' into olruwase/refactor_universal_checkpoint

21aa55a9

Merge branch 'master' into olruwase/refactor_universal_checkpoint

c6838919

Merge branch 'master' into olruwase/refactor_universal_checkpoint

89df0b3c

Merge branch 'master' into olruwase/refactor_universal_checkpoint

622e7ab8

Merge branch 'master' into olruwase/refactor_universal_checkpoint

115fe422

Merge branch 'master' into olruwase/refactor_universal_checkpoint

3c09adaa

Merge branch 'master' into olruwase/refactor_universal_checkpoint

d40b9236

Support zero stage1; Expand TP dim

7cf02358

tjruwase requested a review from

jeffra 3 years ago

tjruwase requested a review from

mrwyattii 3 years ago

tjruwase requested a review from

samyam 3 years ago

tjruwase requested a review from

ShadenSmith 3 years ago

tjruwase requested a review from

conglongli 3 years ago

tjruwase requested a review from

awan-10 3 years ago

tjruwase requested a review from

cli99 3 years ago

tjruwase requested a review from

eltonzheng 3 years ago

tjruwase requested a review from

minjiaz 3 years ago

tjruwase requested a review from

RezaYazdaniAminabadi 3 years ago

tjruwase requested a review from

duli2012 3 years ago

tjruwase requested a review from

yaozhewei 3 years ago

tjruwase requested a review from

arashb 3 years ago

tjruwase requested a review from

xiaoxiawu-microsoft 3 years ago

tjruwase requested a review from

samadejacobs 3 years ago

tjruwase requested a review from

cmikeh2 3 years ago

Merge branch 'master' into olruwase/zero_1_2_universal_ckpt

ece4ce37

Remove debug prints

cae21725

Merge branch 'olruwase/zero_1_2_universal_ckpt' of github.com:microso…

48b62c29

tjruwase closed this 3 years ago

tjruwase reopened this 3 years ago

Merge branch 'master' into olruwase/zero_1_2_universal_ckpt

1ace0257

Detect sharded optimizer state

529f2d88

Merge branch 'master' into olruwase/zero_1_2_universal_ckpt

93246ece

Merge master

45320590

tjruwase removed review request from

arashb 3 years ago

tjruwase removed review request from

ShadenSmith 3 years ago

tjruwase removed review request from

cmikeh2 3 years ago

tjruwase removed review request from

duli2012 3 years ago

tjruwase removed review request from

samyam 3 years ago

tjruwase removed review request from

conglongli 3 years ago

tjruwase removed review request from

awan-10 3 years ago

tjruwase removed review request from

samadejacobs 3 years ago

tjruwase removed review request from

cli99 3 years ago

tjruwase removed review request from

yaozhewei 3 years ago

tjruwase removed review request from

eltonzheng 3 years ago

tjruwase removed review request from

minjiaz 3 years ago

tjruwase removed review request from

RezaYazdaniAminabadi 3 years ago

tjruwase removed review request from

xiaoxiawu-microsoft 3 years ago

Format fixes

024baa8a

Merge branch 'master' into olruwase/zero_1_2_universal_ckpt

a2f592d3

Merge branch 'master' into olruwase/zero_1_2_universal_ckpt

697287d0

tjruwase changed the title ~~Universal checkpoint for zero stage 1 & 2~~ Universal checkpoint for zero stage 1 3 years ago

Merge branch 'master' into olruwase/zero_1_2_universal_ckpt

a5e99007

Merge branch 'master' into olruwase/zero_1_2_universal_ckpt

bfefdec6

tjruwase requested a review from

GuanhuaWang 3 years ago

tjruwase removed review request from

GuanhuaWang 3 years ago

Merge branch 'master' into olruwase/zero_1_2_universal_ckpt

b78c5f64

tjruwase marked this pull request as draft 3 years ago

Encode reshaping guide

e3465292

Merge branch 'olruwase/zero_1_2_universal_ckpt' of github.com:microso…

1447fc23

Merge branch 'master' into olruwase/zero_1_2_universal_ckpt

1bebe2e9

tjruwase marked this pull request as ready for review 3 years ago

Merge branch 'master' into olruwase/zero_1_2_universal_ckpt

a5afb811

Merge branch 'master' into olruwase/zero_1_2_universal_ckpt

bb85c69b

More symbolic constants

c929f89f

Merge branch 'master' into olruwase/zero_1_2_universal_ckpt

6ba5ad46

mrwyattii approved these changes on 2022-09-26

Merge branch 'master' into olruwase/zero_1_2_universal_ckpt

83ecf190

Merge branch 'master' into olruwase/zero_1_2_universal_ckpt

0f1738d6

Merge branch 'master' into olruwase/zero_1_2_universal_ckpt

6c6823fe

Merge branch 'master' into olruwase/zero_1_2_universal_ckpt

16d26b9c

Merge branch 'master' into olruwase/zero_1_2_universal_ckpt

a26458a8

Merge branch 'master' into olruwase/zero_1_2_universal_ckpt

48d291a7

Merge branch 'master' into olruwase/zero_1_2_universal_ckpt

e358ee6d

Merge branch 'master' into olruwase/zero_1_2_universal_ckpt

4f51d0a9

Merge branch 'master' into olruwase/zero_1_2_universal_ckpt

f30ad0fe

Merge branch 'master' into olruwase/zero_1_2_universal_ckpt

88bf0452

Merge branch 'master' into olruwase/zero_1_2_universal_ckpt

329cb826

Merge branch 'master' into olruwase/zero_1_2_universal_ckpt

0d9dc104

Merge branch 'master' into olruwase/zero_1_2_universal_ckpt

37485ae1

tjruwase merged 799120e7 into master 3 years ago

mrwyattii deleted the olruwase/zero_1_2_universal_ckpt branch 2 years ago

DeepSpeed
Universal checkpoint for zero stage 1
#2284

Merged

Universal checkpoint for zero stage 1 #2284

DeepSpeed Universal checkpoint for zero stage 1 #2284 Merged

Universal checkpoint for zero stage 1 #2284

DeepSpeed
Universal checkpoint for zero stage 1
#2284

Merged