PR #1429 add moe+zero ckpt unit test.

add moe+zero ckpt unit test. #1429

awan-10 merged 9 commits into master from amawa/moe-zero-ckpt-test

add moe+zero ckpt unit test. it is failing as expected.

69f127d0

awan-10 requested a review from

cli99 4 years ago

awan-10 requested a review from

conglongli 4 years ago

awan-10 requested a review from

eltonzheng 4 years ago

awan-10 requested a review from

jeffra 4 years ago

awan-10 requested a review from

minjiaz 4 years ago

awan-10 requested a review from

niumanar 4 years ago

awan-10 requested a review from

RezaYazdaniAminabadi 4 years ago

awan-10 requested a review from

samyam 4 years ago

awan-10 requested a review from

ShadenSmith 4 years ago

awan-10 requested a review from

tjruwase 4 years ago

modify the test.

f976921c

Define helper outside. Fix the Expert-parallel only case for zero+moe.

0645e685

Merge branch 'master' into amawa/moe-zero-ckpt-test

ea03da36

fix format.

81d816c4

Get ep-ranks list. Works for Expert-only but not for E+D.

16598acc

fix format.

63d442e9

Fix both cases. Cleanup.

8d2af5fa

Merge branch 'master' into amawa/moe-zero-ckpt-test

8df06b2d

awan-10 enabled auto-merge (squash) 4 years ago

tjruwase approved these changes on 2021-10-22

awan-10 merged 0b77d1d9 into master 4 years ago

mrwyattii deleted the amawa/moe-zero-ckpt-test branch 2 years ago

Reviewers

tjruwase

cli99

conglongli

eltonzheng

jeffra

minjiaz

niumanar

RezaYazdaniAminabadi

samyam

ShadenSmith

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

DeepSpeed add moe+zero ckpt unit test. #1429 Merged

add moe+zero ckpt unit test. #1429

DeepSpeed
add moe+zero ckpt unit test.
#1429

Merged