DeepSpeed

0b77d1d9 - add moe+zero ckpt unit test. (#1429)

add moe+zero ckpt unit test. (#1429) * Add unit test to check moe+zero checkpoints * Fix zero stage2 checkpoint loading logic to deal with experts related state dicts.