[trainer] a few fixes (#9993)

Commit

4 years ago

[trainer] a few fixes (#9993) * trainer fixes * don't switch the model just for deepspeed and mp * correct the fix