accelerate

e965f100 - Reach through the DeepSpeed optimizer wrapper in AcceleratedOptimizer.eval() (#4130)

Reach through the DeepSpeed optimizer wrapper in AcceleratedOptimizer.eval() (#4130) * Reach through the DeepSpeed optimizer wrapper in AcceleratedOptimizer.eval() train() handles the case where DeepSpeed wraps the user optimizer one level deeper, but eval() never got the matching branch, so a schedule-free optimizer under DeepSpeed is switched into train mode and never switched back. There is no error, just eval running with train-mode weights. Mirror the train() branch in eval() and add CPU regression tests for the wrapped, unwrapped and no-mode-support cases. * Give the no-mode control something to assert The control called train()/eval() on a plain optimizer and asserted nothing, so it only proved the calls did not raise. Run it over the wrapper shape as well, which is the negative case for the reach-through this PR adds, and assert the inner optimizer never grows a mode it did not have. Signed-off-by: Vineeth Sai <vineethsai4444@gmail.com> * Update tests/test_optimizer.py * Update tests/test_optimizer.py * Apply style fixes --------- Signed-off-by: Vineeth Sai <vineethsai4444@gmail.com> Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>