[Deepspeed] deepspeed auto grad accum (#3630)
* deepspeed auto grad accum
* add tests for grad accum
* use tiny-random-gpt2
* Update tests/deepspeed/test_deepspeed_gradient_accumulation.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* fix redundant code
* set_gradient_accumulation_boundary is always there
* remove unused helper
* no need for this
* full revert
* Apply style fixes
* get_global_grad_norm is always there
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
