Refactoring Trainer, adds `save_only_model` arg and simplifying FSDP integration (#27652)
* add code changes
1. Refactor FSDP
2. Add `--save_only_model` option: When checkpointing, whether to only save the model, or also the optimizer, scheduler & rng state.
3. Bump up the minimum `accelerate` version to `0.21.0`
* quality
* fix quality?
* Revert "fix quality?"
This reverts commit 149330a6abc078827be274db84c8a2d26a76eba1.
* fix fsdp doc strings
* fix quality
* Update src/transformers/training_args.py
Co-authored-by: Zach Mueller <muellerzr@gmail.com>
* please fix the quality issue 😅
* Apply suggestions from code review
Co-authored-by: Benjamin Bossan <BenjaminBossan@users.noreply.github.com>
* address comment
* simplify conditional check as per the comment
* update documentation
---------
Co-authored-by: Zach Mueller <muellerzr@gmail.com>
Co-authored-by: Benjamin Bossan <BenjaminBossan@users.noreply.github.com>