accelerate

9557598c - Add Upcasting for FSDP in Mixed Precision. Add Concept Guide for FSPD and DeepSpeed. (#2674)

Add Upcasting for FSDP in Mixed Precision. Add Concept Guide for FSPD and DeepSpeed. (#2674) * draft fsdp vs ds * reframe to migration doc * updated functionality section * cast to float32 * improvements to float32 casting * some cleanup * addressed @pacman100's comments * Apply some of @muellerz suggestions Co-authored-by: Zach Mueller <muellerzr@gmail.com> * change to subsections * changed the manner upcasting warnings are surfaced * update document to discuss fsdp and ds plugins. minor fixes. * @muellerzr's new suggestions Co-authored-by: Zach Mueller <muellerzr@gmail.com> * explain all-or-nothing * add @pacman100's comments Co-authored-by: Sourab Mangrulkar <13534540+pacman100@users.noreply.github.com> * minor fix --------- Co-authored-by: Yu Chin Fabian Lim <flim@sg.ibm.com> Co-authored-by: Zach Mueller <muellerzr@gmail.com> Co-authored-by: Sourab Mangrulkar <13534540+pacman100@users.noreply.github.com>