transformers

9d14be5c - Add support for ZeRO-2/3 and ZeRO-offload in fairscale (#10354)

Add support for ZeRO-2/3 and ZeRO-offload in fairscale (#10354) * Ass support for ZeRO-2/3 and ZeRO-offload in fairscale * Quality * Rework from review comments * Add doc * Apply suggestions from code review Co-authored-by: Stas Bekman <stas00@users.noreply.github.com> * Address review comments Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>