FSDP grad accum fix (#34645)

Commit

1 year ago

FSDP grad accum fix (#34645) * add gradient accumulation steps tests for fsdp * invert no_sync context to fix training for fsdp