transformers

93766251 - Fix bug with rotating checkpoints (#28009)

Fix bug with rotating checkpoints (#28009) * Fix bug * Write test * Keep back old modification for grad accum steps * Whitespace... * Whitespace again * Race condition * Wait for everyone