Error with checking args.eval_accumulation_steps to gather tensors (#25819)
* Update trainer.py (error with checking steps in args.eval_accumulation_steps to gather tensors)
While the deprecated code has the correct check (line 3772):
"if args.eval_accumulation_steps is not None and (step + 1) % args.eval_accumulation_steps == 0:"
The current code does not (line 3196):
"if args.eval_accumulation_steps is not None and self.accelerator.sync_gradients:"
We need to check "(step + 1) % args.eval_accumulation_steps == 0". Hence, the line 3196 should be modified to:
"if args.eval_accumulation_steps is not None and (step + 1) % args.eval_accumulation_steps == 0 and self.accelerator.sync_gradients:"
* Fix error with checking args.eval_accumulation_steps to gather tensors