DeepSpeed
43070642 - Track global_samples by actual micro-batch count in unmanaged mode

Commit
4 days ago
Track global_samples by actual micro-batch count in unmanaged mode In unmanaged gradient accumulation, the caller owns the boundary and the number of backward() calls per step is variable, so advancing global_samples by the fixed train_batch_size() was incorrect. Count backward() calls since the last step() and advance global_samples by the actual micro-batch count. Add a test exercising a varying number of backward() calls per step, matching a managed manual-boundary reference and validating the global_samples accounting. Signed-off-by: Olatunji Ruwase <tunji.ruwase@snowflake.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Parents
Loading