pytorch-lightning
Gradient accumulation fix in cross entropy loss
#21386
Open
Overview
Commits
7
Changes
View On GitHub

Gradient accumulation fix in cross entropy loss #21386

Sohaib-Ahmed21 wants to merge 7 commits into Lightning-AI:master from Sohaib-Ahmed21:bugfix/20350_grad_acc_fix
Sohaib-Ahmed21
Sohaib-Ahmed21 Introduce peekable iterator to count number of valid tokens in the gl…
385fd56d
Sohaib-Ahmed21 Scale loss by number of valid tokens in global batch in case of cross…
9b7aa6ff
github-actions github-actions added pl
Sohaib-Ahmed21 Merge branch 'master' into bugfix/20350_grad_acc_fix
7f5f88cd
Sohaib-Ahmed21 Ensure iterator is not None while passing to tee function
95d467d4
Sohaib-Ahmed21 Merge branch 'bugfix/20350_grad_acc_fix' of https://github.com/Sohaib…
fb7dbc87
SkafteNicki
SkafteNicki commented on 2025-12-01
Sohaib-Ahmed21 Merge branch 'master' into bugfix/20350_grad_acc_fix
01fcf620
Sohaib-Ahmed21 Sohaib-Ahmed21 requested a review from SkafteNicki SkafteNicki 15 days ago
Sohaib-Ahmed21 Merge branch 'master' into bugfix/20350_grad_acc_fix
06216207
Sohaib-Ahmed21

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
pl
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum