accelerate
Fix next-token labels and accumulation target weighting
#4303
Merged

Fix next-token labels and accumulation target weighting #4303

Titus-von-Koeller
Titus-von-Koeller Compare autoregressive example updates with a global-token reference
44a635c3
Titus-von-Koeller Preserve the autoregressive example smoke alongside token weighting
2296266b
Titus-von-Koeller Fix next-token labels and normalization in accumulation example
5e7448f8
HuggingFaceDocBuilderDev
Titus-von-Koeller Titus-von-Koeller marked this pull request as ready for review 40 days ago
Titus-von-Koeller Merge upstream main into fix-autoregressive-accumulation-targets
e95678f4
SunMarc
SunMarc commented on 2026-09-29
Titus-von-Koeller
Titus-von-Koeller Titus-von-Koeller closed this 26 days ago
Titus-von-Koeller Fix next-token labels and accumulation target weighting
3fbd5d29
Titus-von-Koeller Merge upstream FSDP2 precision correction
56add855
Titus-von-Koeller Keep accumulation example changes focused on correctness
d87ca412
Titus-von-Koeller Limit autoregressive correction to the example and testing loader
240f0ba3
Titus-von-Koeller Reconcile the original example fix with the focused correction
f1e5e0c1
Titus-von-Koeller Titus-von-Koeller changed the title Fix next-token labels and loss normalization in the accumulation example Fix next-token labels and accumulation target weighting 17 days ago
Titus-von-Koeller Titus-von-Koeller reopened this 17 days ago
Titus-von-Koeller Titus-von-Koeller requested a review from SunMarc SunMarc 17 days ago
SunMarc
SunMarc approved these changes on 2026-10-19
Titus-von-Koeller Titus-von-Koeller merged ffc870d6 into main 12 days ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
Milestone