PR #34645 FSDP grad accum fix

FSDP grad accum fix #34645

ArthurZucker merged 2 commits into huggingface:main from winglian:fsdp-grad-accum-fix

add gradient accumulation steps tests for fsdp

95b718b1

invert no_sync context to fix training for fsdp

edd102fc

muellerzr approved these changes on 2024-11-07

muellerzr requested a review from

ArthurZucker 1 year ago

SunMarc approved these changes on 2024-11-15

SunMarc requested a review from

ydshieh 1 year ago

ydshieh approved these changes on 2024-11-15

ArthurZucker approved these changes on 2024-11-15

ArthurZucker merged b0c0ba7b into main 1 year ago

Reviewers

ArthurZucker

ydshieh

SunMarc

muellerzr

Assignees

No one assigned

Labels

None yet

Milestone

No milestone