PR #3630 [Deepspeed] deepspeed auto grad accum

[Deepspeed] deepspeed auto grad accum #3630

SunMarc merged 12 commits into huggingface:main from kashif:deepspeed-grad-unified

deepspeed auto grad accum

5686122b

add tests for grad accum

635d0687

kashif requested a review from

copilot-pull-request-reviewer 1 year ago

copilot-pull-request-reviewer commented on 2025-06-16

kashif requested a review from

copilot-pull-request-reviewer 1 year ago

copilot-pull-request-reviewer commented on 2025-06-16

use tiny-random-gpt2

85027b60

Update tests/deepspeed/test_deepspeed_gradient_accumulation.py

51311a15

fix redundant code

ca2a629f

set_gradient_accumulation_boundary is always there

8ae4b7e0

SunMarc commented on 2025-06-16

remove unused helper

2e6860f2

no need for this

bbdb0509

full revert

403525ae

SunMarc approved these changes on 2025-06-16

Apply style fixes

3495ac6a

get_global_grad_norm is always there

e3941018

Merge branch 'deepspeed-grad-unified' of github.com:kashif/accelerate…

286f0a45

SunMarc merged 23c1d8db into main 1 year ago

kashif deleted the deepspeed-grad-unified branch 1 year ago

Reviewers

SunMarc

Assignees

No one assigned

Labels

None yet

Milestone

No milestone