accelerate
Make the Scheduler adjust the steps taken relative to the gradient accumulation steps
#1187

Merged

Make the Scheduler adjust the steps taken relative to the gradient accumulation steps #1187

muellerzr merged 22 commits into main from scheduler-docs

Make scheduler actually adjust the length

94181fda

df904b19

Use last_epoch

099bfe06

Try now

8006f8c9

Make scheduler step based on gradient accumulation

01f9a0a9

muellerzr added enhancement

muellerzr requested a review from

sgugger 2 years ago

Put decorator in the right place

8744f408

clean

79ff7c56

sgugger commented on 2023-03-13

Rework it all, better version and working now with tests

60f3009f

Fix tests

250fe900

Bring back grad_accum_steps

71619d5a

muellerzr marked this pull request as draft 2 years ago

Use plugin

f19a4b0e

Fix docstrings

744686e6

Formatting nit

b39fa980

muellerzr marked this pull request as ready for review 2 years ago

sgugger approved these changes on 2023-03-14

Proper kwarg

3acda862

muellerzr requested a review from

sgugger 2 years ago

Raise err

97d1af23

Move around check for TPU

0d922d4a

sgugger approved these changes on 2023-03-14

fix para name

4a473335

Fix steps w/ deepspeed

04495041

Good import

bb5a374b

Try with this

f3bce841

Try now?

79268088

Should be working now, just need to test on CI

248070f4

muellerzr merged e4620984 into main 2 years ago

muellerzr deleted the scheduler-docs branch 2 years ago

Reviewers

sgugger

Assignees

No one assigned

Labels

enhancement

Milestone

No milestone

accelerate Make the Scheduler adjust the steps taken relative to the gradient accumulation steps #1187 Merged

Make the Scheduler adjust the steps taken relative to the gradient accumulation steps #1187

accelerate
Make the Scheduler adjust the steps taken relative to the gradient accumulation steps
#1187

Merged