Inverse Square Root LR Schedule #657
codestar12
merged
28 commits into
mosaicml:main
from
inv-sqrt-lr-sched
Implement inverse square root with warmup scheduler v0
f2142e8c
Merge branch 'mosaicml:main' into inv-sqrt-lr-sched
7858cbd7
Merge branch 'mosaicml:main' into inv-sqrt-lr-sched
7ebb1c02
Merge branch 'mosaicml:main' into inv-sqrt-lr-sched
df6abcb9
quick
92407a15
Merge branch 'mosaicml:main' into inv-sqrt-lr-sched
a5904bea
Merge branch 'mosaicml:main' into inv-sqrt-lr-sched
90338b4e
inverse square root LR Schedule
54007d34
dakinggg
commented
on 2023-10-09
scheduler
1f9ac7ae
b-chu
commented
on 2023-10-09
unit tests
04969539
Update llmfoundry/optim/scheduler.py
e99997ee
Update llmfoundry/optim/scheduler.py
0752dd0c
fixes for PR conversations
de7873e9
merge some commits
166230f8
format
b8e37fce
Merge branch 'main' into inv-sqrt-lr-sched
81d09728
fix type hint
b8f717d2
Merge branch 'inv-sqrt-lr-sched' of github.com:mansheej/llm-foundry i…
26accc46
Merge branch 'main' into inv-sqrt-lr-sched
71bb52a0
Update llmfoundry/optim/scheduler.py
540f9e5f
Update llmfoundry/optim/scheduler.py
70cbef2a
Update llmfoundry/optim/scheduler.py
7a17279b
more fixes
d7d49f93
dakinggg
commented
on 2023-10-11
more fixes
574276c2
dakinggg
approved these changes
on 2023-10-11
more test
c8901617
fixes
608de1ae
fix pyright errors
91b7db3d
fix more errors
a11bcab0
Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub