[trainer] deepspeed integration #9211
stas00
merged
69 commits into
huggingface:master
from
stas00:ds
deepspeed integration
b99b6653
style
cf2f0d2f
add test
d417f553
patrickvonplaten
commented
on 2020-12-21
patrickvonplaten
commented
on 2020-12-21
patrickvonplaten
approved these changes
on 2020-12-21
ds wants to do its own backward
112be601
fp16 assert
4c2809dc
Update src/transformers/training_args.py
f4de6ff0
Merge branch 'ds' of github.com:stas00/transformers into ds
bd350f6c
style
8565c8a6
Merge remote-tracking branch 'origin/master' into ds
9653f8eb
Merge remote-tracking branch 'origin/master' into ds
e980e21b
for clarity extract what args are being passed to deepspeed
9cc3b63d
introduce the concept of self.wrapped_model
15104443
s/self.wrapped_model/self.model_wrapped/
f28566e0
complete transition to self.wrapped_model / self.model
caa32dc7
fix
9f199e71
doc
fb0c13eb
give ds its own init
6af645f3
add custom overrides, handle bs correctly
765594df
fix test
8ebe5c74
clean up model_init logic, fix small bug
e8c10804
complete fix
3b7c5815
collapse --deepspeed_config into --deepspeed
aa8f9a13
style
a83c46a5
start adding doc notes
aaf97e10
style
3dedd7a8
Merge remote-tracking branch 'origin/master' into ds
af22ec1d
implement hf2ds optimizer and scheduler configuration remapping
869173f6
oops
2f73bfe6
call get_num_training_steps absolutely when needed
9c778238
workaround broken auto-formatter
e6610dae
deepspeed_config arg is no longer needed - fixed in deepspeed master
a1ed8387
use hf's fp16 args in config
25a93d9b
sgugger
commented
on 2021-01-05
clean
39f64467
Merge remote-tracking branch 'origin/master' into ds
b3667b08
start on the docs
df1154c8
Merge remote-tracking branch 'origin/master' into ds
c5afaecb
stas00
commented
on 2021-01-07
rebase cleanup
c9a6266f
g-karthik
commented
on 2021-01-07
finish up --fp16
8bb2462a
clarify the supported stages
c9e7de70
Merge remote-tracking branch 'origin/master' into ds
33087ca9
big refactor thanks to discovering deepspeed.init_distributed
dc00de77
cleanup
ad967b45
revert fp16 part
52d80009
add checkpoint-support
31dba171
more init ds into integrations
6c5432a2
extend docs
8364b277
cleanup
f462b5c8
unfix docs
152c2392
Merge remote-tracking branch 'origin/master' into ds
38ce263f
clean up old code
2d4de17b
imports
f90822f0
move docs
42260fe4
fix logic
dda4a44c
tjruwase
commented
on 2021-01-08
make it clear which file it's referring to
f9d60287
document nodes/gpus
57f58883
style
c8ef31e4
wrong format
c0f5e1b8
style
20e3ab6c
deepspeed handles gradient clipping
777ae49c
easier to read
0bbc65e8
major doc rewrite
c65a6808
Apply suggestions from code review
4b9dd767
docs
36c6f57d
switch to AdamW optimizer
7f5e5797
style
b210ae30
Merge remote-tracking branch 'origin/master' into ds
96b7d3ad
Apply suggestions from code review
d8da0c7b
clarify doc
19ad552a
Merge remote-tracking branch 'origin/master' into ds
19e4972c
Reviewers
Assignees
No one assigned
Labels
DeepSpeed
Milestone
No milestone
Login to write a write a comment.
Login via GitHub