Megatron-DeepSpeed
Add generation server scripts using HF accelerate and DS-inference
#328
Merged
Merged
Add generation server scripts using HF accelerate and DS-inference #328
mayank31398
merged
33 commits into
main
from
add-generation-server
stas00
commented
on 2022-08-11
first step towards making libs
47f3fc0c
HF accelerate model
435af43f
refactor accelerate
d1676fcc
refactor DS inference
eef490c9
refactor DS ZeRO
25d0c704
make inference library
7be14108
cli
5c31d9a3
server
29059555
request
12c4cf7b
remove MaxTokensError
46ade324
fix batch size error with DS inference server
c46d9576
type fix
f3dac05a
add latency
44256140
add latency
c97d6ea6
add min_length to default kwargs
f3385f2b
str kwargs
8f25200b
str kwargs
b11bb7fa
stas00
commented
on 2022-08-17
fix comma
99dedb03
add old scripts back
497f00ef
move scripts
aa8c08c1
drop data
25b5d851
minor changes + add README
92017708
update README
649d7f83
Merge branch 'main' into add-generation-server
aa9ebea3
drop nccl
997a5fa2
stas00
commented
on 2022-08-23
stas00
commented
on 2022-08-24
stas00
commented
on 2022-08-24
fix
85d9fcb8
default values
11d50f18
resolve issues
403424bd
handle keyboard interrupt
493b2ee0
remove caching
c84d9b77
use snapshot_download
81d14692
make server class
a85b4886
fix snapshot download
43a844cb
stas00
approved these changes
on 2022-09-01
Login to write a write a comment.
Login via GitHub