DeepSpeed
Add Inference support for running the BigScience-BLOOM Architecture
#2083
Merged
Merged
Add Inference support for running the BigScience-BLOOM Architecture #2083
jeffra
merged
27 commits into
master
from
ds-inference/bloom-support
Add Inference support for running the BigScience-BLOOM Architecture
7cf7c89f
tjruwase
commented
on 2022-07-10
tjruwase
commented
on 2022-07-10
Merge branch 'master' into ds-inference/bloom-support
6d33b820
formatting
6cc23400
add the checkpoint loading at the same of kernel-injection
2a69756b
Merge branch 'ds-inference/bloom-support' of github.com:microsoft/Dee…
e449ac7b
releasing checkpoint CPU-memory after loading it
6e304aa8
some fixes plus formatting
16dd1dc0
fix layer_past; this caused issues when running inference on several …
e0dd488f
Add support for multi-batch inference
02f99456
fix the padding issue for large bach inference
1cef202d
fixing some bug in softmax kernel for batch_size>1
644fea44
align alibi&mask addition with HF new changes
adb0b972
revert back some changes and support for very large batch size
bd3c0a02
reduce the max_token_length for now
1f92e557
fix mask-adding
5d1f351b
fix the large-batch inference for MP > 1
3c12b899
Merge branch 'master' into ds-inference/bloom-support
a5bdd58f
Ds inference/bloom support meta (#2104)
aa5e01ff
fix the Bert and GPT-J unit tests
b6503ed1
fix for OneDevice
a9459d6d
Merge branch 'master' into ds-inference/bloom-support
2cd301e5
added bloom inference tests
72aba56c
fixing the masking stride for the GPT models
332f69db
Merge branch 'ds-inference/bloom-support' of github.com:microsoft/Dee…
0ddf41cc
revert back some changes on replace_module
d38464e0
fix fp32 softmax
ac2d092e
allocate tensors initially on cpu at inference-api
1e3ea74c
jeffra
approved these changes
on 2022-07-18
Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub