DeepSpeed
Increasing the token-length based on available memory for GPT models
#2280
Merged
Merged
Increasing the token-length based on available memory for GPT models #2280
RezaYazdaniAminabadi
merged
14 commits into
cholmes/fix-long-seq-len-inference
from
ds-inference/support-large-token-length
increasing the token-length based on available memory & reduce memory…
6d7c133d
Merge branch 'master' of github.com:microsoft/DeepSpeed into ds-infer…
216b9538
RezaYazdaniAminabadi
changed the base branch from
master
to
cholmes/fix-long-seq-len-inference
3 years ago
merging
b15f2410
Merge branch 'cholmes/fix-long-seq-len-inference' of github.com:micro…
a08a3bfd
formating
12a58142
fix compile issue
89baf13f
Merge branch 'cholmes/fix-long-seq-len-inference' into ds-inference/s…
50d9963e
Merge branch 'cholmes/fix-long-seq-len-inference' into ds-inference/s…
53ed3cc8
fix the max_out_tokens to use a dynamic range based on available memory
a149a5a4
fix the issue with empty prompt
c9b604df
Merge branch 'cholmes/fix-long-seq-len-inference' into ds-inference/s…
b018ec9c
fix residual-add
fb89f198
fix some issues with unit tests
56f80290
fix formatting
ac2698fd
Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub