DeepSpeed
Extend scratch buffer for long prompts
#2212
Merged
Overview
Commits
27
Changes
View On GitHub

Extend scratch buffer for long prompts #2212

jeffra merged 27 commits into master from cholmes/fix-long-seq-len-inference
cmikeh2
cmikeh2 Extend scratch buffer for long prompts
48449661
cmikeh2 cmikeh2 requested a review from jeffra jeffra 3 years ago
cmikeh2 cmikeh2 requested a review from samyam samyam 3 years ago
cmikeh2 cmikeh2 requested a review from tjruwase tjruwase 3 years ago
cmikeh2 cmikeh2 requested a review from ShadenSmith ShadenSmith 3 years ago
cmikeh2 cmikeh2 requested a review from conglongli conglongli 3 years ago
cmikeh2 cmikeh2 requested a review from awan-10 awan-10 3 years ago
cmikeh2 cmikeh2 requested a review from cli99 cli99 3 years ago
cmikeh2 cmikeh2 requested a review from eltonzheng eltonzheng 3 years ago
cmikeh2 cmikeh2 requested a review from minjiaz minjiaz 3 years ago
cmikeh2 cmikeh2 requested a review from RezaYazdaniAminabadi RezaYazdaniAminabadi 3 years ago
cmikeh2 cmikeh2 requested a review from duli2012 duli2012 3 years ago
cmikeh2 cmikeh2 requested a review from mrwyattii mrwyattii 3 years ago
cmikeh2 cmikeh2 requested a review from yaozhewei yaozhewei 3 years ago
cmikeh2 cmikeh2 requested a review from arashb arashb 3 years ago
cmikeh2 cmikeh2 requested a review from xiaoxiawu-microsoft xiaoxiawu-microsoft 3 years ago
cmikeh2 cmikeh2 requested a review from samadejacobs samadejacobs 3 years ago
cmikeh2 Merge branch 'master' into cholmes/fix-long-seq-len-inference
be4714d7
RezaYazdaniAminabadi
RezaYazdaniAminabadi commented on 2022-08-12
RezaYazdaniAminabadi
RezaYazdaniAminabadi commented on 2022-08-12
cmikeh2 Fetch correct tail buffer for batched inputs.
c6411d13
cmikeh2 Style change
c074ed37
cmikeh2 Merge branch 'cholmes/fix-long-seq-len-inference' of https://github.c…
31357744
RezaYazdaniAminabadi Merge branch 'master' into cholmes/fix-long-seq-len-inference
777a36ec
cmikeh2 Fix variable rename
ec6b1ad3
cmikeh2 Merge branch 'cholmes/fix-long-seq-len-inference' of https://github.c…
d897f98f
cmikeh2 Merge branch 'master' into cholmes/fix-long-seq-len-inference
6da92341
cmikeh2 Reduce maximum sequence length
606d3447
cmikeh2 Merge branch 'master' into cholmes/fix-long-seq-len-inference
5269ba10
RezaYazdaniAminabadi Merge branch 'master' into cholmes/fix-long-seq-len-inference
89f2dedf
cmikeh2 Merge branch 'master' into cholmes/fix-long-seq-len-inference
8e298087
cmikeh2 Add debug print
c8243304
cmikeh2 Merge branch 'master' into cholmes/fix-long-seq-len-inference
d9eb076b
cmikeh2 Multi-batch inference fix
aafba00c
add batch-size at the tranform launch for the half-precision implemen…
4abd4555
RezaYazdaniAminabadi Merge branch 'master' into cholmes/fix-long-seq-len-inference
603cc5bf
RezaYazdaniAminabadi Merge branch 'master' into cholmes/fix-long-seq-len-inference
508712a7
RezaYazdaniAminabadi RezaYazdaniAminabadi requested a review from GuanhuaWang GuanhuaWang 3 years ago
no need to throw error when there is no mask passed
51a63715
Merge branch 'cholmes/fix-long-seq-len-inference' of github.com:micro…
9effa9e4
RezaYazdaniAminabadi Merge branch 'master' into cholmes/fix-long-seq-len-inference
c9652ecf
RezaYazdaniAminabadi Increasing the token-length based on available memory for GPT models …
d8f52032
RezaYazdaniAminabadi Merge branch 'master' into cholmes/fix-long-seq-len-inference
b64c1170
fix bert issue & remove some dead code
48a8b969
fix formating
c1d83f99
jeffra Merge branch 'master' into cholmes/fix-long-seq-len-inference
a5f4d31b
jeffra
jeffra approved these changes on 2022-09-23
jeffra jeffra merged 3d097bb8 into master 3 years ago
jeffra jeffra deleted the cholmes/fix-long-seq-len-inference branch 3 years ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum