DeepSpeed
Reset KV-cache at the beginning of text-generation
#2669

Merged

Reset KV-cache at the beginning of text-generation #2669

jeffra merged 8 commits into master from kv-cache-reset

Reset KV-cache at the beginning of text-generation

e3c15690

RezaYazdaniAminabadi requested a review from

jeffra 3 years ago

RezaYazdaniAminabadi requested a review from

mrwyattii 3 years ago

RezaYazdaniAminabadi requested a review from

awan-10 3 years ago

RezaYazdaniAminabadi requested a review from

cmikeh2 3 years ago

RezaYazdaniAminabadi requested a review from

arashb 3 years ago

Merge branch 'master' into kv-cache-reset

bc582b77

Merge branch 'master' into kv-cache-reset

bc00402b

Merge branch 'master' into kv-cache-reset

b90d80ca

reset cache only if using inferece_cuda module

2a2356b0

Merge branch 'kv-cache-reset' of github.com:microsoft/DeepSpeed into …

1c430eb2

RezaYazdaniAminabadi enabled auto-merge (squash) 3 years ago

Merge branch 'master' into kv-cache-reset

0c24ca7c

formatting

44d28709

jeffra approved these changes on 2023-02-03

disabled auto-merge 3 years ago
Manually disabled by user

jeffra merged 9f41ffe4 into master 3 years ago

Reviewers

jeffra

mrwyattii

awan-10

cmikeh2

arashb

Assignees

No one assigned

Labels

None yet

Milestone

No milestone