PR #30862 [WIP] Dynamic length in static cache

[WIP] Dynamic length in static cache #30862

ydshieh wants to merge 18 commits into main from dynamic_length_in_static_cache

ydshieh requested a review from

ArthurZucker 1 year ago

ydshieh requested a review from

gante 1 year ago

ydshieh requested a review from

zucchini-nlp 1 year ago

ydshieh commented on 2024-05-16

gante commented on 2024-05-17

ArthurZucker commented on 2024-05-23

ydshieh force pushed from 862cde4c to b4479014 1 year ago

ydshieh commented on 2024-06-14

ydshieh force pushed from c0300c3a 1 year ago

ydshieh added run-slow

Use cache_info

9ab68d0d

ydshieh force pushed to 9ab68d0d 1 year ago

[run-slow] gemma

81e795a6

[run-slow] gemma

c1098f97

[run-slow] gemma

b7eaf50c

reconstruct cache_position from _length: 0001

888a2c00

reconstruct cache_position from _length: 0002

01cd35fb

reconstruct cache_position from _length: 0003

f93b239e

[run-slow] gemma

8029b7fc

fix

b6f30f53

[run-slow] gemma

33ef0b14

remove cache info

3ca52cf3

fix

dcd292ba

[run-slow] gemma

843876bd

fix

f47f4a89

[run-slow] gemma

8d4e17b3

fix

84e694d4

[run-slow] gemma

4df64e62

ydshieh commented on 2024-07-04

fix

450b1d26

ArthurZucker commented on 2024-08-19

ArthurZucker added run-benchmark

Reviewers

ArthurZucker

gante

zucchini-nlp

Assignees

No one assigned

Labels

run-slow run-benchmark

Milestone

No milestone

transformers [WIP] Dynamic length in static cache #30862 Open

[WIP] Dynamic length in static cache #30862

transformers
[WIP] Dynamic length in static cache
#30862

Open