transformers
feat(cache): StaticCache uses index_copy_ to avoid useless copy
#31857

Merged

feat(cache): StaticCache uses index_copy_ to avoid useless copy #31857

ArthurZucker merged 7 commits into main from alvaro/static_cache_xla

feat(cache): StaticCache uses index_copy_ to avoid useless copy

d329ad28

tengomucho requested a review from

gante 1 year ago

tengomucho requested a review from

ArthurZucker 1 year ago

ArthurZucker approved these changes on 2024-07-09

feat(cache): SlidingWindowCache uses index_copy_ to avoid useless copy

53e99d18

ArthurZucker added run-slow

fix(cache): fallback of index_copy_ when not implemented

8e622ec0

tengomucho force pushed 1 year ago

fix(cache): in index_copy_ ensure tensors are on same device

aba28b57

[run slow] llama

02608dd1

tengomucho force pushed to 02608dd1 1 year ago

gante approved these changes on 2024-07-12

fix(cache): add move of cache_position to same device in SlidingWindo…

4c818f31

tengomucho requested a review from

gante 1 year ago

tengomucho requested a review from

ArthurZucker 1 year ago

ArthurZucker approved these changes on 2024-07-23

Revert "[run slow] llama"

a76cfb7f

ArthurZucker merged 63700628 into main 1 year ago

ArthurZucker deleted the alvaro/static_cache_xla branch 1 year ago

Reviewers

ArthurZucker

gante

Assignees

No one assigned

Labels

run-slow

Milestone

No milestone