transformers
[cache refactor] Move all the caching logic to a per-layer approach
#39106

Merged

[cache refactor] Move all the caching logic to a per-layer approach #39106

Cyrilvallez merged 44 commits into huggingface:main from manueldeprada:cache-refactor-1

Squash for refactor: Replace monolithic cache classes with modular La…

1c3cbccc

manueldeprada requested a review from

gante 248 days ago

fix quantized, add tests

04d7a0b1

manueldeprada force pushed from a2fe24c8 to 04d7a0b1 246 days ago

remove CacheProcessorList

26c28af6

manueldeprada force pushed from d97a02da to 26c28af6 246 days ago

manueldeprada requested a review from

ArthurZucker 246 days ago

zucchini-nlp commented on 2025-07-01

ArthurZucker commented on 2025-07-01

raushan review, arthur review

16a66240

manueldeprada requested a review from

ArthurZucker 244 days ago

gante commented on 2025-07-02

joao review: minor things

aec9ccd6

manueldeprada force pushed from c1d93472 to 52ac7934 243 days ago

manueldeprada force pushed from 52ac7934 to d6843398 243 days ago

manueldeprada force pushed from d6843398 to 4c03e0f3 242 days ago

remove cache configs, make CacheLayer a mixin (joaos review)

e80c68a6

manueldeprada force pushed from 88089304 to e80c68a6 242 days ago

manueldeprada force pushed from 65fbce68 to a21b2f74 237 days ago

manueldeprada force pushed from a21b2f74 to c538ad75 237 days ago

manueldeprada force pushed from c538ad75 to 5dc5fb40 237 days ago

manueldeprada force pushed from 5dc5fb40 to a6b7562b 237 days ago

back to storage inside Cache()

27916bc2

manueldeprada force pushed from a6b7562b to 27916bc2 237 days ago

remove cachebase for decorator

fd83e14b

manueldeprada force pushed from 9e37266d to fd83e14b 237 days ago

manueldeprada force pushed from 6c0dd19f to 129119dd 237 days ago

no more __getattr__

c2004471

manueldeprada force pushed from 129119dd to c2004471 237 days ago

Merge branch 'main' of github.com:huggingface/transformers into cache…

f3275705

fix tests

5b1b1f17

manueldeprada force pushed from 295c8d8a to 5b1b1f17 237 days ago

gante commented on 2025-07-10

manueldeprada force pushed from ba1d14ed to 5b1b1f17 236 days ago

joaos review except docs

58dbcfe2

manueldeprada force pushed from 2b4e7bf7 to 58dbcfe2 236 days ago

fix ast deprecations for python 3.14: replace node.n by node.value an…

0c6d2ff6

Cyrilvallez commented on 2025-07-11

manueldeprada force pushed from b22b1179 to 87f399fd 233 days ago

Revert "back to storage inside Cache()"

6a77408a

manueldeprada force pushed from 87f399fd to 6a77408a 233 days ago

cyril review

13ec4a44

simplify cache export

7029a90c

Merge branch 'main' of github.com:huggingface/transformers into cache…

5fa99012

fix lfm2 cache

dd7458b5

HybridChunked to layer

dc08253c

manueldeprada force pushed from d434a998 to dc08253c 232 days ago

BC proxy object for cache.key_cache[i]=...

a9521249

reorder classes

dbbc4d51

bfff come on LFM2

4bb48fcf

manueldeprada requested a review from

Cyrilvallez 232 days ago

better tests for hybrid and hybridChunked

00b1f96a

Merge branch 'main' of github.com:huggingface/transformers into cache…

def346eb

complete coverage for hybrid chunked caches (prefill chunking)

38e86034

reimplementing HybridChunked

34a3022f

Cyrilvallez commented on 2025-07-15

ArthurZucker removed review request from

ArthurZucker 230 days ago

cyril review

42226536

Merge branch 'main' of github.com:huggingface/transformers into cache…

063459c5

manueldeprada force pushed from 51d1505b to 83a6016b 230 days ago

fix ci

1acc648f

manueldeprada force pushed from 83a6016b to 1acc648f 230 days ago

docs for cache refactor

ca39ffec

manueldeprada force pushed from bd4c9a41 to ca39ffec 229 days ago

docs

731d0b73

Merge branch 'main' of github.com:huggingface/transformers into cache…

574b8205

oopsie

a4794705

oopsie

9c0bdcc5

Merge branch 'main' into cache-refactor-1

04091ed8

Merge branch 'main' of github.com:huggingface/transformers into cache…

8642bb6b

Merge branch 'cache-refactor-1' of https://github.com/manueldeprada/t…

83968bd9

fix after merge

0c4700dc

Cyrilvallez commented on 2025-07-21

Cyrilvallez changed the title ~~[cache refactor] PR 1/3: layered caches with full backward compatibility~~ [cache refactor] Move all the caching logic to a per-layer approach 225 days ago

cyril review

b3a35e92

ArthurZucker commented on 2025-07-22

arthur review

e4878ad0

Merge branch 'main' of github.com:huggingface/transformers into cache…

38fb99d4

opsie

8df15953

fix lfm2

ad65a02d

Cyrilvallez commented on 2025-07-22

opsie2

d9fbb044

Cyrilvallez approved these changes on 2025-07-22

Cyrilvallez merged c338fd43 into main 224 days ago

Reviewers

Cyrilvallez

ArthurZucker

zucchini-nlp

gante

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

transformers [cache refactor] Move all the caching logic to a per-layer approach #39106 Merged

[cache refactor] Move all the caching logic to a per-layer approach #39106

transformers
[cache refactor] Move all the caching logic to a per-layer approach
#39106

Merged