transformers
Support per-layer cache configuration
#48178
Merged
Overview
Commits
32
Changes
View On GitHub

Support per-layer cache configuration #48178

Cyrilvallez merged 32 commits into huggingface:main from eladsegal:cache-heterogeneity-compat
eladsegal
eladsegal Support heterogeneous cache layouts
bbfbd9bf
eladsegal Add heterogeneous cache compatibility tests
368bcb89
eladsegal revert gpt-oss mxfp4 changes
9f8026ae
eladsegal Merge branch 'main' into cache-heterogeneity-compat
160d2e2c
eladsegal Merge branch 'main' into cache-heterogeneity-compat
3d8e17d7
eladsegal Merge branch 'huggingface:main' into cache-heterogeneity-compat
1447d7f3
eladsegal Merge branch 'main' into cache-heterogeneity-compat
f7aecfa8
eladsegal Merge branch 'main' into cache-heterogeneity-compat
221c1b3a
eladsegal Preserve wrapped forward metadata in Accelerate hooks
7937a3b6
eladsegal Revert "Preserve wrapped forward metadata in Accelerate hooks"
fcdc10de
eladsegal Merge branch 'main' into cache-heterogeneity-compat
ad8b7d20
eladsegal Merge branch 'main' into cache-heterogeneity-compat
359ab1c6
eladsegal Merge branch 'main' into cache-heterogeneity-compat
4fbcc1ab
Cyrilvallez
Cyrilvallez commented on 2026-09-02
eladsegal Merge branch 'main' into cache-heterogeneity-compat
b6257f80
eladsegal Remove underscore from _get_layer_types_and_kwargs
3e87fb1f
eladsegal Remove get_representative_kv_layer_idx and restore get_seq_length
7ec7ce5a
eladsegal remove related tests
d5b885ad
eladsegal fix
9e926577
eladsegal Merge commit '254e62f7462ca8d19821a6427828ee75765ce123' into cache-he…
567736c5
eladsegal Clean up masking utils changes
42ef6c9a
eladsegal Fix doc strings
ca05f111
eladsegal Merge branch 'main' into cache-heterogeneity-compat
77f90425
eladsegal Merge branch 'main' into cache-heterogeneity-compat
9902c07d
eladsegal Merge branch 'main' into cache-heterogeneity-compat
392b79e1
eladsegal Merge branch 'main' into cache-heterogeneity-compat
9ca1a90b
eladsegal Fix per-layer static cache initialization and empty cache cropping
ebc51249
eladsegal Merge branch 'main' into cache-heterogeneity-compat
3e3beaee
eladsegal Fix per-layer head shape inference for static caches
ef56e5e4
eladsegal clean up tests
7210434e
eladsegal Merge branch 'main' into cache-heterogeneity-compat
9c36aaaf
Cyrilvallez
Cyrilvallez commented on 2026-10-01
eladsegal Remove unrelated changes based on the review
43d5e6cf
eladsegal eladsegal changed the title Support per-layer cache configuration and attention-mask selection Support per-layer cache configuration 8 days ago
eladsegal Merge branch 'main' into cache-heterogeneity-compat
c0c0e0ec
github-actions
Cyrilvallez
Cyrilvallez approved these changes on 2026-10-02
Cyrilvallez Cyrilvallez merged 35924ec3 into main 8 days ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum