Fix `StaticCache` for Mllama and enable `torch.compile` #48141
Cyrilvallez
merged
16 commits into
huggingface:main
from
jiqing-feng:fix-mllama-static-cache
Fix StaticCache allocation for Mllama cross-attention layers
e8b63ee9
Scope the auto-compile override to `MllamaForConditionalGeneration`
1a7a47ac
Enable torch.compile for Mllama with a static cache
8c0aa851
Shorten comments
e5d00882
Shorten comment
6e366329
Drop PR reference from comment
e5a92521
Don't touch blt, and skip the compile test without an accelerator
9b31decc
Move the redundant `_can_compile_fullgraph` down to blt
3b6fa6fd
Revert "Move the redundant `_can_compile_fullgraph` down to blt"
386a4dd5
Reapply "Move the redundant `_can_compile_fullgraph` down to blt"
dd13fdec
Merge branch 'main' into fix-mllama-static-cache
fcfd347a
Shorten the `_can_compile_fullgraph` comment
7df9c0aa
Merge branch 'main' into fix-mllama-static-cache
a41da51c
Merge branch 'main' into fix-mllama-static-cache
b1163864
Address review: reuse next_sequence_length, handle early-initialized …
1839e62d
Cyrilvallez
approved these changes
on 2026-09-16
Read the cross-attention cache length from the config instead of the …
1d92eba0
Cyrilvallez
approved these changes
on 2026-09-17
Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub