transformers
>3-5x faster torch.compile forward compilation for autoregressive decoder models
#32227
Merged
Merged
>3-5x faster torch.compile forward compilation for autoregressive decoder models #32227
fxmarty
merged
19 commits into
main
from
prepare-4d-causal-mask-in-generate
fxmarty
commented
on 2024-07-25
draft
43598760
apply changes to all relevant archs
7f4a078c
rerun ci - check_docstrings.py failing?
947e04af
fix docstring
8d3d27c8
ArthurZucker
commented
on 2024-07-26
Merge branch 'main' into prepare-4d-causal-mask-in-generate
1ecb89ca
move 2D->4D mask creation to modeling file
7c90be78
repo consistency
292aaa9b
fix the batch size = 1 case - calling contiguous is not enough
27335118
nit
c465b775
style
f2c1987d
gante
approved these changes
on 2024-07-30
propagate to gemma/gemma-2
5f3d84a2
prepare inputs for gemma generation
ed884328
implement test and tiny fix in gemma2
f8471e81
Update src/transformers/models/bloom/modeling_bloom.py
c6fe110a
fix copies
c1b83886
ci pass
23b5c7bd
fix gemma's test_compile_static_cache tests
b03b1bdc
flacky
4f689f05
retrigger ci
952d4369
Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub