transformers
Idefics: Fix information leak with cross attention gate in modeling
#26839
Merged
Merged
Idefics: Fix information leak with cross attention gate in modeling #26839
ArthurZucker
merged
26 commits into
huggingface:main
from
leot13:fix-idefics-image-attention
fix image_attention gate in idefics modeling
aaf79cc0
update comment
3d5c7f4c
cleaner gating
b13ce277
fix gate condition
9c5cad2a
create attention gate once
69b8ab94
update comment
6d5179b1
update doc of cross-attention forward
92e1f504
improve comment
17403d92
Merge branch 'main' into fix-idefics-image-attention
f6ac847b
bring back no_images
598deb12
pass cross_attention_gate similarly to no_images gate
699838cc
add information on gate shape
7a45089b
fix no_images placement
ac493193
make tests for gate
6cc352cf
leot13
commented
on 2023-10-26
take off no_images logic
71d444da
Merge branch 'main' into fix-idefics-image-attention
f5b26b94
update test based on comments
9a000da9
amyeroberts
approved these changes
on 2023-11-02
raise value error if cross_attention_gate is None
a7ff77a5
send cross_attention_gate to device
054f8422
Revert "send cross_attention_gate to device"
b45351d3
send cross_attention_gate to device
58822578
fix device in test + nit
0ed25e3c
fill hidden_states with zeros instead of multiplying with the gate
0af2a820
style
69e7fe2b
Update src/transformers/models/idefics/modeling_idefics.py
1840ccc7
Update src/transformers/models/idefics/modeling_idefics.py
54417d89
Login to write a write a comment.
Login via GitHub