add new model like
2fddcc9d
Merge pull request #9 from huggingface/update
e536f6a3
add state dict slicing + new model config
6ca0bf7e
update palma config and weights, passes vision activations
21db4a79
fix
7985fc43
update
bcb341da
reorder loading/unpacking
38ad70e4
clean up
929746a3
add debug statements
ae55ad93
change device
9d5f8fbf
Merge branch 'add_palma' of github.com:huggingface/new-model-addition…
524a0739
fix
d171c4e4
debugging
1638a73d
fix noncausal mask
4aad8506
fixup sdpa + causal mask
29726744
fix activation function
71fa912c
remove debug before changing modeling file
94e48067
add variants
7b6f0b38
debug attention mask in generate
4e8e1c66
revert to non-debug sdpa
ba8fb4e8
revert gemma modifications
6c2348d7
add custom language modeling
906e87f3
use Processor
f26361d7
add language modeling file to init
b3e4a03a
try thin wrapper around generate
500a3605
Update
96a82e27
update mask
347df2c1
breakpoints galore
d01b502c
remove conflict
bb8030cb
switch to left-padding
a6056c65
add incomplete model doc
dacd2974
add paligemma global files
f133a1d1
batch rename paligemma
37c13686
make generation match outputs and captioning
9935e2ee
style
e7fd0496
style
edcf1b15
remove copied from + doc
2a29a2e7
remove more copied from
5607c9da
remove copy from projector
8ce77f35
minor fix
3563926e
update config and style
f48f61d0
add readme - dummy
b84d1c15
CORRECT image captioning
deb35ba2
moving to args
9275d53a
add siglip proper + fix merging image + text features
54877348
take update_causal_mask from upstream
7f9c479c
remove breakpoint
438b1438
leverage AutoModel
00d29228
fix input_ids slicing
72697b62
make siglip head conditional
17b30fdf
remove encoder_decoder value
26cb46e7
remove unneeded modeling file
f49f3896
add commented 4d attention mask
d1964ea8
FIXED generation with 4D mask
6da4d2e1
Merge branch 'main' of github.com:huggingface/new-model-addition
ef8c0fb4
Update src/transformers/models/siglip/modeling_siglip.py
9f261d77
fix left padding detection
11c34882
shuffle order of verifications
6d527e4a
Merge branch 'main' into add_palma
8b679b14
fix missing labels for training
fdba83a1
fix
eee3c1e3
vectorize merging of features, improve slicing
e4869dd8
improve testing before conversion
b9733653
handle merging in processor
8cab59cf
image token index depends on checkpoint
9186b79e
add variants, save processor too
78398a13
save processors, base tokenizer off spm file
4ede86e6
Merge branch 'main' of github.com:huggingface/transformers into main
c8991363
expand model embeddings due to additional image token
a9122738
pass image processing args
2c8a508c
add convert rgb to siglip processor
7201d437
add \n token separately
c74f3a35
Merge branch 'main' of github.com:huggingface/transformers
23b12a38
Merge branch 'main' of github.com:huggingface/new-model-addition
4e4a9576
fix tokenizer and prompts
1f465c08
fix docstrings
38a04017
change to camel
16baabc0
fix casing
9119db41
debug pos_ids and sdpa
267a0da0
Merge branch 'main' into add_palma
3e321185
pass and use cache_position
4ed05944
add flag for newline tokenization
46c5c0bc
Update src/transformers/models/paligemma/processing_paligemma.py
5b47a5b6
simplify conversion script
b80c5917
add copied from
7d214095
add precision to conversion script
b1eda0e3
Update src/transformers/models/paligemma/modeling_paligemma.py
404abd88
clean up
a7e24463
Merge branch 'add_palma' of github.com:huggingface/new-model-addition…
2f6b17d2
Shift attention mask from `1:`
60ad9c5d
Merge pull request #6 from pcuenca/add_palma_shift_mask
daccbe7d
add docs, fix quality
91c7aabc
Merge branch 'add_palma' of github.com:huggingface/new-model-addition…
372d5663
quality, tied weights inheritance, and logits/label alignment
63464ec3
fix more tests
55187a15
pass attn_implementation to language model correctly
5ea6c9f3
add SiglipVisionTransformer to no split modules
ddb7ac77
skip paligemma test for sdpa dispatch to flash
7f327181
skip incompatible tests
ac9fd9a9
quality
cceb3d0a
[broken archive maps]
a2648244
Apply suggestions
9310873d
Update src/transformers/utils/dummy_pt_objects.py
0711b127
simplify conversion script
e7ec2164
add suggestions
8b0724de
add suggestions
7bcea3e0
add copied from
498bbde3
fix
a8bd223e
move labels out
04d962fd
revert
e7caa8a2
fix
ac5ed67d
remove placeholder labels if None
72f6fdc0
use cache_position
c8247714
fix quality + docstrings
7a8e62ef
fix quality
4913c07e
Merge branch 'main' of github.com:huggingface/transformers into main
0c8f2c96
ArthurZucker
approved these changes
on 2024-05-14
Merge branch 'main' into add_palma
54fd284c
fix paligemma 4d gemma mask incompatibility
99c3ac51
fix config docstring
75c36c26
fix query and attn_mask dtype
9b49838e
Login to write a write a comment.
Login via GitHub