Add ViViT #22518
amyeroberts
merged
79 commits into
huggingface:main
from
add_vivit
amyeroberts
commented
on 2023-06-20
Add model
45be695c
Add ability to get classification head weights
226ea89e
Add docs
89919b17
Add imports to __init__.py
3edc517d
Run style
c733de3d
Fix imports and add mdx doc
7a0c6656
Run style
7cf691fa
Fix copyright
622d92d7
Fix config docstring
6ef85ff5
Remove imports of ViViTLayer and load_tf_weights_in_vivit
1c2c0db4
Remove FeatureExtractor and replace with ImageProcessor everywhere
e0b1836c
Remove ViViTForPreTraining from vivit.mdx
ef8cc508
Change ViViT -> Vivit everywhere
43a9cd71
Add model_doc to _toctree.yml
11f439c5
Replace tuples with lists in arguments of VivitConfig
e78a6596
Rename patch_size to tubelet_size in TubeletEmbeddings
c2ba1acb
Fix checkpoint names
81b257cd
Add tests
d1bf40b1
Remove unused num_frames
ffe6eba2
Fix imports for VivitImageProcessor
e3de4654
Minor fixes
c99621b9
Decrease number of frames in VivitModelTester from 32 to 16
8a725c87
Decrease number of frames in VivitModelTester from 16 to 8
533dbe7b
Add initialization for pos embeddings
0df3c5d9
Rename Vivit -> ViViT in some places
d7034d04
Fix docstring and formatting
77bbe400
Rename TubeletEmbeddings -> VivitTubeletEmbeddings
500eb7c6
Remove load_tf_weights_in_vivit
564f3da5
Change checkpoint name
bc26ffac
Remove Vivit _TOKENIZER_FOR_DOC
c573a113
Fix
d5d6eaca
Fix VivitTubeletEmbeddings and pass config object as parameter
221c8ee2
Use image_size and num_frames instead of video_size
a647f40a
Change conversion script and fix differences with the orig implementa…
9f0acc5c
Fix docstrings
30c32135
Add attention head pruning
ddf53f78
Run style and fixup
28c8513c
Fix tests
40dc58f8
Add ViViT to video_classification.mdx
629ab717
Save processor in conversion script
d0661ed3
Fix
d75f170c
Add image processor test
e36ea3a0
Run fixup and style
c465a7cf
Run fix-copies
015c87b2
Update tests/models/vivit/test_modeling_vivit.py
f1057962
Update tests/models/vivit/test_modeling_vivit.py
2242721a
Update src/transformers/models/vivit/modeling_vivit.py
de5927d1
Use PyAV instead of decord
d494ce2a
Add unittest.skip
75a22bcc
Run style
38380f26
Remove unneeded test
9674097a
Update docs/source/en/model_doc/vivit.mdx
69b226a9
Update src/transformers/models/vivit/configuration_vivit.py
e2888f49
Update src/transformers/models/vivit/modeling_vivit.py
06f5551d
Update src/transformers/models/vivit/image_processing_vivit.py
dae70c2b
Update src/transformers/models/vivit/modeling_vivit.py
182ea59f
Update src/transformers/models/vivit/modeling_vivit.py
7e962088
Update src/transformers/models/vivit/image_processing_vivit.py
6cec9283
Update src/transformers/models/vivit/modeling_vivit.py
d486e7aa
Add model
03330d86
Add docs
32d17a70
Run style
0115ecb9
Fix imports and add mdx doc
34be1cef
Remove FeatureExtractor and replace with ImageProcessor everywhere
656e3bae
Change ViViT -> Vivit everywhere
cb6b86d1
Rename Vivit -> ViViT in some places
88b6544d
Update src/transformers/models/vivit/image_processing_vivit.py
c42624ce
Run make style
8e6d8508
Remove inputs save
3f61cad7
Fix image processor
ff977446
Fix
a4812a5d
Run `make style`
35822e37
Decrease parameters of VivitModelTester
3492cb4d
Decrease tubelet size
a05575db
Rename vivit.mdx
a354dbd8
Update src/transformers/models/vivit/image_processing_vivit.py
ee193b0b
Update src/transformers/models/vivit/image_processing_vivit.py
c942d6a8
Update src/transformers/models/vivit/image_processing_vivit.py
5f14cb80
Fix default values in image_processing_vivit.py
b0fa0701
Login to write a write a comment.
Login via GitHub