Add V-JEPA 2 #38746
qubvel
merged
90 commits into
huggingface:main
from
qubvel:koustuvs/oss
adding model and conversion scripts
6c1c472c
add imports to test vjepa conversion
103bde8f
fix imports and make conversion work
d09c1c02
fix computation for short side
3fd6bf28
replace attention with library attention function
aae79f77
cleanup more attention classes
41957a64
remove config overrides
f46df328
add test cases, fix some of the failing ones
174ba399
fix the model outputs
3e412793
fix outputs of the model per review
87116285
fix too big model test case
da7f76a8
Merge remote-tracking branch 'upstream/main' into koustuvs/oss
beba328c
Merge branch 'koustuvs/oss' of https://github.com/huggingface/new-mod…
d54b0a47
fix styling __init__.py
db6b5e35
fix initialization test
df77afef
remove all asserts per review
239b30fb
update sorting unsorting logic as per feedback
dd4850d4
remove is_video per review
d8b18b16
remove another is_video segment
eb955bfd
remove unwanted stuff
a382ebbd
small fixes
de97de09
add docstrings for the model
30f6febf
revert adding vjepa2 config here
f5a07b2d
update styling
d90da4d6
add config docstrings (wip)
238f8f39
fix dpr issue
9bc533da
removed test failing issues
0354e0c7
update styles
fdb66978
merge predictor configs into main config
e203d419
remove processing code, add video processor
3c60022a
remove permute which is not necessary now
3879772c
fix styles
ac0e97b5
updated vjepa2 to be in video_processing_auto
6ba6f467
update comment for preprocessing
0a98f5f7
test integration test and fix the outputs
43943455
update test values, change test to look at repeated frames for a give…
436fce51
add a simple video processing test
9752cec1
refactoring pixel_values_videos and upload ckpts to original
9f004fac
fix torch_fx test cases
8409e834
remove unused config
6b8c776b
add all config docstrings
5fc0f9dc
add more integration tests
a966ca28
add basic doc
824f7345
revert unwanted styling changes
0f8be62c
working make fixup
6fa271b8
Fix model_type in config
9b5d9dd0
update attention implementation to fit new hf standards
7ce0d8c9
fix the preprocessing logic, ensure it matches the original model
6522b7fe
Merge pull request #3 from huggingface/standardize-state-dict
5042ec86
remove use_rope logic, cleanup
eed77e72
fix docstrings
a04edb2c
Further cleanup, update doc
f1e86054
Fix model prefix
525eaae3
fix get_vision_features
e5faf69c
VJEPA2Embeddings style refactor
79aebe5c
nit, style comment
cffcbf39
change modules default values
1e3fa779
Only `str` activation in config
54795388
GradientCheckpointingLayer
4a320a95
fixup
2ac05f37
fix conversion script
15c1f95e
Remove return_dict
24bb730c
remove None return typehint
71df5525
Refactor VJEPA2Layer, remove use_SiLU
a0242be2
Fix fx tests
bb574e4e
dpr -> drop_path_rates
1b7e8f8f
move *ModelOutput on top
21e956d5
format docs bit
0a7a13f9
update docs
c1365e30
update docs
4cc33e2a
update doc example
b77105d8
remove prune_heads from model
38d25bfd
remove unused config params
918d3020
refactor embed signature
4275757c
qubvel
commented
on 2025-06-11
Add vjepa to docs
ddd1c366
Fix config docstring
b8f67a42
pcuenca
commented
on 2025-06-11
pcuenca
commented
on 2025-06-11
pcuenca
commented
on 2025-06-11
pcuenca
commented
on 2025-06-11
pcuenca
commented
on 2025-06-11
update defaults
6d6aadcf
Update docs/source/en/model_doc/vjepa2.md
96614097
Update docs/source/en/model_doc/vjepa2.md
f67a4d74
Fix import
2908c5c4
Merge branch 'main' into koustuvs/oss
91028fc6
Min refactoring
fb95d52c
Update HUB_SOURCE and HUB_REPO in conversion script
39aafc37
Add missing headers
7e660ef3
VJEPA -> V-JEPA in docs
9061b3ac
Add image to doc
284b3e5e
fix style
a7d750b9
Merge branch 'koustuvs/oss' of https://github.com/qubvel/transformers…
f80ff373
fix init weights
10582465
LysandreJik
approved these changes
on 2025-06-11
change checkpoint name in modeling tests
a85656e5
Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub