Add Phi4 multimodal #36939
Cyrilvallez
merged
113 commits into
main
from
phi4
ArthurZucker
approved these changes
on 2025-03-24
raw start
00bcfd47
update
aef5f66f
update
60595b32
add to imports
ddfe10a6
update
88f473eb
up
5012749a
simplify configs
bc1d1975
clean configs
e56e7b0d
style
8d35ac95
typos
f4904822
Update convert_phi4_multimodal_weights_to_hf.py
c0e1da4e
Update convert_phi4_multimodal_weights_to_hf.py
c435c22e
fix
98b393c1
up
0bd29a3c
up
52bf0e8c
up
a37b0845
Update convert_phi4_multimodal_weights_to_hf.py
ce4735b3
Update convert_phi4_multimodal_weights_to_hf.py
fe9fed1b
up
5fffe534
up
c102b462
up
dbbad21b
Update feature_extraction_phi4_multimodal.py
67cad7f4
up
cc4cd0e8
up
da8b0aa7
up
d41fde15
up
f78aec53
up
18c33dea
simplify configs
abd15e41
typo
438ee1a8
cut code
01f68a09
typo
d942e26a
typo
0c1d0820
typo
e9910cc7
re
28c4d400
typo
1c744f07
up
2659e69f
up
d42a60e4
up
9a0c3742
add tests
7b83b9e7
fix
23bbdd5d
fix
7598e618
Update test_modeling_phi4_multimodal.py
a52acd41
up
42c9ca5f
Update test_modeling_phi4_multimodal.py
c35fdc06
doc
3e507288
fix
6638418e
up
09ea6b74
up
41fc578a
up
e1091098
up
0f4f425e
up
ca1d04b4
up
6377c06e
simplify
cb829f39
up
046cd482
simplify
3cbd8dd7
config docstrings
07f58277
cleanup
28b29e9c
clean
719f204e
typo
5dd09faf
typo
741bfdc0
fix
4f909ab5
Update phi4_multimodal.md
406ed4c6
fix
2de1ac38
fix
65bdb474
Update test_modeling_phi4_multimodal.py
a20dfebb
update
6acd428e
simplify reshapes and permutes
249d1fbf
up
1da7e9be
simplify special tokens
5bcc6c81
simplify processor a lot
014c89e7
Update processing_phi4_multimodal.py
4bf35b97
Update processing_phi4_multimodal.py
d06a8085
switch to fast processor
f754f106
image processor
1d18749f
Update image_processing_phi4_multimodal_fast.py
01778938
add lora extraction to converter
1bbb2987
Update convert_phi4_multimodal_weights_to_hf.py
80d0e837
Update __init__.py
923b2d09
add AudioInput type in audio_utils
136d45aa
rewrite feature_extraction: support torch batched FFT
fcd909bc
input_audio_embeds -> audio_input_features, input_image_embeds -> ima…
11444c7f
test update
44c72966
not mono channel warning update
33c61fdd
remove auto maps from processor
1a1e0241
kargs dispatch in processor
e28dbb04
simplify kwargs dispatch
07c21537
simplify merging
6334dd7c
remove default sampling rate
4aa8086a
style
93323fc6
Update test_modeling_phi4_multimodal.py
95e5597d
update doc
37b3dbe6
doc
bc6d6a58
torch only feature extractor
b2413772
make fake tokens adjustable
9c752b2d
up
e1091098
up
ca1d04b4
simplify
cb829f39
Update test_modeling_phi4_multimodal.py
a20dfebb
Update processing_phi4_multimodal.py
d06a8085
switch to fast processor
f754f106
rewrite feature_extraction: support torch batched FFT
fcd909bc
kargs dispatch in processor
e28dbb04
doc
bc6d6a58
torch only feature extractor
b2413772
fix
d9beef2a
Update processing_phi4_multimodal.py
17985f90
simplify mask
c169f367
last touch
067edbfa
fix copies
9bee9f3e
style
653b8ece
Update audio_utils.py
4213e97c
style
24390037
Update feature_extraction_phi4_multimodal.py
16f5ca84
Update __init__.py
5b773c8b
docstrings
a70f307e
copies
ac699b18
fix all checks
aa6664bf
back to fix-copies
c3a1a898
trigger CIs
095bb8a9
Update feature_extraction_phi4_multimodal.py
bdc8e386
improve tests with multimodal inputs
4f521955
trigger CIs
ec726d7d
Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub