Add image text to text pipeline #34170
yonigozlan
merged
45 commits into
huggingface:main
from
yonigozlan:add-image-text-to-text-pipeline
Rocketknight1
commented
on 2024-10-23
yonigozlan
commented
on 2024-10-25
ArthurZucker
approved these changes
on 2024-10-31
Standardize image-text-to-text-models-output
de37f38e
nit var name post_process_image_text_to_text udop
a7e56fae
nit fix deprecation warnings
9161a52a
Add image-text-to-text pipeline
04d4f77a
add support for image url in chat template for pipeline
0aabcb20
Reformat to be fully compatible with chat templates
5189f169
Add tests chat template
a0c90753
Fix imports and tests
726933fb
Add pipeline tag
b1d7a34f
change logic handling of single prompt ans multiple images
f92628c7
add pipeline mapping to models
fa254114
fix batched inference
9fe26c72
fix tests
316cf7d4
Add manual batching for preprocessing
c633b4b3
Fix outputs with nested images
d6598da4
Add support for all common processing kwargs
c8e58026
Add default padding when multiple text inputs (batch size>1)
20cdd5a1
nit change version deprecation warning
5bc43be8
Add support for text only inference
6cccf5fe
add chat_template warnings
ba8f85f2
Add pipeline tests and add copied from post process function
00174e82
Fix batched pipeline tests
8a65ea46
nit
da05987b
Fix pipeline tests blip2
1f2dafb3
remove unnecessary max_new_tokens
d66e5232
revert processing kosmos2 and remove unnecessary max_new_tokens
5056aa53
fix pipeline tests idefics
fe7e75d5
Force try loading processor if pipeline supports it
b866c279
revert load_processor change
3118dac4
hardcode loading only processor
065542aa
remove unnecessary try except
7f583dfb
skip imagetexttotext tests for kosmos2 as tiny model causes problems
aad9ad42
Make code clearer
e227b832
Address review comments
8f370f44
remove preprocessing logic from pipeline
c82fe29f
fix fuyu
7e1fb070
add BC resize fuyu
f581eaac
Move post_process_image_text_to_text to ProcessorMixin
4eda9631
add guard in post_process
02632213
fix zero shot object detection pipeline
45c17061
add support for generator input in pipeline
2e69b976
nit
58a6fb86
change default image-text-to-text model to llava onevision
66c017cb
fix owlv2 size dict
5772312f
Change legacy deprecation warning to only show when True
61cc5767
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub