vllm
[Core] Initialize LoRA support for tower and connector in multi-modal models
#26674

Merged

[Core] Initialize LoRA support for tower and connector in multi-modal models #26674

vllm-bot merged 91 commits into vllm-project:main from jeejeelee:mlm-full-lora-support

add mm_punica_warpper

e7026a7c

Merge branch 'main' of https://github.com/B-201/vllm into v1-support-…

98debc24

update mm filter

7db0d599

Merge branch 'main' of https://github.com/B-201/vllm into v1-support-…

e2caeb33

update

891df1db

update

236e0fe9

update

cb1a6f07

update

0fa9111e

update

882213cc

update

8863bd2b

Init

5c156c9f

jeejeelee requested a review from

sighingnow 218 days ago

jeejeelee requested a review from

WoosukKwon 218 days ago

jeejeelee requested a review from

robertgshaw2-redhat 218 days ago

jeejeelee requested a review from

njhill 218 days ago

jeejeelee requested a review from

ywang96 218 days ago

jeejeelee requested a review from

comaniac 218 days ago

jeejeelee requested a review from

alexm-redhat 218 days ago

jeejeelee requested a review from

DarkLight1337 218 days ago

jeejeelee requested a review from

NickLucche 218 days ago

jeejeelee marked this pull request as draft 218 days ago

mergify added multi-modality

mergify added qwen

mergify added v1

gemini-code-assist commented on 2025-10-13

chatgpt-codex-connector commented on 2025-10-13

[feat] add connector support

a69bde7e

Merge pull request #7 from prashanth058/mlm-connector-support

112779fd

mergify added tpu

mergify added needs-rebase

Merge remote-tracking branch 'origin/main' into mlm-full-lora-support

ee0cdef3

fix pre-commit

a3647878

mergify removed needs-rebase

qwen2.5 & 3 vl fixes and tests

81573635

Merge pull request #9 from prashanth058/vision-lora-fixes

7e710bc3

remove redundant assingments

181b5f85

Merge pull request #10 from prashanth058/lora-vision-misc-fixes

9d41f6e9

fix bug

92ed13c1

Merge remote-tracking branch 'origin/main' into mlm-full-lora-support

dd912860

mergify added needs-rebase

jeejeelee changed the title ~~[Core] LoRA support for tower modules in multi-modal models~~ [Core] Initialize LoRA support for tower and connector in multi-modal models 167 days ago

Address conflict

83556e9d

mergify removed needs-rebase

Merge branch 'main' into mlm-full-lora-support

e6358610

jeejeelee marked this pull request as ready for review 167 days ago

jeejeelee requested a review from

tjtanaa 167 days ago

jeejeelee added ready

chatgpt-codex-connector commented on 2025-12-03

Merge branch 'mlm-full-lora-support' of https://github.com/jeejeelee/…

e6784ed3

Merge remote-tracking branch 'origin/main' into mlm-full-lora-support

c0cc07e7

Merge branch 'main' into mlm-full-lora-support

c94cdf1c

fix bug

598052b0

Merge branch 'mlm-full-lora-support' of https://github.com/jeejeelee/…

f67ccfae

jeejeelee removed ready

address the ci issue

1745bb93

add a enable option

113eb2e0

linitra24 requested a review from

youkaichao 164 days ago

linitra24 requested a review from

mgoin 164 days ago

linitra24 requested a review from

tlrmchlsmth 164 days ago

linitra24 requested a review from

houseroad 164 days ago

linitra24 requested a review from

hmellor 164 days ago

linitra24 requested a review from

yewentao256 164 days ago

linitra24 requested a review from

ProExpertProg 164 days ago

Merge branch 'main' into mlm-full-lora-support

1fbd7287

address ci issue

3e33423e

Merge remote-tracking branch 'origin/main' into mlm-full-lora-support

5ff0c6fb

Merge branch 'main' into mlm-full-lora-support

f6a1357c

jeejeelee added ready

address ci issue

d3c2f3db

address ci issue

1d2c5397

Merge branch 'mlm-full-lora-support' of https://github.com/jeejeelee/…

bbd90e8b

address pre-commit & ci issue

e2ea025e

Merge branch 'main' into mlm-full-lora-support

d1307e1d

jeejeelee commented on 2025-12-11

update argument name

27448490

remove outdated comment

65e403d3

Merge branch 'mlm-full-lora-support' of https://github.com/jeejeelee/…

dd857e48

Merge branch 'main' into mlm-full-lora-support

e10321bf

jeejeelee commented on 2025-12-11

Fix comments

208dc0c9

jeejeelee removed ready

Move forward

d4f39dc3

Move forward

06426107

update packed modules mapping (#11)

5e78570c

jeejeelee added ready

Move forward

1cb35461

jeejeelee force pushed to 1cb35461 157 days ago

Merge branch 'main' into mlm-full-lora-support

421707de

mergify added documentation

jeejeelee removed ready

Move forward

35acd22a

Isotr0py commented on 2025-12-12

DarkLight1337 commented on 2025-12-12

fix the issue with the MM token count

6a3f0a5a

fix pre-commit

c1bb71ef

update punica_wrapper_mapping

58d2c47b

jeejeelee requested a review from

Isotr0py 153 days ago

jeejeelee requested a review from

DarkLight1337 153 days ago

Merge branch 'main' into mlm-full-lora-support

bdac2b5d

fix bug

57917818

Merge branch 'main' into mlm-full-lora-support

94dce5c3

DarkLight1337 commented on 2025-12-16

added abstract methods to the base class

da0adea8

fixed property bug in processor and added abstract methods in BasePro…

36121c6d

Fix

3d39188d

Merge branch 'main' into mlm-full-lora-support

fe104bd6

fix pre-commit

1c8e3c44

remove hacky code

df3ec221

fix bug

764aa451

Merge branch 'main' into mlm-full-lora-support

463074fa

jeejeelee added ready

jeejeelee removed ready

jeejeelee marked this pull request as draft 150 days ago

Fix

d053aa73

fix

9c9950c0

DarkLight1337 commented on 2025-12-20

correct f-string formatting

4c2e95ad

Merge branch 'main' into mlm-full-lora-support

e5ba472a

jeejeelee marked this pull request as ready for review 149 days ago

jeejeelee added ready

chatgpt-codex-connector commented on 2025-12-20

Isotr0py commented on 2025-12-20

added ProcessingInfoMixin for QwenVL series models

b03d1a04

DarkLight1337 commented on 2025-12-20

Revert the mixin changes

d525556a

Merge branch 'jeejeelee:mlm-full-lora-support' into mlm-full-lora-sup…

cd32aead

extended SupportsMultiModal

c6831e79

fix bug

68116edf

fix pre-commit

cb72a0ef

Merge branch 'mlm-full-lora-support' of https://github.com/jeejeelee/…

2b03137f

removed get_allowed_mm_limits() from models

86c6c5cf

Merge pull request #12 from Anexdeus/mlm-full-lora-support

a3a8fc1f

move mm-token-functions to model

20402090

revert lora_kwargs change

81b5ace1

fix

fa6dd854

Merge branch 'main' into mlm-full-lora-support

8aedddd5

Isotr0py commented on 2025-12-22

fix mm_hash

f3a55ff9

mergify added needs-rebase

tjtanaa added rocm

disable mm cache when enable_tower_connector_lora

f114b4e1

Merge remote-tracking branch 'origin/main' into mlm-full-lora-support

a9b9af91

mergify removed needs-rebase

fix pre-commit

390ac9a4

mergify added needs-rebase

Merge remote-tracking branch 'origin/main' into mlm-full-lora-support

f9ca6854

mergify removed needs-rebase

mergify added needs-rebase

fix pre-commit

ac8afb6a

cleanup

57d7267f

mergify removed needs-rebase

Isotr0py approved these changes on 2025-12-24

Merge branch 'main' into mlm-full-lora-support

b275e8c9

fix ci issue

7183a2fe

Merge branch 'mlm-full-lora-support' of https://github.com/jeejeelee/…

41a04929

vllm-bot merged ce1eafd1 into main 143 days ago

jeejeelee deleted the mlm-full-lora-support branch 141 days ago

Reviewers

Isotr0py

DarkLight1337

chatgpt-codex-connector

linitra24

prashanth058

Anexdeus

gemini-code-assist

sighingnow

WoosukKwon

robertgshaw2-redhat

njhill

ywang96

comaniac

alexm-redhat

NickLucche

tjtanaa

youkaichao

mgoin

tlrmchlsmth

houseroad

hmellor

yewentao256

ProExpertProg

Assignees

No one assigned

Labels

documentation rocm tpu ready v1 multi-modality qwen

Milestone

No milestone

vllm [Core] Initialize LoRA support for tower and connector in multi-modal models #26674 Merged

[Core] Initialize LoRA support for tower and connector in multi-modal models #26674

vllm
[Core] Initialize LoRA support for tower and connector in multi-modal models
#26674

Merged