PR #39242 [vlm] fix loading of retrieval VLMs

[vlm] fix loading of retrieval VLMs #39242

zucchini-nlp merged 11 commits into huggingface:main from zucchini-nlp:fix-rerieval-vlm

fix vlm with retrieval

aae88415

we can't use AutoModel because new ColQwen was released after refactor

e61be9f4

zucchini-nlp requested a review from

ydshieh 1 year ago

zucchini-nlp removed review request from

ydshieh 1 year ago

zucchini-nlp requested a review from

Cyrilvallez 1 year ago

zucchini-nlp requested a review from

ydshieh 1 year ago

no need for colqwen

c7050a1d

ydshieh approved these changes on 2025-07-07

tied weight keys are necessary, if using IMageTextToText

4170e2fa

Merge remote-tracking branch 'upstream/main' into fix-rerieval-vlm

fa66afa4

need to apply renaming in tied weights, only for ColPali

6b76b163

Merge branch 'main' into fix-rerieval-vlm

f8d6dcf4

zucchini-nlp commented on 2025-07-09

Merge branch 'main' into fix-rerieval-vlm

be9f7818

Cyrilvallez commented on 2025-07-14

overwrite tied keys in ColPali

26fc242c

Merge branch 'main' into fix-rerieval-vlm

b7c499bd

fix copies, modular can't handle if-statements

d2eeb29e

zucchini-nlp merged 9f41f671 into main 1 year ago

Reviewers

ydshieh

Cyrilvallez

Assignees

No one assigned

Labels

None yet

Milestone

No milestone