vllm
[Core][Multimodal] Track encode cache entries by mm_hash and enable embedding sharing between requests
#22711

Merged

[Core][Multimodal] Track encode cache entries by mm_hash and enable embedding sharing between requests #22711

ywang96 merged 15 commits into vllm-project:main from fake0fan:cgzheng_encoder_cache

fake0fan requested a review from

WoosukKwon 151 days ago

fake0fan requested a review from

robertgshaw2-redhat 151 days ago

fake0fan requested a review from

njhill 151 days ago

fake0fan requested a review from

ywang96 151 days ago

fake0fan requested a review from

comaniac 151 days ago

fake0fan requested a review from

alexm-redhat 151 days ago

mergify added v1

mergify added tpu

gemini-code-assist commented on 2025-08-12

knlnguyen1802 force pushed to 8d3d91a5 151 days ago

mergify added needs-rebase

DarkLight1337 commented on 2025-08-20

ywang96 commented on 2025-08-20

knlnguyen1802 force pushed to e85fa5f0 143 days ago

mergify removed needs-rebase

knlnguyen1802 commented on 2025-08-21

huachenheli commented on 2025-08-21

mergify added needs-rebase

knlnguyen1802 force pushed 142 days ago

knlnguyen1802 force pushed 141 days ago

mergify removed needs-rebase

DarkLight1337 commented on 2025-08-22

knlnguyen1802 force pushed 141 days ago

DarkLight1337 commented on 2025-08-22

knlnguyen1802 force pushed 141 days ago

DarkLight1337 commented on 2025-08-22

knlnguyen1802 force pushed 141 days ago

ywang96 commented on 2025-08-22

[Feature] Support Encoder MM Cache

95bcc27e

knlnguyen1802 force pushed to 95bcc27e 141 days ago

Fix pre-commit

4be971cd

cleanup

df220515

Merge branch 'main' into cgzheng_encoder_cache

e05f1f87

ywang96 approved these changes on 2025-08-22

ywang96 changed the title ~~[Feature] Support Encoder MM Cache: switch cache key from (req_id, input_id) to mm_hash~~ [Core][Multimodal] Track encode cache entries by mm_hash and enable embedding sharing between requests 141 days ago

ywang96 added ready

fix test assumption

3fda1fda

Merge branch 'main' into cgzheng_encoder_cache

cdf20ece

precommit

487188d2

ywang96 requested changes on 2025-08-23

ywang96 commented on 2025-08-23

fix other test and precommit

6016ba39

update

f2da3532

Merge branch 'main' into cgzheng_encoder_cache

18347590

fix test

b0c126d4

Free encoder cache for pretempted request

bbcd7bf8

Merge pull request #9 from fake0fan/cgzheng_encoder_cache_free_pretem…

2b49bf51

move-up

d3c86f59

Merge branch 'main' into cgzheng_encoder_cache

47086266

ywang96 approved these changes on 2025-08-25

ywang96 merged d765cf01 into main 138 days ago

fake0fan deleted the cgzheng_encoder_cache branch 123 days ago

Reviewers

ywang96

DarkLight1337

russellb

knlnguyen1802

huachenheli

gemini-code-assist

WoosukKwon

robertgshaw2-redhat

njhill

comaniac

alexm-redhat

Assignees

No one assigned

Labels

tpu ready v1

Milestone

No milestone

vllm [Core][Multimodal] Track encode cache entries by mm_hash and enable embedding sharing between requests #22711 Merged

[Core][Multimodal] Track encode cache entries by mm_hash and enable embedding sharing between requests #22711

vllm
[Core][Multimodal] Track encode cache entries by mm_hash and enable embedding sharing between requests
#22711

Merged