vllm
perf: Avoid copying inputs_embeds tensors to GPU unless prompt_embeds is enabled
#25739

Merged

perf: Avoid copying inputs_embeds tensors to GPU unless prompt_embeds is enabled #25739

vllm-bot merged 2 commits into vllm-project:main from protopia-ai:only-copy-to-gpu-on-prompt-embeds

perf: Avoid copying inputs_embeds tensors to GPU unless prompt_embeds…

b3e5dc63

qthequartermasterman requested a review from

WoosukKwon 310 days ago

qthequartermasterman requested a review from

robertgshaw2-redhat 310 days ago

qthequartermasterman requested a review from

njhill 310 days ago

qthequartermasterman requested a review from

ywang96 310 days ago

qthequartermasterman requested a review from

comaniac 310 days ago

qthequartermasterman requested a review from

alexm-redhat 310 days ago

mergify added v1

gemini-code-assist commented on 2025-09-26

fix: do not copy token ids from input batch to CPU tensor unless prom…

af40fa66

DarkLight1337 approved these changes on 2025-09-26

DarkLight1337 enabled auto-merge (squash) 310 days ago

github-actions added ready

vllm-bot merged d48f4d6d into main 310 days ago

Reviewers

DarkLight1337

gemini-code-assist

WoosukKwon

robertgshaw2-redhat

njhill

ywang96

comaniac

alexm-redhat

Assignees

No one assigned

Labels

ready v1

Milestone

No milestone