vllm
[Frontend][V1] Online serving performance improvements
#12287

Merged

[Frontend][V1] Online serving performance improvements #12287

mgoin merged 5 commits into vllm-project:main from njhill:v1-perf-smoothing

njhill requested a review from

WoosukKwon 1 year ago

njhill requested a review from

robertgshaw2-redhat 1 year ago

njhill requested a review from

ywang96 1 year ago

njhill requested a review from

comaniac 1 year ago

njhill requested a review from

alexm-redhat 1 year ago

mergify added frontend

[Frontend][V1] Online serving performance improvements

55dd1192

njhill force pushed to 55dd1192 1 year ago

njhill commented on 2025-01-22

Parallelize output socket IO on client side

0e92b61a

robertgshaw2-redhat commented on 2025-01-22

mgoin commented on 2025-01-22

ywang96 commented on 2025-01-22

Make max processing chunk size overridable, fix linting

aa7f0311

Merge remote-tracking branch 'refs/remotes/origin/main' into v1-perf-…

e6fc61f6

mgoin approved these changes on 2025-01-22

mgoin added ready

mergify added needs-rebase

Merge remote-tracking branch 'origin/main' into v1-perf-smoothing

eafe7cb0

mergify removed needs-rebase

mgoin enabled auto-merge (squash) 1 year ago

mgoin merged aea94362 into main 1 year ago

njhill deleted the v1-perf-smoothing branch 1 year ago

Reviewers

mgoin

ywang96

DarkLight1337

robertgshaw2-redhat

WoosukKwon

comaniac

alexm-redhat

Assignees

No one assigned

Labels

frontend ready

Milestone

No milestone

vllm [Frontend][V1] Online serving performance improvements #12287 Merged

[Frontend][V1] Online serving performance improvements #12287

vllm
[Frontend][V1] Online serving performance improvements
#12287

Merged