vllm
[Bugfix] Fix request cancellation without polling
#11190

Merged

[Bugfix] Fix request cancellation without polling #11190

simon-mo merged 9 commits into vllm-project:main from joerunde:cancel-fix

:bug: Cancel requests without polling

e93f0c6f

mergify added frontend

:fire: remove cancel handler where it doesn't work

ce1effcf

DarkLight1337 requested a review from

njhill 1 year ago

:bug: Add cancel hook to async llm engine

9af87481

:bug: add cancel to encode on async engine

6356ec83

:test_tube: test openai server cancellation

4a3831e7

joerunde requested a review from

DarkLight1337 1 year ago

joerunde requested a review from

robertgshaw2-redhat 1 year ago

joerunde requested a review from

simon-mo 1 year ago

mgoin added ready

mgoin approved these changes on 2024-12-16

:bug: dial back the cancellation test

c71878d3

Merge remote-tracking branch 'upstream/main' into cancel-fix

6432af75

:bug: add time back

01db8c49

:bug: fit test in memory :(

bd312bed

simon-mo merged 2d1b9baa into main 1 year ago

Reviewers

mgoin

njhill

DarkLight1337

robertgshaw2-redhat

simon-mo

Assignees

No one assigned

Labels

frontend ready

Milestone

No milestone