vllm
[Bugfix] Fix request cancellation without polling
#11190

Merged

Commits

:bug: Cancel requests without polling

joerunde committed 1 year ago
:fire: remove cancel handler where it doesn't work

joerunde committed 1 year ago
:bug: Add cancel hook to async llm engine

joerunde committed 1 year ago
:bug: add cancel to encode on async engine

joerunde committed 1 year ago
:test_tube: test openai server cancellation

joerunde committed 1 year ago
:bug: dial back the cancellation test

joerunde committed 1 year ago
Merge remote-tracking branch 'upstream/main' into cancel-fix

joerunde committed 1 year ago
:bug: add time back

joerunde committed 1 year ago
:bug: fit test in memory :(

joerunde committed 1 year ago