generate (disagg) (#42644)

Commit

1 day ago

[Bugfix] Thread kv_transfer_params into engine for /inference/v1/generate (disagg) (#42644) Signed-off-by: hallerite <git@hallerite.com> Co-authored-by: OpenAI Codex <noreply@openai.com>

References

#42644 - [Bugfix] Thread kv_transfer_params into engine for /inference/v1/generate (disagg)

Author

hallerite

Parents

2ab18743

vllm c71f6f8a - [Bugfix] Thread kv_transfer_params into engine for /inference/v1/generate (disagg) (#42644)

vllm
c71f6f8a - [Bugfix] Thread kv_transfer_params into engine for /inference/v1/generate (disagg) (#42644)