vllm
90eb28ca - [V1][Scheduler] Use dict for running queue

Commit
276 days ago
[V1][Scheduler] Use dict for running queue This is just a random idea, still need to benchmark Potential advantages for large batch sizes: - Don't need to copy entire list every iteration - O(1) removal of aborted requests Signed-off-by: Nick Hill <nhill@redhat.com>
Author
Parents
Loading