[Optimization] Cache sampled token ids in model runner #20291
WoosukKwon
merged
25 commits into
main
from
woosuk/token-id
Implement Async Scheduler
487efdc3
Merge branch 'main' into woosuk/async-sched
a3c320d7
optimization
bde64a48
Flatten cached_reqs_data
e4f91493
fix nccl connector
5804545c
shared storage connector
5fcb42d5
fix test
f06cb35a
fix more tests
c849f586
Merge branch 'main' into woosuk/async-sched
662a60db
Merge branch 'woosuk/serial' into woosuk/async-sched
85256779
Merge branch 'woosuk/async-sched' of https://github.com/vllm-project/…
388774b7
fix
85384796
Merge branch 'main' into woosuk/async-sched
3c783e66
Merge branch 'main' into woosuk/token-id
7ade9aac
[Optimization] Cache sampled token ids in model runner
a3b9964f
minor
278c9779
minor
26728f53
gemini-code-assist
commented
on 2025-07-01
fix shared storage connector
20642834
Merge branch 'main' into woosuk/token-id
0a0e177c
yapf
ea07d183
minor
e0f8c514
fix
d577f7b0
LucasWilkinson
commented
on 2025-07-01
LucasWilkinson
commented
on 2025-07-01
address review
c51372e1
LucasWilkinson
approved these changes
on 2025-07-01
Merge branch 'main' into woosuk/token-id
3441d025
Fix test_gpu_model_runner
23172945
Reviewers
Assignees
No one assigned
Labels
ready
v1
Milestone
No milestone
Login to write a write a comment.
Login via GitHub