vllm
ad9d09e2 - [Perf] [Hybrid] Copy num_accepted_tokens in non-blocking way when not using prefix caching (#35442)

Commit

4 days ago

[Perf] [Hybrid] Copy num_accepted_tokens in non-blocking way when not using prefix caching (#35442) Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>

References

#35442 - [Perf] [Hybrid] Copy num_accepted_tokens in non-blocking way when not using prefix caching

Author

tdoublep

Parents

4beebfd1

vllm ad9d09e2 - [Perf] [Hybrid] Copy num_accepted_tokens in non-blocking way when not using prefix caching (#35442)

vllm
ad9d09e2 - [Perf] [Hybrid] Copy num_accepted_tokens in non-blocking way when not using prefix caching (#35442)