vllm
[Benchmark] Add benchmark script for CPU offloading
#11533

Merged

Commits

[Add] benchmark script for CPU offloading (long document QA use case)

ApostaC committed 348 days ago
Simplify the script and address the reviews

ApostaC committed 343 days ago
[fix] docstring style

ApostaC committed 343 days ago

Loading