vllm
dd2a94fd
- fix assert error num_tokens_across_dp is None
Go
Login via GitHub
Home
Pricing
FAQ
Install
Login
via GitHub
Commit
View On
GitHub
Commit
251 days ago
fix assert error num_tokens_across_dp is None or num_tokens_across_dp[dp_rank] == batchsize Signed-off-by: yewentao256 <zhyanwentao@126.com>
References
#23693 - [Core/DBO][1/N] Add Dual-Batch Overlap mechanism to VLLM
Author
yewentao256
Parents
e526b1c0
Loading