vllm
601e1c7c
- [Perf] Give a prefill-only batch the model state's number of logit rows (#57416)
Go
Login via GitHub
Home
FAQ
Install
Login
via GitHub
Commit
View On
GitHub
Commit
63 days ago
[Perf] Give a prefill-only batch the model state's number of logit rows (#57416) Signed-off-by: Matt Mastracci <matthew@mastracci.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
References
#57416 - [Perf] Give a prefill-only batch the model state's number of logit rows
Author
mmastrac
Parents
0c90804c
Loading