vllm
601e1c7c - [Perf] Give a prefill-only batch the model state's number of logit rows (#57416)

Commit
63 days ago
[Perf] Give a prefill-only batch the model state's number of logit rows (#57416) Signed-off-by: Matt Mastracci <matthew@mastracci.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Author
Parents
Loading