vllm
24a03915 - mla: don't update kv cache on dummy forwards (#36282)

Commit

48 days ago

mla: don't update kv cache on dummy forwards (#36282) Signed-off-by: Itay Alroy <ialroy@nvidia.com>

References

Author

itayalroy

Parents