vllm
de3fe8dc - [Bugfix] release KV blocks for skipped P-ranks to prevent invalid KV errors and timeouts when P_tp > D_tp and MLA (#40449)

Commit

82 days ago

[Bugfix] release KV blocks for skipped P-ranks to prevent invalid KV errors and timeouts when P_tp > D_tp and MLA (#40449) Signed-off-by: yangruize <yangruize7@163.com> Co-authored-by: Roger Wang <hey@rogerw.io>

References

#40449 - [Bugfix] release KV blocks for skipped P-ranks to prevent invalid KV errors and timeouts when P_tp > D_tp and MLA

Author

yangrz7

Parents

0899f436

vllm de3fe8dc - [Bugfix] release KV blocks for skipped P-ranks to prevent invalid KV errors and timeouts when P_tp > D_tp and MLA (#40449)

vllm
de3fe8dc - [Bugfix] release KV blocks for skipped P-ranks to prevent invalid KV errors and timeouts when P_tp > D_tp and MLA (#40449)