vllm
[Perf][DSA][PCP] Async CP: take the per-layer context gathers of PCP + DCP prefill off the main stream
#69259
Open

Commits
  • [Bugfix][DSA] Free each dense indexer logits window before allocating the next
    Heisenberg-Yin committed 2 hours ago
  • [Bugfix][DSA][PCP] Plan PCP + DCP indexer prefill chunks for fused_mqa_topk
    Heisenberg-Yin committed 2 hours ago
  • [Perf][DSA][PCP] Take the per-layer context gathers of PCP + DCP prefill off the main stream
    Heisenberg-Yin committed 2 hours ago
  • [Perf][DSA][PCP] Rebuild the indexer K replica of a request it does not hold
    Heisenberg-Yin committed 2 hours ago
Loading