vllm
[Spec Decode] Update extract_hidden_states to use deferred kv_connector clear
#37013
Merged
Overview
Commits
6
Changes
View On GitHub

[Spec Decode] Update extract_hidden_states to use deferred kv_connector clear #37013

mgoin merged 6 commits into vllm-project:main from fynnsu:extract_hidden_states_update
fynnsu
fynnsu Update extract_hidden_states to deferred kv_connector clear
159911e7
fynnsu Patch bug in ExampleHiddenStatesConnector that causes occasional crashes
ab64c43e
fynnsu fynnsu requested a review from njhill njhill 143 days ago
fynnsu fynnsu requested a review from benchislett benchislett 143 days ago
fynnsu fynnsu requested a review from luccafong luccafong 143 days ago
fynnsu fynnsu requested a review from MatthewBonanni MatthewBonanni 143 days ago
fynnsu fynnsu requested a review from NickLucche NickLucche 143 days ago
fynnsu fynnsu requested a review from ApostaC ApostaC 143 days ago
fynnsu fynnsu requested a review from orozery orozery 143 days ago
mergify mergify added speculative-decoding
mergify mergify added v1
mergify mergify added kv-connector
gemini-code-assist
gemini-code-assist commented on 2026-03-13
tlrmchlsmth
tlrmchlsmth approved these changes on 2026-03-13
tlrmchlsmth tlrmchlsmth added ready
fynnsu lint
460e38f8
fynnsu fix docstring
c22855b3
fynnsu fynnsu force pushed to c22855b3 143 days ago
fynnsu fix missing attribute
308b63c2
fynnsu fix shape mismatch
30eb4cfe
mgoin
mgoin approved these changes on 2026-03-16
mgoin mgoin merged 04bf5a35 into main 140 days ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
speculative-decoding ready v1 kv-connector
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum