vllm
fix(ngram): match async ngram_gpu acceptance rate to CPU
#44056

Open

fix(ngram): match async ngram_gpu acceptance rate to CPU #44056

shiyangyang2001-lgtm wants to merge 1 commit into vllm-project:main from shiyangyang2001-lgtm:async-acceptance-rate-fix

shiyangyang2001-lgtm requested a review from

njhill 5 days ago

shiyangyang2001-lgtm requested a review from

benchislett 5 days ago

shiyangyang2001-lgtm requested a review from

luccafong 5 days ago

shiyangyang2001-lgtm requested a review from

MatthewBonanni 5 days ago

shiyangyang2001-lgtm requested a review from

WoosukKwon 5 days ago

shiyangyang2001-lgtm requested a review from

robertgshaw2-redhat 5 days ago

shiyangyang2001-lgtm requested a review from

ywang96 5 days ago

shiyangyang2001-lgtm requested a review from

alexm-redhat 5 days ago

shiyangyang2001-lgtm requested a review from

heheda12345 5 days ago

shiyangyang2001-lgtm requested a review from

ApostaC 5 days ago

shiyangyang2001-lgtm requested a review from

orozery 5 days ago

mergify added speculative-decoding

mergify added v1

shiyangyang2001-lgtm changed the title ~~[ngram][async] Fix ngram_gpu acceptance rate to match CPU ngram~~ fix(ngram): match async ngram_gpu acceptance rate to CPU 3 days ago

[spec decode] Pass ngram-trimmed invalid tokens to scheduler stats

9d284c3a

shiyangyang2001-lgtm force pushed from 08e5c0a1 to 9d284c3a 3 days ago

Reviewers

njhill

benchislett

luccafong

MatthewBonanni

WoosukKwon

robertgshaw2-redhat

ywang96

alexm-redhat

heheda12345

ApostaC

orozery

Assignees

No one assigned

Labels

speculative-decoding v1

Milestone

No milestone

vllm fix(ngram): match async ngram_gpu acceptance rate to CPU #44056 Open

fix(ngram): match async ngram_gpu acceptance rate to CPU #44056

vllm
fix(ngram): match async ngram_gpu acceptance rate to CPU
#44056

Open