llama.cpp
Make the drafter probabilistic and the target verify by rejection sampling for simple draft and MTP
#27694
Merged
Merged
Make the drafter probabilistic and the target verify by rejection sampling for simple draft and MTP #27694
ggerganov
merged
9 commits into
ggml-org:master
from
praneshgo:pgonegandla/spec-draft-rejection-sampling
danbev
commented
on 2026-09-09
danbev
approved these changes
on 2026-09-15
ggerganov
approved these changes
on 2026-10-02
Make the drafter probabilistic and the target verify by rejection sam…
269bb4fa
Drop stale spec_draft_q before drafting
2f82a679
Fallback to argmax sampling for grammar-constrained requests and addi…
94d05d5f
Support grammar-constrained requests in rejection sampling
b1b5b01c
Fix - renormalize distribution after masking
bba1f2e1
copy rng on sampler copy and re-accept drafted tokens on replay
d9eb9f0d
Fix draft sampler sharing the target's rng stream
b5c26731
Simplify the rejection sampler's inputs and move replay to the server
61de64e1
Truncate the draft candidates along with the draft
c19c18a7
ServeurpersoCom
approved these changes
on 2026-10-02
Reviewers
Assignees
Labels
server
merge ready
Milestone
No milestone
Login to write a write a comment.
Login via GitHub