server : add speculative decoding support #10455
ggerganov
merged
2 commits into
master
from
gg/speculative-server
Base automatically changed from
gg/speculative-refactor
to
master
1 year ago
server : add speculative decoding support
156aa6d9
server : add helper function slot.can_speculate()
0ba40c36
Reviewers
No reviews
Assignees
No one assigned
Labels
examples
server
Milestone
No milestone
Login to write a write a comment.
Login via GitHub