speculative : add tree-based sampling example #3624
ggerganov
merged
18 commits into
master
from
speculative-tree
sampling : one sequence per sampling context
5261aee8
speculative : add tree-based sampling support
4de5a2d4
ggerganov
changed the title 
speculative : reuse the n_parallel CLI param
32a67cbd
speculative : refactor sampling
4a7f43f2
examples : fix build after sampling refactoring
7e48e21b
batched : fix n_seq_id
0d96efab
sampling : fix malloc
b5554b9e
swift : fix build
b8acb6c9
swift : try to fix build
5b34bfa2
prompts : add assistant.txt
00594910
common : add llama_batch_add() and llama_batch_clear() helpers
360a3331
speculative : minor refactor
1c626e2f
minor : comments + rename
373d782d
speculative : fix off-by-one for n_drafted
f07cd35d
speculative : fix the n_drafted fix + p constants
e6dd81f0
Merge branch 'master' into speculative-tree
010c52ec
Merge branch 'master' into speculative-tree
bd9451ca
Merge branch 'master' into speculative-tree
ad2727d0
Reviewers
No reviews
Assignees
No one assigned
Labels
refactoring
need feedback
Milestone
No milestone
Login to write a write a comment.
Login via GitHub