llama.cpp
server: Ensure batches are either all embed or all completion (#8076)
#8420
Merged
Merged
server: Ensure batches are either all embed or all completion (#8076) #8420
ggerganov
merged
2 commits into
ggml-org:master
from
iamlemec:server-embed
make sure batches are all embed or all non-embed
40c99abb
iamlemec
changed the title 
compilade
commented
on 2024-07-10
non-embedding batch for sampled tokens; fix unused params warning
371cb8df
ggerganov
approved these changes
on 2024-07-12
Login to write a write a comment.
Login via GitHub