llama.cpp
perplexity : support using multiple sequences to allow larger batch sizes
#5946

Merged

perplexity : support using multiple sequences to allow larger batch sizes #5946

slaren merged 3 commits into master from sl/ppl-batching

perplexity : support using multiple sequences to allow larger batch s…

7d999555

slaren force pushed to 7d999555 2 years ago

compilade commented on 2024-03-09

set cparams.n_parallel to the number of sequences

ac07f7d0

compilade commented on 2024-03-09

ggerganov approved these changes on 2024-03-09

print tested n_ctx, add assert

23dbcfa2

slaren merged d894f352 into master 2 years ago

slaren deleted the sl/ppl-batching branch 2 years ago

Reviewers

ggerganov

compilade

Assignees

No one assigned

Labels

None yet

Milestone

No milestone