llama.cpp
server: run sampling in a threadpool
#24914
Open

server: run sampling in a threadpool #24914

ngxson wants to merge 4 commits into master from xsn/server_multithread_sampling
ngxson
ngxson server: run sampling in a threadpool
fe03cce8
ngxson wip
41ed530b
ngxson working
c62fdd5f
ngxson ngxson requested a review 6 days ago
ngxson ngxson changed the title Xsn/server multithread sampling server: run sampling in a threadpool 6 days ago
github-actions github-actions added examples
github-actions github-actions added server
ngxson add arg --threads-sampling
095058ca
ngxson ngxson requested a review 6 days ago
ngxson
ngxson commented on 2026-06-22
ggerganov
ggerganov commented on 2026-06-24
ngxson ngxson marked this pull request as draft 3 days ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
Milestone