llama.cpp
server: support multiple generations from one prompt (OAI "n" option)
#17775

Merged

server: support multiple generations from one prompt (OAI "n" option) #17775

ngxson merged 11 commits into ggml-org:master from ngxson:xsn/add_n_support

backend support

15ce574d

server: support multiple generations from one prompt (OAI "n" option)

0d842cb5

github-actions added examples

github-actions added server

fix invalid batch

bf33d13b

format oai

a768a5e8

clean up

5cc3156f

disable ctx shift

2a7728f5

add test

e0660710

update comments

46f6fd26

ngxson marked this pull request as ready for review 189 days ago

ngxson requested a review from

ggerganov 189 days ago

github-actions added python

fix style

b65ee647

add n_cmpl to docs [no ci]

6fb3226d

ggerganov approved these changes on 2025-12-06

allowing using both n_cmpl and n

ea7f0669

ngxson merged c42712b0 into master 188 days ago

ngxson commented on 2026-01-07

Reviewers

ggerganov

Assignees

No one assigned

Labels

examples python server

Milestone

No milestone