llama.cpp
sampling : optimize samplers by reusing bucket sort
#15665

Merged

sampling : optimize samplers by reusing bucket sort #15665

ggerganov merged 9 commits into master from gg/sampling-sort-opt

github-actions added testing

JohannesGaessler approved these changes on 2025-08-29

ggerganov requested a review from

ngxson 146 days ago

github-actions added examples

github-actions added server

ggerganov commented on 2025-08-30

ggerganov force pushed to 2efc7e46 146 days ago

ggerganov requested a review from

slaren 146 days ago

JohannesGaessler commented on 2025-08-30

slaren approved these changes on 2025-08-30

sampling : optimize sorting using bucket sort in more places

7d34a4bf

sampling : do not sort in dist sampler

97167e61

sampling : avoid heap allocations for sort buffers

15557b8f

common : add option to sort sampling candidates by probability

70bce496

sampling : revert the change for preserving sort buffers

c8a2adaf

sampling : use std::copy instead of memcpy

de2902dc

sampling : clarify purpose of partial sort helpers

6d2a38c7

ggerganov force pushed from d74a6ab4 to 6d2a38c7 145 days ago

cont : remove wrong comment [no ci]

08d5ff4f

common : update comment

1136efb2

ggerganov merged e92d53b2 into master 145 days ago

ggerganov deleted the gg/sampling-sort-opt branch 145 days ago

Reviewers

slaren

JohannesGaessler

ngxson

Assignees

No one assigned

Labels

testing examples server

Milestone

No milestone