llama.cpp
57bb2c40 - server : fix logprobs, make it OAI-compatible (#10783)

Commit
362 days ago
server : fix logprobs, make it OAI-compatible (#10783) * server : fix logprobs, make it openai-compatible * update docs * add std::log * return pre-sampling p * sort before apply softmax * add comment * fix test * set p for sampled token * update docs * add --multi-token-probs * update docs * add `post_sampling_probs` option * update docs [no ci] * remove --multi-token-probs * "top_probs" with "post_sampling_probs" * resolve review comments * rename struct token_prob to prob_info * correct comment placement * fix setting prob for sampled token
Author
Parents
Loading