server : fix logprobs, make it OAI-compatible #10783
ngxson
merged
21 commits into
ggml-org:master
from
ngxson:xsn/fix_logprobs
server : fix logprobs, make it openai-compatible
74dc729c
update docs
78280136
ggerganov
commented
on 2024-12-11
ngxson
commented
on 2024-12-11
add std::log
01afafef
return pre-sampling p
cc90cdbc
sort before apply softmax
29c1495a
add comment
396ade0b
fix test
22b72c85
set p for sampled token
ed7f2d57
p-e-w
commented
on 2024-12-13
update docs
06bb38e7
ngxson
commented
on 2024-12-13
add --multi-token-probs
196e237e
Merge branch 'master' into xsn/fix_logprobs
262950db
update docs
630ddcc5
Merge branch 'master' into xsn/fix_logprobs
c0cca53d
add `post_sampling_probs` option
ecadd37c
update docs [no ci]
75fe7751
remove --multi-token-probs
8734df73
"top_probs" with "post_sampling_probs"
fd4cf34b
ggerganov
commented
on 2024-12-19
resolve review comments
d2463dc8
rename struct token_prob to prob_info
65ef1c8d
correct comment placement
a217382b
ggerganov
approved these changes
on 2024-12-19
fix setting prob for sampled token
5b966df1
ggerganov
approved these changes
on 2024-12-19
Reviewers
Assignees
No one assigned
Labels
breaking change
examples
python
server
Milestone
No milestone
Login to write a write a comment.
Login via GitHub