llama.cpp
5d016702 - server : include speculative decoding stats when timings_per_token is enabled (#12603)

Commit
363 days ago
server : include speculative decoding stats when timings_per_token is enabled (#12603) * Include speculative decoding stats when timings_per_token is true New fields added to the `timings` object: - draft_n : number of draft tokens generated - draft_accepted_n : number of draft tokens accepted - draft_accept_ratio: ratio of accepted/generated * Remove redundant draft_accept_ratio var * add draft acceptance rate to server console output
Author
Parents
Loading