llama.cpp
b65bb4ba - server: expose prompt token counts in /slots endpoint (#23454)

Commit
11 days ago
server: expose prompt token counts in /slots endpoint (#23454) Add n_prompt_tokens, n_prompt_tokens_processed, and n_prompt_tokens_cache to the /slots JSON response. These fields are already tracked internally but were not exposed, making it impossible for clients to monitor prompt evaluation progress during processing.
Author
Parents
Loading