common : reimplement the logger
log : print if build is debug [no ci]
examples : move gpt_init() after parsing the cli args
log : add comments + adjust defaults
server : improve log format
server : fix verbose check
log : option to disable the log prefix
log : cleanup, comments, build flags
Please don't take away the optional JSON formatting for logs.
This has been an important step to automate sending warnings and errors to the front-end when hosting the server process. Parsing the often very unstructured text output is tedious and error prone, while having the structured format like JSON makes it a whizz. Some logs can even function as progress notifications for a front-end, such as LOG_INFO("model loaded")
@bviksoe The server logs were never meant to be used in such way. These messages can be disabled and changed (even when in JSON format) without notice and therefore 3rd-party code should never rely on them. Instead, your frontend can query the server through the available endpoints. If you have a specific functionality in mind that is currently missing, submit a feature request and it will get implemented. Model loading and server status can already be queried properly through the existing API.
Thanks. I understand your desire to clean up and streamline the various parts of the project.
My argument is specifically actual error and warning messages produces during loading and even during streaming inference.
In server there are many such messages that are conveyed only in log. An example: LOG_ERROR("failed to get embeddings"...) If you want to create a user-friendly front-end, these should be accessible to the website/api user.
If you provide sample curl requests, combined with the output that you would expect server to return for each of them, we can extend the API and the responses. Feel free to open a feature request and list the missing functionality.
Prior to this, it was possible to use --log-disable to stop llama-cli from printing all model loading debug logs. It would only print the prompt loaded with -p flag and of course model output + user input when in interactive mode. Now this is broken, it doesn't print the prompt and the model output. As far as I can see the verbosity options are not implemented completely yet to be able to facilitate the old behavior.
I hope with gets fixed in the future, as right now it only works if --log-disable is removed. But that will make it dump everything instead of just the prompt. Also, --log-enable was removed, which was useful too, but my guess is once the verbosity level settings work as expected this will cover that aspect.
All messages with non-zero log level are output to stderr, so you can redirect stderr to /dev/null to get the old behavior.
Hi Georgi,
Understood, I did not check if it was still using stderr or not. It just wasn't obvious at a glance. I suppose users can continue using the stderr redirection hack for the time being. My hope that in the future though, more logging verbose levels get implemented so that it is more obvious and easier to control for the end users.
Also, even if you redirect to stderr, it doesn't remove everything. For example, you still have this message:
"== Running in interactive mode. =="
At least it's not too bad to just edit the source code and change the logging function it uses.
Kind Regards,
Kyryl
@ggerganov , I noticed the same thing. --log-disable now breaks conversation mode
Don't use --log-disable. Instead redirect stderr to /dev/null if you don't need it.
The problem with redirecting stderr to /dev/null, is when a real error happens the user does not see it, for example if the model does not exists or not readable because of a permissions error.
| 1822 | 1819 | } | |
| 1823 | fprintf(stderr, "%.2f minutes\n", total_seconds / 60.0); | ||
| 1824 | |||
| 1825 | printf("\nchunk PPL ln(PPL(Q)/PPL(base)) KL Divergence Δp RMS Same top p\n"); | ||
| 1820 | LOG("%.2f minutes\n", total_seconds / 60.0); | ||
| 1826 | 1821 | } | |
| 1822 | LOG("\n"); | ||
| 1823 | LOG("chunk PPL ln(PPL(Q)/PPL(base)) KL Divergence Δp RMS Same top p\n"); |
Djip007220 days agowhy write it on every line and not as a header?
Login to write a write a comment.
ref #8566
Merge ETA: Sep 15
Overhaul
common/log.h. The main goal is to offload print IO to a separate thread in order to not affect the performance of the examples. Also add convenience options for timestamps, colors based on the log type and output to file.By default, the logs should look the same as they do on
master.Adding the following options will make them look like this:
Another important change is that the logs in
llama-serverhave been significantly reformatted. I've always had trouble reading the output, so I changed the text in a way that is IMO easier to read. Also removed the option to output json logs as I don't think it has any practical value.