llama.cpp

18b5f8b1 - server : accumulate generated text and tokens as parse input (#29876)

server : accumulate generated text and tokens as parse input (#29876) * server : collect text and token input * common : add a tokenize helper that aligns tokens with bytes * common : simplify tokenize logic