llama.cpp
2ce20d22 - unicode : reject overlong UTF-8 and out-of-range codepoints (#30485)

Commit
5 days ago
unicode : reject overlong UTF-8 and out-of-range codepoints (#30485) * unicode : reject overlong UTF-8 and out-of-range codepoints * Update tests/test-unicode.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co> * tests: document byte-wise UTF-8 error recovery * break out of first test on failure --------- Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
Parents
Loading