llama.cpp
2b2afade - convert : fix encoding of WPM vocab for BERT models (#18500)

Commit
8 days ago
convert : fix encoding of WPM vocab for BERT models (#18500) * convert: avoid token collision when stripping ## prefix * convert: use token types for BERT special tokens check * Update convert_hf_to_gguf.py Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> --------- Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
Author
Parents
Loading