llama.cpp
91f64993 - Respect tokenizer.ggml.add_bos_token value when tokenizing (#4040)

Commit
2 years ago
Respect tokenizer.ggml.add_bos_token value when tokenizing (#4040) * gguf-py: gguf-dump: Respect --no-tensor flag in JSON mode. * Respect add_bos_token GGUF metadata value * gguf-py: Try to fix SpecialVocab giving up too easily for the Nth time
Author
Parents
Loading