llama.cpp
477a66b0 - convert : correctly handle LLaMA tokenizer for Jamba (#16470)

Commit
75 days ago
convert : correctly handle LLaMA tokenizer for Jamba (#16470) * fix: convert_hf_to_gguf - change Jamba non-sentencepiece mode (tokenizer.json) vocab construction * fix: convert_hf_to_gguf - jamba non-sentencepiece tokenizer to use _set_vocab_llama_hf func * fix: convert_hf_to_gguf - removed get_vocab_base_pre from jamba
Author
Parents
Loading