llama.cpp
c837981b - py : add Phi-1.5/Phi-2 tokenizer (#9361)

Commit

334 days ago

py : add Phi-1.5/Phi-2 tokenizer (#9361) * add phi2 tokenizer * add phi name to convert_hf_to_gguf_update.py * make tokenizer_pre consistent; llama.cpp work

References

#9361 - feat: add Phi-1.5/Phi-2 tokenizer

Author

daminho

Parents

3c26a164

Files2

convert_hf_to_gguf.py
convert_hf_to_gguf_update.py

llama.cpp c837981b - py : add Phi-1.5/Phi-2 tokenizer (#9361)

llama.cpp
c837981b - py : add Phi-1.5/Phi-2 tokenizer (#9361)