llama.cpp
73122473 - fix(gguf-py): special tokens are no longer skipped when add_<token>_token is set to false (#5487)

Commit
1 year ago
fix(gguf-py): special tokens are no longer skipped when add_<token>_token is set to false (#5487) * fix(gguf-py): special tokens are no longer skipped when add_<token>_token is set to false * fix(gguf-py): added missing cls and mask token ids to the gguf metadata
Parents
Loading