llama.cpp
d4df7858 - convert_hf : reduce usages of the UNKNOWN token type

Commit

1 year ago

convert_hf : reduce usages of the UNKNOWN token type

References

#8228 - llama : fix pre-tokenization of non-special added tokens

Author

compilade

compilade

Parents

Loading