llama.cpp
e76627bc - py : improve BPE tokenizer support (#5189)

Commit

1 year ago

py : improve BPE tokenizer support (#5189)

References

#5189 - Support for all cases that have/haven't `["model"]["vocab"]`.

Author

Sang-Kil Park

Parents

Loading