llama.cpp
d1a1b614
- spm : fix special_add_bos default
Go
Login via GitHub
Home
Pricing
FAQ
Install
Login
via GitHub
Commit
View On
GitHub
Commit
1 year ago
spm : fix special_add_bos default When this is not set in HF `tokenizer_config.json`, it should default to true.
References
#6498 - BERT tokenizer fixes
Author
cebtenzzre
Parents
45983e3a
Loading