llama.cpp
Minor improvements in GPT2 tokenizer
#3567
Merged

Commits
  • Fixing minor bugs in bpe_gpt2_preprocess
    goerch committed 2 years ago
  • Don't add bos token in test
    goerch committed 2 years ago
Loading