transformers
f74560d0 - Fix missing spaces in system prompt of Llama2 tokenizer (#24930)

Commit
2 years ago
Fix missing spaces in system prompt of Llama2 tokenizer (#24930) * Update tokenization_llama.py * Update tokenization_llama_fast.py * Update src/transformers/models/llama/tokenization_llama_fast.py Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com> * Update src/transformers/models/llama/tokenization_llama.py Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com> * Update src/transformers/models/llama/tokenization_llama.py Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com> * Update src/transformers/models/llama/tokenization_llama_fast.py Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com> --------- Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
Author
Parents
Loading