llama : lookup word in vocab before doing BPE merges (#7193)
* fix: llama-3 ignore_merges
* test: add test for llama-3 bpe ignore_merges
* fix: set ignore_merges only for llama-3
* fix: test-tokenizer-1-bpe --ingore-merges detection
* fix: copy to fix fallthrough
* fix: change ignore_merges to bool
* fix: add ignore merges tests to cmake
* llama : alternative merge ignore logic
---------
Co-authored-by: Haoxiang Fei <feihaoxiang@idea.edu.cn>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
