Fix gemma2 tokenizer convert (#8244)

Commit

1 year ago

Fix gemma2 tokenizer convert (#8244) * fix gemma2 tokenizer convert * remove scores * improve code, fix new line issue