PR #8244 Fix gemma2 tokenizer convert

Fix gemma2 tokenizer convert #8244

ngxson merged 3 commits into ggml-org:master from ngxson:xsn/fix_gemma2_tokenizer

fix gemma2 tokenizer convert

62dc4a92

ngxson requested a review from

ggerganov 1 year ago

github-actions added python

remove scores

1a99b5ec

improve code, fix new line issue

922a5e29

ngxson requested a review from

abetlen 1 year ago

abetlen approved these changes on 2024-07-01

ngxson merged 5fac350b into master 1 year ago

eran-medan commented on 2024-07-02

Reviewers

abetlen

eran-medan

ggerganov

Assignees

No one assigned

Labels

python

Milestone

No milestone