PR #8135 Added support for Viking pre-tokenizer

Added support for Viking pre-tokenizer #8135

slaren merged 1 commit into ggml-org:master from master

github-actions added python

slaren approved these changes on 2024-06-27

Added support for Viking pre-tokenizer

a50d4a8b

ggerganov approved these changes on 2024-06-27

slaren merged f675b20a into master 1 year ago

Reviewers

ggerganov

slaren

Assignees

No one assigned

Labels

python

Milestone

No milestone