llama.cpp
Add more tokenizer tests
#3742
Merged
Overview
Commits
7
Changes
View On GitHub

Add more tokenizer tests #3742

goerch merged 7 commits into ggml-org:master from Galunid:add-tokenizer-tests
Galunid
Galunid Add more tokenizer tests
6919b8ce
Galunid Add starcoder
769135b6
goerch
goerch
Galunid
goerch
Galunid
Galunid Update test vocab files
226f0a85
Galunid
goerch
Galunid Restrict bpe tokenizer tests to unicode planes
a06e82bb
goerch
goerch commented on 2023-10-23
Galunid Update comment
1244b006
goerch
goerch approved these changes on 2023-10-23
goerch
Galunid Galunid force pushed from efd3c228 to 1244b006 1 year ago
Galunid Comment cosmetics
38de5d04
Galunid
ggerganov
ggerganov commented on 2023-10-23
Galunid Remove bloom vocab/test
a01ddfbc
goerch
goerch goerch merged daab3d7f into master 1 year ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum