PR #5474 tests : multi-thread the tokenizer tests

tests : multi-thread the tokenizer tests #5474

ggerganov merged 3 commits into master from gg/mt-tokenizer-tests

tests : multi-thread the tokenizer tests

21851c11

ggerganov force pushed 1 year ago

unicode : fix data race for unidentified codepoints

6c533edb

ggerganov force pushed to 6c533edb 1 year ago

unicode : minor style fixes

bbc0ebb9

ggerganov merged cf45252a into master 1 year ago

ggerganov deleted the gg/mt-tokenizer-tests branch 1 year ago

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Milestone

No milestone