PR #8611 tests : re-enable tokenizer tests

tests : re-enable tokenizer tests #8611

ggerganov merged 5 commits into master from gg/update-tokenizer-tests

models : remove duplicated gpt-2 vocab

a6ece88d

models : remove old stablelm vocab

eeeffb84

tests : re-enable MPT tokenizer tests

1d44fc0f

tests : re-enable DeepSeek tokenizer tests

717f0e51

cmake : sort

38babee5

github-actions added testing

ggerganov merged e093dd23 into master 1 year ago

ggerganov deleted the gg/update-tokenizer-tests branch 1 year ago

Reviewers

No reviews

Assignees

No one assigned

Labels

testing

Milestone

No milestone