openvino_tokenizers
2368834b - Update Tokenizer Caching (#83)

Commit
2 years ago
Update Tokenizer Caching (#83) * Add Model Caching To Sentencepiece evaluate * Rewrite caching for Wordpiece and Regex Ops * Rewrite caching for BPE
Author
Parents
Loading