transformers
1f62d4ed - Update ModelType for Unigram tokenizer (#43860)

Commit
17 hours ago
Update ModelType for Unigram tokenizer (#43860) Correct Unigram model type is 1 not 2. https://github.com/google/sentencepiece/blob/master/src/sentencepiece_model.proto#L48-L52
Author
Parents
Loading