transformers.js
9a42cf32 - Add support for new `tokenizers>=0.2.0` BPE serialization format

Commit
1 year ago
Add support for new `tokenizers>=0.2.0` BPE serialization format
Author
Parents
Loading