transformers.js
Add support for using the tokenizer without loading onnx and tripping warnings
#1773

Open

Add support for using the tokenizer without loading onnx and tripping warnings #1773

koush wants to merge 2 commits into huggingface:main from koush:main

Add tokenizer-only package entry point

daf16186

support generic tokenizer backend alias

5ca2df39

xenova closed this 17 days ago

xenova reopened this 17 days ago

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Milestone

No milestone