transformers.js
4a6addab - Return empty tokens array if text is empty after normalization (#535)

Commit

1 year ago

Return empty tokens array if text is empty after normalization (#535)

References

#535 - Return empty tokens array if text is empty after normalization

Author

xenova

xenova

Parents

Files1

src
- tokenizers.js