transformers.js
Return empty tokens array if text is empty after normalization
#535
Merged
Overview
Commits
1
Changes
View On GitHub

Return empty tokens array if text is empty after normalization #535

xenova merged 1 commit into main from update-spm-tokenization
xenova
xenova Return empty tokens array if text is empty after normalization
5a9fc6a7
HuggingFaceDocBuilderDev
xenova xenova merged 4a6addab into main 1 year ago
xenova xenova deleted the update-spm-tokenization branch 1 year ago

Login to write a write a comment.

Login via GitHub

Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum