transformers
Fast tokenizer encoding doesn't handle empty string input
#37537
Open
Overview
Commits
11
Changes
View On GitHub

Fast tokenizer encoding doesn't handle empty string input #37537

itazap wants to merge 11 commits into main from batched_handle_empty_string
itazap
itazap itazap marked this pull request as ready for review 348 days ago
github-actions github-actions requested a review from ArthurZucker ArthurZucker 348 days ago
HuggingFaceDocBuilderDev
itazap itazap force pushed from 9061ca99 to 5b081807 342 days ago
itazap itazap force pushed from 3e798fcd to 36099882 336 days ago
itazap handle when text str is ''
7e510e14
itazap revert files
47985993
itazap revert files
a6ecd8ba
itazap revert files
2e7e0f67
itazap adding test for empty string
9024123e
itazap check bytelevel bug, skip
37b52214
itazap layoutlmv2
59f9c9c5
itazap udop
ca8858eb
itazap markuplm
ff54c1d0
itazap copies
ceb666f8
itazap itazap force pushed from 36099882 to ceb666f8 335 days ago
itazap undo
c083a0ef
itazap
ArthurZucker
ArthurZucker approved these changes on 2025-06-02

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum