transformers
Updating `char_to_token` documentation to note behaviour when `trim_offsets` is True
#33919
Merged
Merged
Updating `char_to_token` documentation to note behaviour when `trim_offsets` is True #33919
ArthurZucker
merged
1 commit into
huggingface:main
from
Craigacp:tokenizer-doc-fix
Updating char_to_token documentation.
b59fc191
itazap
approved these changes
on 2024-10-05
ArthurZucker
approved these changes
on 2024-10-05
Login to write a write a comment.
Login via GitHub