tokenizers
Adding an API for decode streaming.
#1677
Merged
Overview
Commits
9
Changes
View On GitHub

Adding an API for decode streaming. #1677

Narsil merged 9 commits into main from decode_stream
Narsil
Narsil Adding an API for decode streaming.
c97389bc
HuggingFaceDocBuilderDev
Narsil Add another missing test case (proving the effect of state.)
c3578d45
Narsil Ellide lifetime.
bdcb2b9c
Narsil Ellide bis.
af7d82eb
Narsil Fixing the streaming implementation.
5a5406e6
ArthurZucker
ArthurZucker commented on 2024-11-07
Narsil
Narsil Adding more docs.
18b999c6
Narsil End of list.
c32a2c24
Narsil Fix internal link.
a326447c
Narsil Skip doctest on Windows (no tokenizer file because no make)
218fd3b7
ArthurZucker
ArthurZucker approved these changes on 2024-11-14
Narsil Narsil merged 500db282 into main 1 year ago
Narsil Narsil deleted the decode_stream branch 1 year ago
ghost
ghost commented on 2024-11-18

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum