transformers.js
Add `ignore_merges` option to BPE tokenizers
#716
Merged
Overview
Commits
1
Changes
View On GitHub

Add `ignore_merges` option to BPE tokenizers #716

xenova merged 1 commit into main from tokenizers-ignore-merges
xenova
xenova Add `ignore_merges` option to BPE tokenizers
3b252171
HuggingFaceDocBuilderDev
xenova xenova merged 6d5901e6 into main 2 years ago
xenova xenova deleted the tokenizers-ignore-merges branch 2 years ago

Login to write a write a comment.

Login via GitHub

Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum