transformers.js
Improve tokenizer types based on input function parameters
#1641

Merged

Improve tokenizer types based on input function parameters #1641

xenova merged 6 commits into main from improve-tokenizer-types

Add tokenizer types tests

ac9a2233

update tests

45b8efcf

Improve types

634b7d01

xenova requested a review from

nico-martin 39 days ago

xenova commented on 2026-04-13

add back docs

7aa7011d

Update preprocess.js

0e20126f

fix build

1e70a062

nico-martin approved these changes on 2026-04-13

xenova merged ed84c15f into main 39 days ago

xenova deleted the improve-tokenizer-types branch 39 days ago

Reviewers

nico-martin

Assignees

No one assigned

Labels

None yet

Milestone

No milestone