Add sequence post processor #771
Add `Sequence` PostProcessor
c1332f8c
Support `return_token_type_ids`
c018f274
Add llama3 tokenizer to unit tests
d68afe29
Add test for allowing user to request for token type ids
0e5fb414
Add JSDoc
3b2f5cd6
Update generate_tests.py
078672b2
xenova
merged
db9250b3
into main 2 years ago
xenova
deleted the add-sequence-post-processor branch 2 years ago
Assignees
No one assigned
Login to write a write a comment.
Login via GitHub