optimum
Fix get_wikitext2 tokenization bug causing sequence length warning
#2407
Open
Open
Fix get_wikitext2 tokenization bug causing sequence length warning #2407
Mr-Neutr0n
wants to merge
1 commit into
huggingface:main
from
Mr-Neutr0n:fix/get-wikitext2-tokenization-bug
Fix get_wikitext2 tokenization bug that causes sequence length warning
28128452
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub