Fix StopStringCriteria to handle tokens above len(tokenizer) #35797
Rocketknight1
marked this pull request as ready for review 1 year ago
Fix StopStringCriteria to handle tokens above len(tokenizer)
ef03b4c8
Use self.stop_strings instead of stop_strings
647d0db8
Handle clipping correctly
b8e17cbd
make fixup
f690398b
Update test to the new embedding vecs
097c57de
Use much bigger values in the mismatch test
cb8a0f93
Typo fix
15ff9efc
Slight simplification
a322e8d5
Rocketknight1
deleted the fix-stop-string-criteria-vocab-size branch 1 year ago
Assignees
No one assigned
Login to write a write a comment.
Login via GitHub