PR #36818 Use `lru_cache` for tokenization tests

Use `lru_cache` for tokenization tests #36818

ydshieh merged 6 commits into main from tokenizer_wtf

github-actions marked this pull request as draft 1 year ago

ydshieh marked this pull request as ready for review 1 year ago

github-actions requested a review from

ArthurZucker 1 year ago

github-actions requested a review from

Rocketknight1 1 year ago

ArthurZucker removed review request from

Rocketknight1 1 year ago

ArthurZucker removed review request from

ArthurZucker 1 year ago

ydshieh force pushed from 612e8025 to b5e24c93 1 year ago

ydshieh force pushed from fbf9bd3c to 18c7b40e 1 year ago

ydshieh changed the title ~~check tok~~ Use `lru_cache` for tokenization tests 1 year ago

fix

78fe0a8a

ydshieh force pushed from 284f9452 to 78fe0a8a 1 year ago

fix

155ee8ef

fix

e95edbf4

fix

464bce74

Merge branch 'main' into tokenizer_wtf

3b40c12b

ydshieh requested a review from

ArthurZucker 1 year ago

ArthurZucker approved these changes on 2025-03-27

Merge branch 'main' into tokenizer_wtf

ba859fe8

ydshieh merged 1fcaad6d into main 1 year ago

ydshieh deleted the tokenizer_wtf branch 1 year ago

Reviewers

ArthurZucker

Assignees

No one assigned

Labels

None yet

Milestone

No milestone