PR #18753 Fixing OPT fast tokenizer option.

Fixing OPT fast tokenizer option. #18753

Narsil merged 4 commits into huggingface:main from Narsil:fix_opt_fast

Narsil requested a review from

patrickvonplaten 3 years ago

SaulLu commented on 2022-08-25

patrickvonplaten approved these changes on 2022-08-31

patrickvonplaten approved these changes on 2022-09-02

Fixing OPT fast tokenizer option.

73b89323

Remove dependency on `pt`.

2c248c6c

Move it to GPT2 tokenization tests.

9ca6e933

Added a few tests.

1e4b9ee6

Narsil force pushed to 1e4b9ee6 3 years ago

Narsil merged 68bb33d7 into main 3 years ago

Narsil deleted the fix_opt_fast branch 3 years ago

Reviewers

patrickvonplaten

SaulLu

Assignees

No one assigned

Labels

None yet

Milestone

No milestone