transformers
Adding support for BPE merge creation from scores instead of ids.
#22582

Merged

Adding support for BPE merge creation from scores instead of ids. #22582

Narsil merged 4 commits into main from update_convert_slow

Adding support for BPE merge creation from scores instead of ids.

bd85ad8e

Revert warn -> raise.

05aba79f

Narsil requested a review from

ArthurZucker 2 years ago

Narsil requested a review from

sgugger 2 years ago

ArthurZucker commented on 2023-04-05

Narsil commented on 2023-04-05

Update src/transformers/convert_slow_tokenizer.py

d217eebc

Quality.

dd4cc762

sgugger approved these changes on 2023-04-05

Narsil merged 126eafe3 into main 2 years ago

Narsil deleted the update_convert_slow branch 2 years ago

Reviewers

sgugger

ArthurZucker

Assignees

No one assigned

Labels

None yet

Milestone

No milestone