PR #19980 Update Special Language Tokens for PLBART

Update Special Language Tokens for PLBART #19980

sgugger merged 11 commits into huggingface:main from jordiclive:plbart_tok

Update Special Language Tokens for PLBART

5e0d262d

fix format

5d84f04d

making mapping for language codes and updating tests:

0aead7c0

fix format

ac35a89f

jordiclive force pushed 3 years ago

jordiclive force pushed to ac35a89f 3 years ago

fix consistency

28096ee9

add assert to both tokenizer tests.

f1b79143

fix format

baa12eec

ArthurZucker commented on 2022-11-14

Update src/transformers/models/plbart/tokenization_plbart.py

8ef02da3

improvin readability, setting self.tgt_lang

d41a3e80

fixing

42f3a03c

ArthurZucker approved these changes on 2022-11-17

sgugger approved these changes on 2022-11-17

readability

e4a375ce

sgugger merged 149483b2 into main 3 years ago

Reviewers

ArthurZucker

sgugger

Assignees

No one assigned

Labels

None yet

Milestone

No milestone