PR #9524 Refactor `prepare_seq2seq_batch`

Refactor `prepare_seq2seq_batch` #9524

sgugger merged 8 commits into master from tokenize_target

Add target contextmanager and rework prepare_seq2seq_batch

78120262

Fix tests, treat BART and Barthez

33549496

Add last tokenizers

e684149f

sgugger requested a review from

patrickvonplaten 5 years ago

sgugger requested a review from

patil-suraj 5 years ago

sgugger requested a review from

LysandreJik 5 years ago

sgugger commented on 2021-01-11

Fix test

aa83f4d0

sgugger commented on 2021-01-11

LysandreJik approved these changes on 2021-01-12

patrickvonplaten commented on 2021-01-12

Set src token before calling the superclass

39e9914b

Remove special behavior for T5

49e039a0

Remove needless imports

d1c630d3

Remove needless asserts

1b3c7e9b

sgugger merged 063d8d27 into master 5 years ago

sgugger deleted the tokenize_target branch 5 years ago

Reviewers

LysandreJik

patrickvonplaten

patil-suraj

Assignees

No one assigned

Labels

None yet

Milestone

No milestone