younesbelkada/transformers

Pull Requests Commits

Added sentence piece and yttk vocab extraction in transformers-cli

mfuntowicz committed 6 years ago

2d0f9420

Albert integration with tokenizers.

mfuntowicz committed 6 years ago

b0d57472

Copying tokenizers sentence piece conversion class into transformers.

mfuntowicz committed 6 years ago

eeeba7a2

More rust tokenizer output cleaning on python side.

mfuntowicz committed 6 years ago

701511c2

Fix missing <s> ... </s> tokens around Roberta inputs.

mfuntowicz committed 6 years ago

c1271bae

Fix Roberta missing vocab files overriding.

mfuntowicz committed 6 years ago

0e6f131e

Integration of Roberta tokenizer.

mfuntowicz committed 6 years ago

74f98458

Overrided save_vocabulary to forward to tokenizers.

mfuntowicz committed 6 years ago

df42a533

Integration of GPT2 tokenizer.

mfuntowicz committed 6 years ago

e610fad8

Integration of Bert tokenizer.

mfuntowicz committed 6 years ago

aa4d1e96

Laying down the tokenizers library integration into transformers.

mfuntowicz committed 6 years ago

afa8b0f3

Bump tokenizers requirements to latest.

mfuntowicz committed 6 years ago

643bd995

Fix misleading RoBERTa token type ids

LysandreJik committed 6 years ago

dfe012ad

Improve padding side documentation

LysandreJik committed 6 years ago

c024ab98

Improve padding side documentation

LysandreJik committed 6 years ago

9aeb0b9b

Merge branch 'master' into from_scratch_training

julien-c committed 6 years ago

715fa638

Bias should be resized with the weights

LysandreJik committed 6 years ago

100e3b6f

Size > Dimensionality + Remove final TODOs

LysandreJik committed 6 years ago

6c32d8bb

RoBERTa example

LysandreJik committed 6 years ago

760164d6

Added example usage

LysandreJik committed 6 years ago

387217bd

Add missing XLNet and XLM models

LysandreJik committed 6 years ago

7d1bb7f2

Wrap up configurations

LysandreJik committed 6 years ago

a1cb1004

Update links in all configurations

LysandreJik committed 6 years ago

c11b6fd3

Updated Configurations

LysandreJik committed 6 years ago

63268272

Merge pull request #2384 from dimagalat/master

thomwolf committed 6 years ago

Verified 2b566c18

Update test_tokenization_auto.py

julien-c committed 6 years ago

Verified 764f836d

Update test_tokenization_auto.py

julien-c committed 6 years ago

Verified d5831acb

Update test_tokenization_auto.py

julien-c committed 6 years ago

Verified ed6cd597

Update test_tokenization_auto.py

julien-c committed 6 years ago

Verified 5cb463a7

In a parallel setup this could fail

julien-c committed 6 years ago

afc24ea5

Older