NielsRogge/transformers

Pull Requests Commits

Merge branch 'master' into nlp

thomwolf committed 5 years ago

fdcdd4dd

thomwolf committed 5 years ago

05c817a7

thomwolf committed 5 years ago

e6d732e6

Cleanup rebase and fix added tokens serialization/loading

thomwolf committed 5 years ago

b157d13a

WIP adding nlp in GLUE example

thomwolf committed 5 years ago

208d8727

avoid set in unique_added_tokens_encoder for deterministic hashing

thomwolf committed 5 years ago

2ffb7b58

update tests for fast tokenizers + fix small bug in saving/loading

thomwolf committed 5 years ago

8082dfc1

update docstrings

thomwolf committed 5 years ago

30a3972f

thomwolf committed 5 years ago

04fe7318

Various cleans up - add verbose

thomwolf committed 5 years ago

81bc6da1

style and quality

thomwolf committed 5 years ago

625e334e

fix backward compatibility for max_length only

thomwolf committed 5 years ago

57fb0c54

fix QA pipeline

thomwolf committed 5 years ago

80f9f4bc

style and quality - tests on pad

thomwolf committed 5 years ago

beeee447

updating tests for custom tokenizers

thomwolf committed 5 years ago

de72cb31

switched padding/truncation API - simpler better backward compat

thomwolf committed 5 years ago

db50c222

style and quality

thomwolf committed 5 years ago

4d1647f1

[HUGE] refactoring tokenizers - padding - truncation - tests

thomwolf committed 5 years ago

dee99337

tokenizers clean up - new padding_strategy - split the files

thomwolf committed 5 years ago

2b694f73

failing pretrokenized test

thomwolf committed 5 years ago

e3896ba2

Add reference to NLP dataset (#5028)

mrm8488 committed 5 years ago

Verified 0c55a384

Add reference to NLP (package) dataset (#5029)

mrm8488 committed 5 years ago

Verified 0946d120

refactor(wandb): consolidate import (#5044)

borisdayma committed 5 years ago

Verified edcb3ac5

Ability to pickle/unpickle BatchEncoding pickle (reimport) (#5039)

mfuntowicz committed 5 years ago

Verified 9e033649

Add DistilBertForMultipleChoice (#5032)

sgugger committed 5 years ago

Verified f9f8a531

[HUGE] Refactoring tokenizers backend - padding - truncation - pre-tokenized pipeline - fast tokenizers - tests (#4510)

n1t0 committed 5 years ago

Verified 36434220

[Bart] Question Answering Model is added to tests (#5024)

patrickvonplaten committed 5 years ago

Verified ebba39e4

Add position_ids (#5021)

sgugger committed 5 years ago

Verified bbad4c69

feat(TFTrainer): improve logging (#4946)

borisdayma committed 5 years ago

Verified 1bf4098e

Fix importing transformers on Windows (#4997)

mfuntowicz committed 5 years ago

Verified 7b5a1e7d

Older