NielsRogge/transformers

Pull Requests Commits

FastPreTrainedTokenizer => PreTrainedTokenizerFast

n1t0 committed 6 years ago

7ead04ce

Update for changes in tokenizers API

n1t0 committed 6 years ago

1f82a5d9

Merge pull request #2324 from kashif/patch-1

thomwolf committed 6 years ago

Verified 8c67b529

Typo in serving.py

kashif committed 6 years ago

Verified 7211541a

improve comments for examples

patrickvonplaten committed 6 years ago

0f6017be

add example for ctrl text generation in docs

patrickvonplaten committed 6 years ago

87c8fca9

merge conflicts - renamed to previous_token singular

patrickvonplaten committed 6 years ago

88def24c

duplicated line for repeating_words_penalty_for_language_generation

patrickvonplaten committed 6 years ago

822f725a

adapt style to predefined style layout

patrickvonplaten committed 6 years ago

fc84bd52

add prepare inputs for transfo_xl and xlnet

patrickvonplaten committed 6 years ago

deff792b

add easy tensor shape match test

patrickvonplaten committed 6 years ago

9398058e

add past re-ordering for beam search

patrickvonplaten committed 6 years ago

90cda45e

check for self.config.mem_len instead of self.mem_len in _do_output_past

patrickvonplaten committed 6 years ago

6bca56fd

make if statements cleaner for prepare_inputs_for_generation

patrickvonplaten committed 6 years ago

365ccd0a

better naming for if statement

patrickvonplaten committed 6 years ago

d039c679

changed do_output_past function to check for self.config.output_past instead of self.output_past

patrickvonplaten committed 6 years ago

7e0c5c73

rename comments

patrickvonplaten committed 6 years ago

eeaa402c

remove ipdb debugging statements

patrickvonplaten committed 6 years ago

7bb42712

add and improve comments

patrickvonplaten committed 6 years ago

267587c2

add past hidden key states for more efficient language generation & add prepare_inputs for gpt2 and ctrl model

patrickvonplaten committed 6 years ago

d891fd0a

Merge pull request #2303 from patrickvonplaten/fix_error_with_repetition_penalty

thomwolf committed 6 years ago

Verified aeef4823

Merge pull request #2291 from aaugustin/fix-flake8-F841

thomwolf committed 6 years ago

Verified 0412f3d9

Merge pull request #2289 from patrickvonplaten/fix_effective_batch_size_lang_gen_xlm

thomwolf committed 6 years ago

Verified 8742c954

Merge pull request #2312 from vitaliyradchenko/fix_special_and_add_tokens_loading

thomwolf committed 6 years ago

Verified 1240be3e

add special tokens to unique_added_tokens_encoder

vitaliyradchenko committed 6 years ago

b262577d

fixed lack of added and special tokens

vitaliyradchenko committed 6 years ago

83a23479

Merge pull request #2310 from ShnitzelKiller/scatter-unfix

thomwolf committed 6 years ago

Verified cea04a24

use positional arguments due to inconsistent API

ShnitzelKiller committed 6 years ago

e1844d9a

revert erroneous fix

ShnitzelKiller committed 6 years ago

9fb7addd

tokenizers is now a real dependency

n1t0 committed 6 years ago

734d29b0

Newer Older