huggingface/transformers

Pull Requests Commits

Remove unused imports

thomasw21 committed 3 years ago

ba58e5b1

Actually I might need to just override the tie_weights method

thomasw21 committed 3 years ago

fec68bf1

thomasw21 committed 3 years ago

201d5715

Try to handle both post_init and from_pretrained

thomasw21 committed 3 years ago

426858e6

Turns out state_dict returns shared weights multiple times

thomasw21 committed 3 years ago

36beec7c

Turns out weight tie is not done in post_init but in from_pretrained

thomasw21 committed 3 years ago

15635235

thomasw21 committed 3 years ago

40eaf208

Create test to check that fp32 model is the same as the fp16 one with force_lm_head_in_fp32 set to True

thomasw21 committed 3 years ago

d5265d03

thomasw21 committed 3 years ago

457019ce

Change the way I run torch.no_grad

thomasw21 committed 3 years ago

e5216f67

Maybe I can first require torch and then run torch.no_grad

thomasw21 committed 3 years ago

b02d741d

Rename to force_lm_head_in_fp32

thomasw21 committed 3 years ago

92c93b10

Import warnings

thomasw21 committed 3 years ago

23478460

thomasw21 committed 3 years ago

6377a322

This can only run on gpu

thomasw21 committed 3 years ago

0714d6af

thomasw21 committed 3 years ago

f1579ee1

thomasw21 committed 3 years ago

0dca0ffc

thomasw21 committed 3 years ago

19f8e5a7

thomasw21 committed 3 years ago

9e4e4a39

Turns out that matching fp32 is a lot more complicated

thomasw21 committed 3 years ago

83e065d1

thomasw21 committed 3 years ago

eb4a6b9f

We have to use the same pretrained weights for testing multiple precision

thomasw21 committed 3 years ago

c0f9cab6

Turns out dtype does nothing

thomasw21 committed 3 years ago

6126d30e

Remove unecessary files

thomasw21 committed 3 years ago

286ca9cb

Remove unecessary files + add test for word_embeddings_in_fp32

thomasw21 committed 3 years ago

dcb1eebf

thomasw21 committed 3 years ago

0feba3a2

Turns out max-collpase is a real thing

thomasw21 committed 3 years ago

1d5573a4

Turns out scaling before or after changes the results

thomasw21 committed 3 years ago

eb528718

Cast then scaling then softmax then cast

thomasw21 committed 3 years ago

33c430b5

thomasw21 committed 3 years ago

b6a5607c

Older