transformers
[Perceiver] Add calculation of decoders/preprocessors output channels
#26
Open
Overview
Commits
99
Changes
View On GitHub

[Perceiver] Add calculation of decoders/preprocessors output channels #26

esceptico wants to merge 99 commits into NielsRogge:modeling_perceiver from esceptico:multimodal_shapes
esceptico
NielsRogge First draft
fc77bd68
NielsRogge Style and remove mlm
177ad35a
NielsRogge Make forward pass work
c9b0064e
NielsRogge More improvements
df5be95f
NielsRogge More improvements
66a42eea
NielsRogge Fix bug
4d25043f
NielsRogge More improvements
381e0d18
NielsRogge More improvements
77bf7954
NielsRogge Add PerceiverTokenizer first draft
697af73d
NielsRogge Improve conversion script
b98b1dfa
NielsRogge More improvements
fef92083
NielsRogge Make conversion script work for the encoder
2e6055a7
NielsRogge Make conversion script work with local pickle files
0a100053
NielsRogge Style & quality, fix-copies
3b7e9ecf
NielsRogge Add dummy input to conversion script
25de5e02
NielsRogge Add absolute position embeddings to TextPreProcessor
b223379d
NielsRogge Make forward pass of encoder work
6d379155
NielsRogge More improvements
ebb49a16
NielsRogge Move text preprocessor to separate script
43417c25
NielsRogge More improvements
d429cfd2
NielsRogge More improvements
4188df26
NielsRogge Add post processor
b15241da
NielsRogge Make MLM model work
5179d575
NielsRogge Style
87f24c87
NielsRogge Add PerceiverForMaskedLM
db9ff200
NielsRogge Add PerceiverImagePreprocessor
ab0981bd
NielsRogge Make style
0ccdea9f
NielsRogge Make PerceiverForImageClassification work
06850888
NielsRogge More improvements
089f9a0e
NielsRogge More improvements
e32cde75
NielsRogge Use tokenizer in conversion script
a49bf661
NielsRogge Use PerceiverForMaskedLM in conversion script
6ab614c9
NielsRogge Define custom PerceiverModelOutput
a5e83e73
NielsRogge Improve PerceiverAttention to make it work for both MLM and image cla…
6a0b8ffe
NielsRogge More improvements
7557b090
NielsRogge More improvements
18524162
NielsRogge More improvements to the conversion script
113f0efa
NielsRogge Make conversion script work for both MLM and image classification
0d5666a9
NielsRogge Add PerceiverFeatureExtractor
ef9dcdfb
NielsRogge More improvements
70fabe19
NielsRogge Style and quality
24014aaf
NielsRogge Add center cropping
50ea7703
NielsRogge Fix bug
9f8d2ada
NielsRogge Small fix
758b9128
NielsRogge Add print statement
f04a1d35
NielsRogge Fix bug in image preprocessor
59260bf0
NielsRogge Fix bug with conversion script
f9c65e9f
NielsRogge Make output position embeddings an nn.Parameter layer instead of nn.E…
7269ce0b
NielsRogge Comment out print statements
436e467a
NielsRogge Add position encoding classes
a0881ca7
NielsRogge More improvements
b63ffb5a
NielsRogge Use position_encoding_kwargs
3eb3c11a
NielsRogge Add PerceiverForImageClassificationFourier
505d6cdf
NielsRogge Make style & quality
4b08c193
NielsRogge Add PerceiverForImageClassificationConvProcessing
e89774ea
NielsRogge Style & quality
18670938
NielsRogge Add flow model
df1b14be
NielsRogge Move processors to modeling file
7399cc8d
NielsRogge Make position encodings modular
9f188855
NielsRogge Make basic decoder use modular position encodings
ddf0e01d
NielsRogge Add PerceiverForOpticalFlow to conversion script
4012143d
NielsRogge Add AudioPreprocessor
64099ac2
NielsRogge Make it possible for the basic decoder to use Fourier position embedd…
56636219
NielsRogge Add PerceiverForMultimodalAutoencoding
3a665497
NielsRogge Improve model for optical flow
0cf0c46f
NielsRogge Improve _build_network_inputs method
03bd7e1f
NielsRogge Add print statement
4489c1e9
NielsRogge Fix device issue
f98a69d7
NielsRogge Fix device of Fourier embeddings
b0fbc00c
NielsRogge Add print statements for debugging
87457b86
NielsRogge Add another print statement
99befc51
NielsRogge Add another print statement
ce41b86d
NielsRogge Add another print statement
371fb007
NielsRogge Add another print statement
9cfb3361
NielsRogge Improve PerceiverAudioPreprocessor
1c2fb11d
NielsRogge Improve conversion script for multimodal modal
3be52ae7
NielsRogge More improvements
47f23d9c
NielsRogge More improvements
1cbb13b0
NielsRogge Improve multimodal model
ffa25df6
NielsRogge Make forward pass multimodal model work
4f0404b4
NielsRogge More improvements
57da1cc6
NielsRogge Improve tests
a292f218
NielsRogge Fix some more tests
8aa31448
NielsRogge Add output dataclasses
6ffdf396
NielsRogge Make more tests pass
0c9b51b8
NielsRogge Add print statements for debuggin
2e1af165
NielsRogge Add tests for image classification
a77d9fd5
NielsRogge Add PerceiverClassifierOutput
49ab6082
NielsRogge More improvements
96fe9d08
NielsRogge Make more tests pass for the optical flow model
82f9f453
NielsRogge Make style & quality
9543cc49
NielsRogge Small improvements
d6f928c8
NielsRogge Don't support training for optical flow model for now
a6902668
NielsRogge Fix _prepare_for_class for tests
85079ec2
NielsRogge Make more tests pass, add some docs
4f8cb339
NielsRogge Add multimodal model to tests
dd6c5ce7
esceptico Added output_size calculation for preprocessors
87cd3a9b
esceptico Fix tensor shape in OneHotPreprocessor test
1fbb1e92
esceptico Added num_query_channels property for decoders
7e89ecd7
NielsRogge NielsRogge force-pushed the modeling_perceiver branch from dd6c5ce7 to 56fe5c0a 4 years ago

Login to write a write a comment.

Login via GitHub

Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum