CLIP (#11445)
* begin second draft
* fix import, style
* add loss
* fix embeds, logits_scale, and projection
* fix imports
* add conversion script
* add feature_extractor and processor
* style
* add tests for tokenizer, extractor and processor
* add vision model tests
* add weight init
* add more tests
* fix save_load test
* model output, dosstrings, causal mask
* config doc
* add clip model tests
* return dict
* bigin integration test
* add integration tests
* fix-copies
* fix init
* Clip => CLIP
* fix module name
* docs
* fix doc
* output_dim => projection_dim
* fix checkpoint names
* remoe fast tokenizer file
* fix conversion script
* fix tests, quality
* put causal mask on device
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* fix attribute test
* style
* address sylvains comments
* style
* fix docstrings
* add qucik_gelu in activations, docstrings
* clean-up attention test
* fix act fun
* fix config
* fix torchscript tests
* even batch_size
* remove comment
* fix ouput tu_tuple
* fix save load tests
* fix add tokens test
* add fast tokenizer
* update copyright
* new processor API
* fix docs
* docstrings
* docs
* fix doc
* fix doc
* fix tokenizer
* fix import in doc example
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* check types of config
* valhalla => openai
* load image using url
* fix test
* typo
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
