Add LWDetr model (#40991)
* feat: add LWDetr model
* fix: changed LwDetrVit base classes from VitDet to ViT
* tests: added tests for LWDetr
* refactor: fix all issues and created docs
* tests: added missing lw_detr_vit tests
* docs: add lwdetr docs
* fix: fixed implementation error and associated tests
* chore: removed testing lib in imports
* refactor: replace LwDetrImageProcessor with DeformableDetrImageProcessor
* refactor: remove two-stage detection and bounding box reparameterization parameters from LwDetrConfig
* refactor: rename LwDetrCSPRepLayer to LwDetrC2FLayer
* refactor: introduce LwDetrMLP for feedforward layers in decoder
* refactor: replace build_position_encoding with LwDetrSinePositionEmbedding
* refactor: remove use_cae parameter and related logic from configuration and modeling files in LwDetrVit
* refactor: remove unused variables and simplify certain instructions
* refactor: removed unnecessary one line instruction method with_pos_embed
* refactor: use llama attention formatting for hidden shape
* docs: add comments about group detr
* fix: removed wrong sigmoid and fixed init for class_embed
* refactor: removed unused positional embeddings classes and weights from backbone
* chore: removed unused import
* chore: make style and repo-consistency after positional embeddings removal
* refactor: removed unused drop path rate
* fix: ingest latest changes from rebase
* fix: attn_implementation setter
* fix: is causal set to False
* refactor: renamed ffn to mlp and moved layer norm out of mlp
* fix: check model inputs
* fix: moved super init call in LwDetrConfig
* fix: super class in GradientCheckpointingLayer
* fix: replaced RTDetr occurences by LwDetr in test modeling file
* refactor: removed head_mask from LwDetrViT
* docs: added release date in docs
* fix: added missing attention mask argument
* chore: make style & repo-consistency
* fix: ensure tensor dtype consistency in loss calculations and test cases
* docs: fixed model release date
* refactor: removed unnecessary module cloning
* tests: added missing _prepare_for_class method and removed batching_equivalence overwrite
* tests: added xlarge integration test
* chore: added lw_detr reference in image processing auto
* chore: removed unnecessary properties from LwDetrConfig
* fix: fix for latest main changes
* fix: apply modular changes from mail
* docs: update model doc and docstrings
* fix: style
* fix: update output values in convert script
* feat: added proper last_hidden_states in LwDetrDecoderOutput and separated logits and pred_boxes from outputs_class and outputs_coord
* fix: guard accelerate imports
* fix: removed LWDetrConfig attribute map and changed LwDetrAttention init to reflect
* fix: parameterize amap based on config
* fix: remove redundant decorator
* chore: moved LwDetrViT to LwDetr single modular file
* fix: remove unnecessary attribute_map in LwDetrViT
* chore: simplified LwDetr modules methods with proper hidden_states return tuple
* fix: replaced hardcoded value by variable
* tests: added VitDet and attention tests
* fix: modular conversion
* tests: moved LwDetrViT tests to test_modeling_lw_detr file
* docs: add lwdetr advances in docs
* refactor: removed arguments to classes as much as possible and rely on config
* reapply style, remove LlamaAttention inheritance to remove decorator
* chore: updated licence and year
* fix: removed torch.nn.functional from modular
* docs: removed redundant docstring arguments covered by autodocstring decorator
* refactor: removed backbone api statements
* fix: added back num_key_value_groups in LwDetrAttention
* chore: removed unnecessary copied from statement
* chore: moved LwDetrViT modules above LwDetr modules
* tests: removed unnecessary overwrite and “test_” attributes
* docs: added missing docs
* style: remove unnecessary parentheses
* docs: added back logits docstring
* docs: added docs dates
* style details
* unessecary utf8
* might as well skip all config checks
* embeddings are large, increase model_split_percents
* fix device issue
* update logits
* set device in expectations
* add to toctree
---------
Co-authored-by: Cyril Vallez <cyril.vallez@huggingface.co>
Co-authored-by: Cyril Vallez <cyril.vallez@gmail.com>
