Add Gated-SiLU to T5 (#17420)

Commit

3 years ago

Add Gated-SiLU to T5 (#17420) * Add gated-silu to t5 architecture to support UL2 * Fix error message * formatting * formatting again * refactor * fix classnames in _init_weights * remove is_gated * add test * fix test * Try without the test? * Add back the test. * Improve error message. Co-authored-by: Daniel Hesslow <daniel@lighton.ai>

References

#17420 - Add Gated-SiLU to T5

#19449 - [WIP] Fix weights initialization of several vision models

#27720 - Add common processor tests

#29969 - [SigLIP] Add fast tokenizer

#32831 - [Docs] Update resources

#33111 - [Backbone] Remove out_features everywhere

#33174 - [Zero-shot image classification pipeline] Remove tokenizer_kwargs

#39821 - Support MetaCLIP 2

#58 - Add EoMT DINOv3 model

#59 - Fix attention mask handling in EoMT-DINOv3 converter

#41212 - Add EoMT with DINOv3 backbone