transformers
9270ab08 - [`Flash Attention 2`] Add flash attention 2 for GPT-Neo-X (#26463)

Commit

2 years ago

[`Flash Attention 2`] Add flash attention 2 for GPT-Neo-X (#26463) * add flash-attn-2 support for GPT-neo-x * fixup * add comment * revert * fixes * update docs * comment * again * fix copies * add plot + fix copies * Update docs/source/en/model_doc/gpt_neox.md

References

#32831 - [Docs] Update resources

#29969 - [SigLIP] Add fast tokenizer

#41212 - Add EoMT with DINOv3 backbone

#39821 - Support MetaCLIP 2

#33111 - [Backbone] Remove out_features everywhere

#33174 - [Zero-shot image classification pipeline] Remove tokenizer_kwargs

#62 - Add initial DEIMv2 model implementation

#58 - Add EoMT DINOv3 model

#59 - Fix attention mask handling in EoMT-DINOv3 converter

#26463 - [`Flash Attention 2`] Add flash attention 2 for GPT-Neo-X

Author

younesbelkada

Parents

87714b3d

transformers 9270ab08 - [`Flash Attention 2`] Add flash attention 2 for GPT-Neo-X (#26463)

transformers
9270ab08 - [`Flash Attention 2`] Add flash attention 2 for GPT-Neo-X (#26463)