Torch2.0 scaled_dot_product_attention processor (#2303)
* add sdpa processor
* don't use it by default
* add some checks and style
* typo
* support torch sdpa in dreambooth example
* use torch attn proc by default when available
* typo
* add attn mask
* fix naming
* being doc
* doc
* Apply suggestions from code review
* polish
* torctree
* Apply suggestions from code review
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* better name
* style
* add benchamrk table
* Update docs/source/en/optimization/torch2.0.mdx
* up
* fix example
* check if processor is None
* Apply suggestions from code review
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
* add fp32 benchmakr
* Apply suggestions from code review
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
