[core] Pyramid Attention Broadcast (#9562)
* start pyramid attention broadcast
* add coauthor
Co-Authored-By: Xuanlei Zhao <43881818+oahzxl@users.noreply.github.com>
* update
* make style
* update
* make style
* add docs
* add tests
* update
* Update docs/source/en/api/pipelines/cogvideox.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/api/pipelines/cogvideox.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Pyramid Attention Broadcast rewrite + introduce hooks (#9826)
* rewrite implementation with hooks
* make style
* update
* merge pyramid-attention-rewrite-2
* make style
* remove changes from latte transformer
* revert docs changes
* better debug message
* add todos for future
* update tests
* make style
* cleanup
* fix
* improve log message; fix latte test
* refactor
* update
* update
* update
* revert changes to tests
* update docs
* update tests
* Apply suggestions from code review
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* update
* fix flux test
* reorder
* refactor
* make fix-copies
* update docs
* fixes
* more fixes
* make style
* update tests
* update code example
* make fix-copies
* refactor based on reviews
* use maybe_free_model_hooks
* CacheMixin
* make style
* update
* add current_timestep property; update docs
* make fix-copies
* update
* improve tests
* try circular import fix
* apply suggestions from review
* address review comments
* Apply suggestions from code review
* refactor hook implementation
* add test suite for hooks
* PAB Refactor (#10667)
* update
* update
* update
---------
Co-authored-by: DN6 <dhruv.nair@gmail.com>
* update
* fix remove hook behaviour
---------
Co-authored-by: Xuanlei Zhao <43881818+oahzxl@users.noreply.github.com>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
Co-authored-by: DN6 <dhruv.nair@gmail.com>
