[`Community Pipeline`] Add 🪆Matryoshka Diffusion Models #9157
yiyixuxu
merged
137 commits into
huggingface:main
from
tolgacangoz:Add-Matryoshka-Diffusion-Models
a
f666908a
tolgacangoz
changed the title
Merge branch 'main' into Add-Matryoshka-Diffusion-Models
c0927285
Merge branch 'main' into Add-Matryoshka-Diffusion-Models
cfe8dcc7
Merge branch 'main' into Add-Matryoshka-Diffusion-Models
c1b6c0f7
Merge branch 'main' into Add-Matryoshka-Diffusion-Models
60021b8b
Merge branch 'main' into Add-Matryoshka-Diffusion-Models
0ad71019
refactor: add `ff_act_fn` parameter to `UNet2DConditionModel` and `ge…
aabac0a2
Merge branch 'main' into Add-Matryoshka-Diffusion-Models
ad4c6a33
Merge branch 'main' into Add-Matryoshka-Diffusion-Models
649baa63
Study as an ordinary UNet model
279d6138
make style
5f5bd081
make fix-copies
bfd8b9dc
Merge branch 'main' into Add-Matryoshka-Diffusion-Models
c3b004b1
Up
eaef0377
Merge branch 'main' into Add-Matryoshka-Diffusion-Models
2e99ec7e
Up
99d9099a
tolgacangoz
changed the title
Merge branch 'main' into Add-Matryoshka-Diffusion-Models
33dd50de
Fix timestep embedding conditioning in `MatryoshkaCombinedTimestepTex…
56e61f09
make style
376500ab
Merge branch 'main' into Add-Matryoshka-Diffusion-Models
4c16e5bb
Revert; cuz I should have created (probably) a new attention processo…
8c4dcb39
Revert to create your own custom transformer block
ef38541f
Merge branch 'main' into Add-Matryoshka-Diffusion-Models
8eadb301
Init template for the pipeline
19d6c178
Add `MatryoshkaTransformerBlock` and `MatryoshkaFeedForward` classes
7d1a0ab3
Revert
5754bc66
Add `GELU` activation function module
bc1f68ba
Merge branch 'main' into Add-Matryoshka-Diffusion-Models
906298bf
Revert
23f4ced0
Revert
a2ca8eff
make fix-copies
bcd89399
All in one file
e014e3e0
Up
f264b9f3
Replace `MatryoshkaTransformerBlock` with `MatryoshkaTransformer2DModel`
1a40f688
make style
221c9541
Refactor `MatryoshkaTransformer2DModel` to add `forward()`and add `Ma…
c75e7237
make style
e5db6e31
Up
728fb421
Merge branch 'main' into Add-Matryoshka-Diffusion-Models
5b5747d2
Remove redundant attention projections in `MatryoshkaTransformerBlock`
464600dc
Up
36d9d295
Merge branch 'main' into Add-Matryoshka-Diffusion-Models
b0bf23f9
Fix shape issue
9e37e00c
Up
b5731821
make style
f35a8f9f
Up
0f6bce5b
Refactor condition embedding in `MatryoshkaCombinedTimestepTextEmbedd…
1d48420d
Adapt `DDIMScheduler` for `x_0` prediction by exploiting `gammas`
b476da95
Merge branch 'main' into Add-Matryoshka-Diffusion-Models
a3fc84da
Fix `prev_timestep` index
6a978b26
Up
368e044a
Merge branch 'main' into Add-Matryoshka-Diffusion-Models
02e67c33
Fix normalization group size in `MatryoshkaTransformerBlock`
a146ae4f
Refactor class names
abbb3d4d
Add `NestedUNet2DConditionModel` template
a09266e5
Adapt `NestedUNet2DConditionModel` initialization and configuration
85241b35
make style
b7df3bb4
Add template of `forward` for `NestedUNet2DConditionModel`
22c148f1
Refactor `NestedUNet2DConditionModel` forward method
651cd768
Refactor `NestedUNet2DConditionModel` forward method
ea60da33
Merge branch 'main' into Add-Matryoshka-Diffusion-Models
8b29e7dd
Fix `NestedUNet2DConditionModel` initialization
e01421cd
Up
4d06f295
Generalize `MatryoshkaCombinedTimestepTextEmbedding` for nesting leve…
29fa2572
make style
1a22767c
Up
d0fa5ca5
Generalize time projection for different model types in `MatryoshkaUN…
6b65f9fc
Fix `cond_emb` usage
62db4b0d
Up
a57b5fcd
style
db809dc9
`Up`
ff301b61
style
77732bbc
Merge branch 'main' into Add-Matryoshka-Diffusion-Models
577875ab
Refactor `NestedUNet2DConditionModel` to handle `sample_low` conditio…
154c1be7
Simplify
b363cc17
Refactor `MatryoshkaDDIMScheduler` to use `alpha_prod` instead of `ga…
028a685d
Refactor `MatryoshkaDDIMScheduler` to remove unused import and simpli…
a5b3c37c
Refactor `NestedUNet2DConditionModel` to handle `inner_config` condit…
30c6881a
Refactor `_set_time_proj` to handle with `micro_conditioning_scale` c…
e34fb48b
Generalize for `nesting_level=2`
dd88c374
Merge branch 'main' into Add-Matryoshka-Diffusion-Models
02cec3f6
Refactor `MatryoshkaUNet2DConditionModel` and `NestedUNet2DConditionM…
a7c7c9ab
Cleansing
0144e140
Clean up the `NestedUNet2DConditionModel` constructor
5e2e9395
No need for VAE
5fbba0e2
Up
29345704
style
f2f2f9c0
Merge branch 'main' into Add-Matryoshka-Diffusion-Models
9d85d0df
Refactor `MatryoshkaUNet2DConditionModel` and `MatryoshkaPipeline`
6ed3d633
Remove safety checker
1df22a64
style
a4be940d
Refactor 'NestedUNet2DConditionModel' to add 'sample_size' parameter …
ba39b8d2
Up
319a4d63
Refactor 'MatryoshkaPipeline' to process multiple images for nesting_…
95a293c0
revert the last
b54d9ef7
Refactor 'MatryoshkaDDIMScheduler' to handle multiple model outputs f…
c9f17bb7
Refactor 'MatryoshkaPipeline' to remove unused 'model_type' property
67a2917c
Fix masking
60e9e773
style
261f135f
Merge branch 'main' into Add-Matryoshka-Diffusion-Models
aebb2660
Fix and improve mask handling
25b56a87
style
38c5455d
Merge branch 'main' into Add-Matryoshka-Diffusion-Models
d13081e5
style
3ada184c
Fix mask handling
0efa0aec
Refactor attention mask handling in Matryoshka models
5737d95c
Refactor attention mask handling in Matryoshka models
b691f16e
Fix mask handling for `nesting_level=2`
ccdee358
Attempt for scheduler usage generalization
2efe7b0f
Equalize tokenizer usage fully
5c90be9d
style
31c73fa6
Up
a21e1104
Refactor `matryoshka.py` to include proper licensing and attribution
bc073fcf
Refactor `matryoshka.py` to remove deprecated `_encode_prompt()` method
33edbdd0
Refactor `matryoshka.py` to include nesting levels for the UNet model
96a788cb
Up
d4f29111
Fix scaling issue for high resolutions
3bc6f80f
Add `self.change_nesting_level(int)` function
e4259acc
Refactor `matryoshka.py` to handle multiple model outputs in `Matryos…
942c54af
This model uses this.
737bca02
Move `extra_step_kwargs`
aabba080
style
360f57ed
Refactor optional components in `MatryoshkaPipeline`
83262f83
Simplify
e5433792
Add 🪆Matryoshka Diffusion Models to community pipelines in `Readme.md`
bd915850
Update example usage
34303458
Refactor `MatryoshkaTransformerBlock` to use `MatryoshkaFusedAttnProc…
149e8b59
style
ecca7e33
simplify
6fd62e09
Merge branch 'main' into Add-Matryoshka-Diffusion-Models
c90993d1
Add `trust_remote_code=True` requirement for custom components
5009be12
Merge branch 'main' into Add-Matryoshka-Diffusion-Models
29eceb7a
revert
4c3ba487
Merge branch 'Add-Matryoshka-Diffusion-Models' of github.com:tolgacan…
8f0e8883
Update README.md
1b756d18
Merge branch 'main' into Add-Matryoshka-Diffusion-Models
787b1a57
Merge branch 'main' into Add-Matryoshka-Diffusion-Models
d197cc11
yiyixuxu
approved these changes
on 2024-10-14
Login to write a write a comment.
Login via GitHub