diffusers
[WIP]Add Wan2.2 Animate Pipeline (Continuation of #12442 by tolgacangoz)
#12526
Merged
Merged
[WIP]Add Wan2.2 Animate Pipeline (Continuation of #12442 by tolgacangoz) #12526
yiyixuxu
merged
98 commits into
huggingface:main
from
dg845:add-wan2.2-animate-pipeline
template1
3529a0ad
temp2
4f2ee5e7
up
778fb54f
up
d77b6baf
fix-copies
2fc6ac26
Add support for Wan2.2-Animate-14B model in convert_wan_to_diffusers.py
d667d03a
style
6182d44f
Refactor WanAnimate model components
8c9fd890
Enhance `WanAnimatePipeline` with new parameters for mode and tempora…
d01e9419
Update `WanAnimatePipeline` to require additional video inputs and im…
7af953b2
Add Wan 2.2 Animate 14B model support and introduce Wan-Animate frame…
a0372e36
Add unit test template for `WanAnimatePipeline` functionality
05a01c66
Add unit tests for `WanAnimateTransformer3DModel` in GGUF format
22b83ce8
style
7fb67322
Improve the template of `transformer_wan_animate.py`
3e6f893c
Update `WanAnimatePipeline`
624a3148
style
fc0edb59
Refactor test for `WanAnimatePipeline` to include new input structure
eb7eeddd
from `einops` to `torch`
8968b429
Merge branch 'main' into integrations/wan2.2-animate
dce83a8e
Add padding functionality to `WanAnimatePipeline` for video frames
75b2382d
style
802896e9
Enhance `WanAnimatePipeline` with additional input parameters for imp…
e06098f8
up
84768f6d
Refactor `WanAnimatePipeline` for improved tensor handling and mask g…
06e61380
Refactor `WanAnimatePipeline` to streamline latent tensor processing …
5777ce04
style
b8337c69
Add new layers and functions to `transformer_wan_animate.py` for enha…
f4eb9a05
Merge branch 'main' into integrations/wan2.2-animate
4e6651b9
Refactor `transformer_wan_animate.py` to improve modularity and type …
d80ae195
Refactor `transformer_wan_animate.py` to enhance modularity and updat…
348a9450
Update the `ConvLayer` class to conditionally apply bias based on act…
7774421c
Simplify
a5536e2f
refactor transformer
6a8662d6
Enhance `convert_wan_to_diffusers.py` for Animate model integration
96a126ad
Merge branch 'main' into integrations/wan2.2-animate
050b3137
Enhance `convert_wan_to_diffusers.py` and `WanAnimatePipeline` for im…
0566e5df
simplify
fe02c25c
Refactor `WanAnimatePipeline` to enhance reference image handling and…
04ab2623
Enhance weight conversion logic in `convert_wan_to_diffusers.py`
7bfbd935
Enhance documentation and tests for WanAnimatePipeline, adding exampl…
7092a28d
Merge branch 'main' into integrations/wan2.2-animate
5d01574d
Clarify contribution of M. Tolga Cangöz
9c0a65d8
Update face_embedder key mappings in `convert_wan_to_diffusers.py`
28ac516f
up
b71d3a9a
up
5818d71c
Fix image embedding extraction in WanAnimatePipeline to return the la…
bfda25dd
Adjust default parameters in WanAnimatePipeline for num_frames, num_i…
0ac259c6
Update example docstring parameters for num_frames and guidance_scale…
e2e95ede
Refactor tests in WanAnimatePipeline: remove redundant assertions and…
7146bb05
Add fused relu for Wan animate activations
6ffdb993
tolgacangoz
commented
on 2025-10-22
yiyixuxu
changed the title
Refactor motion encoder to use custom Conv2d and Linear with weight s…
4556730c
Refactor WanAnimateFaceEncoder to make it easier to understand
c3e69fc2
Refactor Wan Animate transformer to reuse WanTimeTextImageEmbedding
7f4dde9d
Refactor Wan Animate face blocks to use an attention processor
4f204ec5
Refactor Wan Animate transformer, taking into account previous changes
57e9ea30
Remove unused imports in transformer_wan_animate
091b7ce2
Merge branch 'main' into add-wan2.2-animate-pipeline
8216aef3
Add initial Wan Animate transformer tests
50329d73
Refactor face block attn into its own Attention class and fix some bugs
275d324f
Fix issues (such as device placement issues) to get remaining transfo…
ac2962db
Update Wan Animate conversion script to reflect changes to transformer
0145135f
Add _repeated_blocks to Wan Animate transformer for regional compilation
bdbd1419
Refactor Wan Animate pipeline to make latent preparation code more clear
25371339
Update Wan Animate pipeline tests after transformer an pipeline changes
332d3c2c
Merge branch 'main' into add-wan2.2-animate-pipeline
99e56e33
Fix some batching and device placement issues in Wan Animate pipeline
00ddbb98
Remove reference_images tests for Wan Animate
1e1e7069
Get Wan Animate pipeline fp16 inference tests working
a56bee14
Skip test_callback_inputs since the Wan Animate pipline is not compat…
6fb5ca8d
Fix mask video shapes for Wan Animate replacement
1e61ed7a
Use a separate VaeImageProcessor for the reference image as it uses d…
e2846f6d
Fix some more Wan Animate pipeline shape errors
3a80241a
Fix more bugs in Wan Animate pipeline
86be6005
Ensure that the replacement mask only has one channel
6748d25e
Support Wan Animate image preprocessing, fix bugs, clean up code
f696682d
Add docs for WanAnimateTransformer3DModel
80d9f8bf
make style and make quality
d9c6bc68
Merge branch 'main' into add-wan2.2-animate-pipeline
4e415d3d
Fix first segment I2V mask for prev segement cond latents
cbfc0ad4
Use same Open CLIP checkpoint as other Wan2.1-based models
b80be86c
Merge branch 'main' into add-wan2.2-animate-pipeline
d87baa50
yiyixuxu
commented
on 2025-11-11
Copy Wan blocks for Wan Animate with # Copied from
6420f0e7
Get regional compilation working without recompilation
dd680ee6
Remove Wan2.2 TI2V timestep logic as Wan Animate is based on Wan 2.1
6d92b3e8
Move motion encoder batch inference logic to forward and remove the m…
d0c77503
Move (de)standardize latents logic into Wan Animate pipeline __call__
c2ec703f
Move Wan Animate ref image processing logic to its own VaeImageProces…
2f549ee6
make style and make quality
68da86a9
Make motion encoder inference batch size configurable from Wan Animat…
cb7977ec
Avoid list comprehension for batched motion encoder inference as it u…
847e4a2d
Address more review comments
e4b1db00
Merge branch 'main' into add-wan2.2-animate-pipeline
f0a0d21d
make style, make quality, make fix-copies
e96f6385
Make motion_encode_batch_size configurable in pipeline __call__
a6ddd02d
Merge branch 'main' into add-wan2.2-animate-pipeline
6ad82e51
Update Wan Animate pipeline example
e74373b7
Have Wan image processor take into account the spatial patch size as …
2259ded8
Login to write a write a comment.
Login via GitHub