diffusers
fix wan i2v pipeline bugs
#10975
Merged
Overview
Commits
6
Changes
View On GitHub

fix wan i2v pipeline bugs #10975

yiyixuxu merged 6 commits into huggingface:main from yupeng1111:fix_wan_i2v
yupeng1111
yupeng111157 days ago

I fix some bug in wan i2v pipeline and add a new example

yupeng1111 fix wan i2v pipeline bugs
465e12e5
yupeng1111 update t2v example
46702877
yiyixuxu
yiyixuxu approved these changes on 2025-03-06
yiyixuxu57 days ago

thanks!

yiyixuxu
yiyixuxu57 days ago

@bot /style

yiyixuxu yiyixuxu requested a review from a-r-r-o-w a-r-r-o-w 57 days ago
github-actions[bot] Apply style fixes
efbdd344
github-actions
github-actions57 days ago

Style fixes have been applied. View the workflow run here.

yiyixuxu Merge branch 'main' into fix_wan_i2v
a40b1277
HuggingFaceDocBuilderDev
HuggingFaceDocBuilderDev57 days ago

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

yiyixuxu
yiyixuxu commented on 2025-03-06
Conversation is marked as resolved
Show resolved
src/diffusers/pipelines/wan/pipeline_wan.py
60 ... flow_shift=flow_shift,
61 ... )
5462 >>> pipe = WanPipeline.from_pretrained(model_id, vae=vae, torch_dtype=torch.bfloat16)
63
 >>> pipe.scheduler = scheduler
yiyixuxu57 days ago

I think it's easier this way

Suggested change
>>> pipe.scheduler = scheduler
>>> flow_shift = 5.0 # 5.0 for 720P, 3.0 for 480P
>>> pipe.scheduler = UniPCMultistepScheduler.from_config(pipe.scheduler.config, flow_shift=flow_shift)
Conversation is marked as resolved
Show resolved
src/diffusers/pipelines/wan/pipeline_wan.py
5152 >>> # Available models: Wan-AI/Wan2.1-T2V-14B-Diffusers, Wan-AI/Wan2.1-T2V-1.3B-Diffusers
5253 >>> model_id = "Wan-AI/Wan2.1-T2V-14B-Diffusers"
5354 >>> vae = AutoencoderKLWan.from_pretrained(model_id, subfolder="vae", torch_dtype=torch.float32)
55
 >>> flow_shift = 5.0 # 5.0 for 720P, 3.0 for 480P
56
 >>> scheduler = UniPCMultistepScheduler(
57
 ... prediction_type="flow_prediction",
58
 ... use_flow_sigmas=True,
59
 ... num_train_timesteps=1000,
60
 ... flow_shift=flow_shift,
61
 ... )
yiyixuxu57 days ago
Suggested change
>>> flow_shift = 5.0 # 5.0 for 720P, 3.0 for 480P
>>> scheduler = UniPCMultistepScheduler(
... prediction_type="flow_prediction",
... use_flow_sigmas=True,
... num_train_timesteps=1000,
... flow_shift=flow_shift,
... )
yiyixuxu yiyixuxu added close-to-merge
a-r-r-o-w
a-r-r-o-w approved these changes on 2025-03-06
a-r-r-o-w56 days ago

Thank you!

yupeng1111 Merge branch 'main' into fix_wan_i2v
86f94d6e
shethaadit
shethaadit approved these changes on 2025-03-07
yiyixuxu Apply suggestions from code review
9bfacc4f
yiyixuxu yiyixuxu merged d55f4110 into main 56 days ago
yiyixuxu yiyixuxu removed close-to-merge
yiyixuxu
yiyixuxu commented on 2025-03-07
src/diffusers/pipelines/wan/pipeline_wan_i2v.py
205223 image = self.image_processor(images=image, return_tensors="pt").to(self.device)
206224 image_embeds = self.image_encoder(**image, output_hidden_states=True)
207 return image_embeds.hidden_states[-1]
225
 return image_embeds.hidden_states[-2]
yiyixuxu55 days ago

@a-r-r-o-w the change of output is probably from here

yiyixuxu
yiyixuxu commented on 2025-03-12
src/diffusers/pipelines/wan/pipeline_wan_i2v.py
138156 tokenizer: AutoTokenizer,
139157 text_encoder: UMT5EncoderModel,
140 image_encoder: CLIPVisionModelWithProjection,
158
 image_encoder: CLIPVisionModel,
yiyixuxu50 days ago

need to match with here https://huggingface.co/Wan-AI/Wan2.1-I2V-14B-480P-Diffusers/blob/main/model_index.json#L6

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum