diffusers
[Flux with CFG] add flux pipeline with cfg support
#9445
Merged
Overview
Commits
8
Changes
View On GitHub

[Flux with CFG] add flux pipeline with cfg support #9445

yiyixuxu merged 8 commits into huggingface:main from linoytsaban:cfg-flux-exploring
linoytsaban
linoytsaban244 days ago (edited 240 days ago)❤ 4

In the recent PuLID Flux adaptation, authors (ToTheBeginning Yanze Wu, guozinan126, guozinan126) also introduce true CFG - a simple & straight forward way to support negative prompts and "real" CFG for the distilled flux models.

Based on their idea and code, this PR adds a flux pipeline supporting negative prompts.
Inference can be done as follows -

import torch
import diffusers
from pipeline_flux_with_cfg import FluxCFGPipeline

pipe = FluxCFGPipeline.from_pretrained("black-forest-labs/FLUX.1-dev", 
                                    torch_dtype=torch.bfloat16)
pipe.enable_model_cpu_offload()

prompt = "a watercolor painting of a unicorn"
negative_prompt = "pink"


#### without true cfg
torch.manual_seed(0)
img_1 = pipe(prompt).images[0].resize((512,512))


#### with true cfg
torch.manual_seed(0)
img_2 = pipe(prompt=prompt, negative_prompt=negative_prompt, true_cfg=1.5, guidance_scale=3.5).images[0].resize((512,512))

output examples courtesy of @apolinario 🤗:

Group 1-37

linoytsaban true_cfg
b5b50e5b
linoytsaban add check negative prompt/embeds inputs
6312118d
linoytsaban move to community pipelines
78d85594
linoytsaban move to community pipelines
4bc7332a
linoytsaban revert true cfg changes to the orig pipline
a9618eba
linoytsaban style
34a8cced
yiyixuxu Merge branch 'main' into cfg-flux-exploring
c052657f
yiyixuxu
yiyixuxu approved these changes on 2024-09-16
yiyixuxu243 days ago❤ 1

thanks!

yiyixuxu Merge branch 'main' into cfg-flux-exploring
355cbf3c
yiyixuxu yiyixuxu merged 8fcfb2a4 into main 243 days ago
linoytsaban linoytsaban deleted the cfg-flux-exploring branch 243 days ago
arnold408
arnold408239 days ago (edited 239 days ago)

Hi,
Will this work with Flux.1-schnell ? What should be the "guidance_scale" ?
Thanks

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum