feat: add JoyImage edit plus (#14032)
* feat: add image edit plus
* refactor: remove debug code
* fix: address review issues for JoyImage Edit Plus
- Remove einops dependency: replace rearrange with reshape/permute
- Remove sglang-specific code from transformer forward
- Remove unused import inspect from transformer
- Fix hardcoded device_type="cuda" to use device.type
- Simplify scheduler sigma math: delegate to retrieve_timesteps
- Remove unused enable_denormalization parameter
- Fix callback latents variable binding
- Fix output_type="pt" to return stacked tensor
- Set return_dict default to True in transformer forward
- Add dummy objects for JoyImageEditPlus classes
- Add transformer and pipeline test files
* fix: add missing newline at end of pipeline_output.py
* fix: add missing newline at end of pipeline_output.py
* doc: add joyimage-edit-plus doc
* refactor: update code format
* refactor: update code formate
* refactor: merge edit-plus conversion script into convert_joyimage_edit_to_diffusers.py
JoyImage Edit and Edit Plus share identical VAE and transformer weight
layouts — only the target model class differs. Consolidate both into a
single script with a --model_type flag (edit | edit_plus) instead of
maintaining two nearly-duplicate files.
* revert: remove unnecessary attention_mask change from transformer_joyimage.py
* Apply style fixes
* fix: remove stale Copied-from annotations that diverged from source
* fix: address CI check failures for edit-plus PR
- Add missing Returns section to JoyImageEditPlusTransformer3DModel.forward docstring
- Fix alphabetical ordering of dummy classes in dummy_pt_objects.py
* fix: use VAE dtype instead of float32 cast in edit-plus pipeline
Replace `torch.autocast(dtype=torch.float32)` + `.float()` with
`.to(self.vae.dtype)` for both VAE encode and decode calls.
The previous approach caused dtype mismatch (float32 input vs bfloat16
bias) on CPU where autocast does not automatically cast conv weights,
breaking the CI `test_layerwise_casting_inference` test.
* refactor: address dg845 review comments on edit-plus
Transformer:
- Simplify RoPE by removing dead txt_freqs path
- Fix patch_size type hint to list[int]
- Use self.config.xxx instead of explicit attributes
Pipeline:
- Merge VaeImageProcessor into JoyImageEditImageProcessor
- Inline normalize/denormalize latents at call sites
- Use image_processor.preprocess() for reference images
- Remove duplicate resize_center_crop in prepare_latents
- Rename padded_latents to latents in __call__
- Support pre-computed latents parameter
- Remove redundant padding token zeroing (attention mask suffices)
- Use image_processor.postprocess() for output conversion
- Return (image,) tuple when return_dict=False
* fix: guard against division-by-zero in CFG norm rescaling
Add .clamp_min(1e-6) to noise_norm to prevent NaN when comb_pred
is all-zeros (e.g. padded patches), matching pipeline_joyimage_edit.py.
* style: run ruff format on edit-plus pipeline
* refactor: address dg845 second-round review comments
- Fix vae_image_processor -> image_processor in docs
- Remove empty __init__ from AttnProcessor
- Restore # Copied from on retrieve_timesteps and sync with source
- Add check_inputs method for basic input validation
- Remove unnecessary test skip for test_from_save_pretrained_dtype_inference
* Apply style fixes
---------
Co-authored-by: tangyanfei.8 <tangyanfei.8@jd.com>
Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
