Add ace step lora support (#14193)
* feat: add LoRA support to AceStepPipeline
Add standard diffusers LoRA API (load_lora_weights, set_adapters,
fuse_lora, save_lora_weights) for AceStepPipeline, targeting the
transformer (AceStepTransformer1DModel) only.
Includes automatic conversion of ACE-Step-1.5 original PEFT format
LoRA weights (q_proj/k_proj/v_proj/o_proj naming) to diffusers format
(to_q/to_k/to_v/to_out.0 naming) during loading.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use HuggingFace Hub ID for tokenizer in AceStep LoRA test and apply ruff formatting
- Replace hardcoded local path with "Qwen/Qwen3-Embedding-0.6B" so tests
run on CI
- Apply ruff formatting fix (slice spacing)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: override test_lora_fuse_nan instead of skipping it
The base test hardcodes block attribute names (transformer_blocks,
blocks, etc.) but AceStep uses 'layers'. Override with AceStep-specific
block access path instead of skipping.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add joint_attention_kwargs to AceStepPipeline for per-step LoRA scale
- Add @apply_lora_scale("joint_attention_kwargs") decorator and
joint_attention_kwargs param to AceStepTransformer1DModel.forward
- Add joint_attention_kwargs param to AceStepPipeline.__call__ and wire
it through to all three transformer calls
- Un-skip 4 LoRA scale tests that now pass (39 passed, 13 skipped)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: override test_lora_fuse_nan instead of skipping it
The base test hardcodes block attribute names (transformer_blocks,
blocks, etc.) but AceStep uses 'layers'. Override with AceStep-specific
block access path instead of skipping.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add joint_attention_kwargs to AceStepPipeline for per-step LoRA scale
- Add @apply_lora_scale("joint_attention_kwargs") decorator and
joint_attention_kwargs param to AceStepTransformer1DModel.forward
- Add joint_attention_kwargs param to AceStepPipeline.__call__ and wire
it through to all three transformer calls
- Un-skip 4 LoRA scale tests that now pass (39 passed, 13 skipped)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: use attention_kwargs instead of joint_attention_kwargs, use torch_device in test
- Rename joint_attention_kwargs to attention_kwargs in transformer
forward and pipeline __call__ for consistency with the diffusers
naming convention for new pipelines
- Add docstring for the attention_kwargs parameter
- Use torch_device instead of hardcoded "cpu" in test_lora_fuse_nan
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Apply style fixes
* docs: document ACE-Step attention kwargs
* docs: document ACE-Step LoRA loader mixin
---------
Co-authored-by: zcy <zhuchenyang.zcy@alibaba-inc.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
