diffusers
1eb05e31 - [LTX-2.5] move the diffusion decoder's denoising loop into the pipeline

Commit
45 days ago
[LTX-2.5] move the diffusion decoder's denoising loop into the pipeline `LTX2VideoDiffusionDecoderModel.forward` was a whole decode: draw noise, run every stage, integrate a hand-written Euler loop. Denoising loops belong to pipelines, and this one had no reason to be the exception — it just happened to have the tiling wrapped around it, and the tiling had to come along. The model is now a denoiser like any other: `forward(hidden_states, latent_context, timestep, return_dict)` is a single step, and the two `encode_context_*` methods build the conditioning it consumes. `LTX2VideoDiffusionDecodePipeline` owns the rest, with the scheduler driving the integration in place of the open-coded update. Two things stayed put on purpose: - The tile *sizes*. `enable_tiling` is configuration, not behaviour, so it reads the way `vae.enable_tiling()` does everywhere else, existing callers keep working, and there is no new pipeline-level tiling API to justify. The pipeline reads the settings and performs the tiling. - The inner `LTX2VideoDiffusionDecoder3d`. Collapsing it into the `ModelMixin` would drop the `decoder.` prefix from every state-dict key and invalidate checkpoints already converted; that is its own change. The loop and the tile schedule are module-level functions taking `(decoder, scheduler, ...)`, with `decode` / `tiled_decode` as thin methods over them, so `LTX2DiffusionVaeDecoderStep` can reuse them under `# Copied from` — modular blocks must not import from `diffusers.pipelines.*`. That step gains its own `diffusion_decoder_scheduler` component (created from config, so repos predating it still load) and a `decode_num_inference_steps` input, both named apart from the transformer's to avoid colliding in shared state. Falling out of the move: `num_inference_steps` and a custom `sigmas` schedule are now `__call__` arguments, and `@apply_forward_hook` moved from `decode` onto both context methods — accelerate's offload hooks fire on `forward`, and the pipeline calls those before it calls one, so `enable_model_cpu_offload` left the weights on the CPU until that was fixed. `test_model_cpu_offload_decodes` covers it. Verified bit-exact against the pre-move decode on all six {x0,v} x {1,2 step} x {tiled,untiled} cases; the scheduler reproduces the hand-written update exactly. Model tiling tests moved to the pipeline suite. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Author
Committer
Parents
Loading