Add tests for refactored QKV fusion API in AttentionMixin/AttentionModuleMixin
Tests are in tests/models/test_attention_mixins.py and cover the four minimal
concrete AttentionModuleMixin fixtures (_MinimalSelfAttn, _MinimalCrossAttn,
_MinimalAddedKVAttn, _MinimalAddedQKVAttn):
TestAttentionModuleMixin (53 tests):
- Idempotency of fuse_projections/unfuse_projections
- Module attribute invariants for non-inplace and inplace paths
- Weight/bias correctness: fused weight equals concatenation of split weights
- Inplace round-trip weight preservation and storage-sharing (no copy on unfuse)
- Cross-attention to_kv path, added-KV to_added_kv path (Wan-style), and
added-QKV to_added_qkv path (Flux-style)
- get_qkv and get_added_qkv numerical correctness in split and fused cases
- LoRA guard: fuse_projections/unfuse_projections raise ValueError when PEFT-style
lora_A/lora_B submodules are detected on split or fused projections
TestAttentionMixin (6 tests):
- fuse_qkv_projections/unfuse_qkv_projections propagate to all eligible blocks
- restore_checkpoint_fusion_state respects _native_fused_projections=None/True/False
per block, including mixed-state models
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>