transformers
58d5a2b7 - Fix VoxtralRealtime rejecting non-static cache implementations (#48082)

Commit
68 days ago
Fix VoxtralRealtime rejecting non-static cache implementations (#48082) * Fix VoxtralRealtime rejecting non-static cache implementations `_prepare_cache_for_generation` is overridden only to pre-allocate a `StaticCache` for the audio encoder, but it also raised on any `cache_implementation` that is not static. This was inconsistent: the default (`cache_implementation=None`) already goes through a `DynamicCache`, lazily created by the encoder, and produces the exact same output as the static path. Explicitly asking for `"dynamic"` (or `"offloaded"`) therefore failed for no reason. Only pre-allocate the encoder cache for static implementations and let the generic `generate` logic handle the rest, so unsupported combinations are reported by the framework instead of a model-level allowlist. Also fix the `test_generate_with_quant_cache` skip reason: quantized cache is unsupported because the model uses sliding attention layers. * Drop the offloaded cache from the new generation test `offloaded` follows the exact same code path as `dynamic` here (no pre-allocation, the encoder lazily creates its own `DynamicCache`), so it added no coverage while requiring an accelerator: `Cache.__init__` builds a prefetch stream upfront, which fails on CPU-only runners. * Address review: mark generate test and rely on generic quant cache skip - add `@pytest.mark.generate` to `test_generate_with_cache_implementation` - drop the model-level `test_generate_with_quant_cache` skip and make the generic skip in `GenerationTesterMixin` use `get_layer_types_and_kwargs`, the same helper `QuantizedCache` uses, so models that only set `sliding_window` (without an explicit `layer_types`) are skipped too
Author
Parents
Loading