transformers
a19f0d7b - Raise when a paged attention forward is called with no cache (#48297)

Commit
47 days ago
Raise when a paged attention forward is called with no cache (#48297) * Fall back to the base attention implementation when a paged attention forward has no cache A standard forward on a model switched to a paged attention implementation (as done by ContinuousBatchingManager) silently computed non-causal attention: sdpa_attention_paged_forward treats cache as optional and mask creation early-exits to None for paged implementations. Now, when the continuous batching kwargs are absent, the paged sdpa/flash forwards delegate to their base implementation, the eager paged forward materializes a causal mask, and mask creation resolves 2D padding masks with the base implementation's mask function. * Raise instead of falling back when a paged attention forward has no cache Drops the redirect to the base implementation and the paged-aware mask creation, per review. `paged|sdpa` and `paged|eager` now refuse a call with no paged cache rather than silently attending bidirectionally. --------- Co-authored-by: Rémi Ouazan <83456801+remi-or@users.noreply.github.com>
Author
Parents
Loading