transformers
ac372e10 - [deepseek_v4] fix CSA per-query masking in eager path

Commit
166 days ago
[deepseek_v4] fix CSA per-query masking in eager path Two related bugs in DeepseekV4Indexer / DeepseekV4CSACompressor that made multi-token forwards (S > 1) silently use the wrong CSA attention pattern: * Indexer.topk had no causal pre-mask, so query t could select compressed blocks containing tokens at position >= t. Add the same -inf mask + -1 sentinel that DeepSeek's reference inference/model.py uses. * CSACompressor flattens gathered top-k entries to [B, 1, S*k, D] and the caller right-padded the attention mask with 0.0, so every query attended to every other query's selection. Build a per-query block bias (-inf off-diagonal, with the sentinel routed through) and use it instead of the zero pad. HCA compressor return signature aligned to (kv, None) for parity.
Author
Parents
Loading