transformers
83445dd7 - Keep `attention_mask` as `None` in OPT's causal mask creation (#49002)

Commit
40 days ago
Keep `attention_mask` as `None` in OPT's causal mask creation (#49002) * Keep `attention_mask` as `None` in OPT's causal mask creation `OPTDecoder` replaces a `None` `attention_mask` with a dense all-ones tensor so that the learned positional embeddings can be computed. That tensor was then also forwarded to `create_causal_mask`, which therefore had to inspect its values and could never fall back to sdpa's `is_causal` argument. Keep the dense tensor local to the positional embeddings and pass the original `attention_mask` to `create_causal_mask`. * Move the mask test into OPTModelTester * Add a compiled flash-attention forward as an end-to-end check
Author
Parents
Loading