Fix mask creation not being skipped under `torch.compile` (#48975)
* Skip mask creation under torch.compile when there is no padding mask
`_ignore_causal_mask_sdpa` and `_ignore_bidirectional_mask_sdpa` bailed out on
`is_tracing(padding_mask)`, which is always True under `torch.compile`. As a
result the 4D mask was materialized even without a padding mask, so sdpa could
never dispatch to its flash/oneDNN kernels via `is_causal`.
Whether a padding mask is passed at all is a static property that dynamo guards
on. Only the checks reading the mask values are a data-dependent control flow,
so the tracing guard is moved down to those, and `torch.export` keeps bailing out
early through `is_torchdynamo_exporting()`.
OPT additionally overwrote `attention_mask` with a dense all-ones mask to infer
its learned positional embeddings, which then leaked into `create_causal_mask`
and defeated the skip. The dense mask is now kept in a local variable.
Fixes #48924
Fixes #48925
* Gate the compile skip test on torch>=2.14 and return bools from the skip helpers
`is_torchdynamo_exporting()` maps to `torch.compiler.is_exporting()`, which dynamo
hard-coded to `True` in its `tracing_state_functions` table, i.e. it also reported
exporting under plain `torch.compile`. That was only fixed by pytorch#176499, released
in torch 2.14, so on older versions the export guard still triggers under compile and
the mask keeps being materialized (conservative and correct, just without the speedup).
Since setup.py allows torch>=2.5, gate the new compile test with
`@require_torch_greater_or_equal("2.14")` so it does not fail on older versions, and
document the dependency next to the guard.
Note that this cannot be worked around from user code: on 2.7.1 every tracing-state
signal has identical values under `torch.compile` and strict `torch.export`, and reading
`torch.compiler._is_exporting_flag` directly is unsafe because it is `False` during
strict export and would bake the skip into the exported program.
Also return a Python `bool` via `bool(fast_all(padding_mask))` from
`_ignore_bidirectional_mask_sdpa` / `_can_skip_bidirectional_mask_xpu` instead of a
0-dim tensor from `padding_mask.all()`, matching their `-> bool` annotation and the
`fast_all` helper already used by the causal path.
* Reduce the diff to a single shared tracing guard
Address review feedback: instead of reordering the skip helpers, keep their
original structure and only replace the tracing guard. The new helper makes
the export check explicit and removes the duplicated `is_tracing` calls.
Verified to be decision-identical to the previous version across the full
grid of padding masks, query lengths, cache offsets and local attention
sizes, in eager, `torch.compile` and `torch.export`.
* Move the OPT fix to a separate PR
The `modeling_opt.py` change is independent of the mask skipping logic and is
better reviewed on its own, so this PR is now limited to `masking_utils.py`.
* Simplify the tracing guard docstring
* Inline the tracing guard and keep the original comments
Drops the helper introduced earlier: the condition is now inlined at the three
call sites, so masking_utils.py is down to +7/-3 with every original comment
restored. Also rephrases the torch 2.14 note, which was misleading: dynamo
replaces calls to torch.compiler.is_exporting() by a constant True while
tracing, it is not the function itself that is hard-coded.
* Address review nits: single NOTE block, comment in docstring, no loop