Fix slow integration tests on XPU (#48611)
* Fix Glm46V batched integration tests on XPU
The three batched `Glm46VIntegrationTest` generation tests hardcode the
CUDA output, so they fail on XPU with a different (but valid) completion.
Two distinct causes, measured by generating 4x per test with
`use_deterministic_algorithms` off and on and counting distinct outputs:
- `test_small_model_integration_test_batch` is genuinely non-deterministic
on XPU (2 distinct outputs out of 4 runs); determinism collapses it to 1,
so it gets `@require_deterministic_for_xpu`.
- `test_small_model_integration_test_batch_wo_image` and
`test_small_model_integration_test_batch_different_resolutions` are
perfectly stable run to run and simply diverge from CUDA numerically, so
they only need an `("xpu", None)` `Expectations` entry.
The existing values move under `(None, None)`, leaving CUDA untouched. The
unbatched `test_small_model_integration_test` matches CUDA exactly and is
not changed - only the batched paths diverge.
Validated on Intel Arc Pro B60.
* Fix Bark test_generate_batching on XPU by enforcing determinism
`torch.nn.LSTM` is non-deterministic on XPU: calling the same module twice
with the same input tensor under `eval()`/`no_grad()` yields results that
differ by ~5e-08. `Conv1d`/`ConvTranspose1d` are bit-exact, and CPU LSTM is
bit-exact, so this is specific to the XPU LSTM kernel.
`BarkModel.codec_decode` runs the Encodec decoder, which uses LSTMs, so
`test_generate_batching` fails on its final assertion:
self.assertTrue((outputs == output1).all().item())
That assertion requires *bitwise* equality between two `generate()` calls.
Two `generate()` calls with identical arguments already differ on XPU, so
the failure is unrelated to `return_output_lengths`. The two
`assert_close(rtol=2e-3, atol=2e-3)` assertions earlier in the test pass.
Enabling deterministic algorithms makes the LSTM bit-exact (5/5 mismatches
-> 0/5), so mark the test with the existing `require_deterministic_for_xpu`
decorator, as already done for florence2, falcon_mamba, cosmos3_omni, hqq
and fbgemm_fp8. The decorator is a no-op off XPU, so CUDA/CPU are unaffected.
* Fix Glm4v batched integration tests on XPU
Same issue as the Glm46V tests fixed in the previous commit: the three
batched `Glm4vIntegrationTest` generation tests hardcode the CUDA output
and fail on XPU with a different (but valid) completion. Both classes
drive the same `THUDM/GLM-4.1V-9B-Thinking` checkpoint, and the XPU texts
turn out to be identical.
Measured by generating 6x per test with `use_deterministic_algorithms`
off and on and counting distinct outputs:
- `test_small_model_integration_test_batch` is non-deterministic on XPU
(2 distinct outputs with determinism off, 1 with it on), so it gets
`@require_deterministic_for_xpu` in addition to the new expectation.
- `test_small_model_integration_test_batch_wo_image` and
`test_small_model_integration_test_batch_different_resolutions` only
need an `("xpu", None)` `Expectations` entry.
The existing values move under `(None, None)`, leaving CUDA untouched.
Validated on Intel Arc Pro B60.
* Re-baseline Glm4v/Glm46V XPU expectations on healthy hardware
The XPU expectations added in the previous commits were recorded on a machine
with a faulty GPU, so some of them captured corrupted output. Two values are
corrected here, and `test_small_model_integration_test` gains the XPU
expectation it was missing.
The new values were verified to be independent of how many GPUs are visible:
`device_map="auto"` is pipeline sharding, so each layer runs whole on one device
and the inter-layer transfer is a plain copy. Running with 2, 4 and 6 GPUs gives
byte-identical logits and byte-identical generated text.
* Fix Qwen3VLMoe integration tests on XPU
Three Qwen3-VL-30B-A3B integration tests hardcode CUDA-generated text and
fail on XPU with a semantically equivalent but differently worded
completion. Wrap the expected strings in Expectations so both backends
are covered.
The XPU values were harvested on 4 and 6 devices and are identical, so
the number of shards used by device_map="auto" does not affect them.
* Fix Florence2 integration tests on XPU
Three tests in Florence2ForConditionalGenerationIntegrationTest hardcoded
CUDA-only baselines and failed on XPU:
- test_base_model_inference_sdpa
- test_base_model_batching_inference_eager
- test_large_model_batching_inference_sdpa
The outputs differ only by small numeric drift. Most differences are
<loc_N> coordinate tokens off by 1-3. In the large batched sdpa case the
drift reorders beams (Florence-2 generates with num_beams=3), so the
caption changes wording ("a green volkswagen beetle" vs "a green car")
while the OCR text is identical on both platforms.
Wrap the affected values in Expectations with an ("xpu", 5) entry,
following the convention already used elsewhere in this file, and keep
the existing values under (None, None).
The XPU values were harvested on two different cards and are identical,
so they are stable rather than nondeterministic. No determinism
decorator is added, matching the other non-batching tests here.
* Fix XPU integration test expectations across 7 models
Re-baseline integration tests that fail on Intel Arc Pro B60 (Battlemage).
`Expectations` derives its XPU key from the device generation, which is 5 on
Battlemage. Several of these tests already carried an `("xpu", 3)` entry from
Intel's PVC-era CI: that key scores higher than the `(None, None)` default, so
the gen-3 baseline was silently applied to gen-5 hardware and the test failed on
a small numeric delta. Rather than overwriting those values (which would break
gen-3 machines), add a separate `("xpu", 5)` entry next to them.
- focalnet, eomt_dinov3, sam2_video: numeric drift in the last digits
- llava: gen-5 output matches the CUDA text, not the existing `("xpu", 3)` one
- cosmos3_omni, gemma3n, emu3: single-word generation fork
`dac` stores its expected values in a shared JSON fixture with no device
dimension. Add `get_json_expectation()` to `testing_utils` so a fixture value
can hold the same mapping as an `Expectations` dict, with the keys written as
their repr since JSON has no tuple keys.
* Key the new XPU expectations on gen 5 rather than on every XPU
These values were only measured on Battlemage (gen 5), so `("xpu", None)`
overstated them: it also applies to PVC-era devices, which are covered by the
`(None, None)` default today. Narrowing to `("xpu", 5)` keeps every other
device on its current expectation. Pre-existing `("xpu", None)` entries added
by others are left alone.
* Dequantize the mxfp4 checkpoint in gpt-oss `test_training_step`
The test only passes `dtype=torch.bfloat16`, so on any machine where mxfp4 is
supported (every XPU, CUDA with compute capability >= 7.5, plus triton and
kernels) the checkpoint stays quantized. `Mxfp4HfQuantizer.is_trainable` is
False and `Mxfp4GptOssExperts` holds `requires_grad=False` weights behind a
raw triton kernel, so `backward()` never reaches the MoE branch: every
`mlp.router.*`, `mlp.experts.*_bias` and `post_attention_layernorm.weight`
comes back with `grad is None` and the test fails on the first router.
The `quantized` parameter was only ever used to build the results-JSON key,
and the quantized variants are skipped, so requesting the dequantized weights
is what the non-quantized cases already meant to do.
* Record the measured CUDA value explicitly in the dac fixture
* Guard the dac integration tests with require_torch_accelerator
* Re-measure two XPU expectations missed after the rebase
Rebasing onto main pulled in #48589, which repointed several test images at `hf-internal-testing/transformers-synthetic-assets` and collapsed the stale `Expectations` dicts recorded against the old assets down to a single `(None, None)` measured on cuda. That correctly dropped this branch's `("xpu", 5)` entries, but the new cuda baseline does not hold on XPU, so `llava::test_small_model_integration_test_llama_single` still fails there; re-measured on Arc Pro B60 and added back. The same run also showed `sam2_video::test_inference_mask_generation_video_one_point_propagate_in_video_directly` failing (4/12 elements, greatest abs diff 0.005428 for atol 1e-4) - it is the same computation as `..._one_point`, which this branch already fixed, just with `start_frame_idx` passed explicitly, so it gets the same `Expectations` treatment and the same values.
Both verified on 7x Intel Arc Pro B60: llava 1 passed, sam2_video 10 passed / 3 subtests passed.
