openvino
82d4b708 - Disable CPU functional test on Linux ARM64 - depends on the order of execution (#33206)

Commit
123 days ago
Disable CPU functional test on Linux ARM64 - depends on the order of execution (#33206) ### Details: This PR disables the test to avoid its failure: ``` [ RUN ] smoke_ScaledAttn_CPU/ScaledAttnLayerCPUTest.CompareWithRefs/netPRC=f32_IS=[?.8.?.64]_[?.8.?.64]_[?.1.?.?]_TS=(1.8.100.64)_(1.8.1.64)_(2.8.10.64)_(2.8.10.64)_(1.8.100.64)_(1.8.1.64)_(2.8.10.64)_(2.8.10.64)_(1.1.1.100)_(1.1.1.1)_(2.1.1.10)_(2.1.10.10)_is_causal=0_has_attn=0_has_scale=0_trgDev=CPU_primitive=ref_any MEM_USAGE=2100944KB Coordinate: 0 Expected: 7.21875 Actual: 2365.05 Diff: 2357.83 abs_threshold: 0.000721994 src/tests/functional/base_func_tests/src/base/ov_subgraph.cpp:97: Failure [ COMPARATION ] COMPARATION FAILED! abs_threshold: 0.000000 rel_threshold: 0.000100 [ FAILED ] smoke_ScaledAttn_CPU/ScaledAttnLayerCPUTest.CompareWithRefs/netPRC=f32_IS=[?.8.?.64]_[?.8.?.64]_[?.1.?.?]_TS=(1.8.100.64)_(1.8.1.64)_(2.8.10.64)_(2.8.10.64)_(1.8.100.64)_(1.8.1.64)_(2.8.10.64)_(2.8.10.64)_(1.1.1.100)_(1.1.1.1)_(2.1.1.10)_(2.1.10.10)_is_causal=0_has_attn=0_has_scale=0_trgDev=CPU_primitive=ref_any, where GetParam() = (4-byte object <04-00 00-00>, { ({ ?, 8, ?, 64 }, { { 1, 8, 100, 64 }, { 1, 8, 1, 64 }, { 2, 8, 10, 64 }, { 2, 8, 10, 64 } }), ({ ?, 8, ?, 64 }, { { 1, 8, 100, 64 }, { 1, 8, 1, 64 }, { 2, 8, 10, 64 }, { 2, 8, 10, 64 } }), ({ ?, 1, ?, ? }, { { 1, 1, 1, 100 }, { 1, 1, 1, 1 }, { 2, 1, 1, 10 }, { 2, 1, 10, 10 } }) }, false, false, false, "CPU", ({}, {}, { "ref_any" }, "ref_any")) (154 ms) ``` It started on November 25th, after which the test execution cache was disabled, which somewhat mitigated the problem, but a few days ago the issue reproduced in one of the PRs, so we started to dig deeper. CPU functional tests in GitHub Actions are executed using `src/tests/test_utils/functional_test_utils/layer_tests_summary/run_parallel.py`. When no `--cache_path` argument is provided to this script, it assigns tests to threads by test suite, then it saves the list of assignments into `test_cache.lst` file, which can be used later. Apparently, the assignment is not determined, so when the failure reproduced in the PR https://github.com/openvinotoolkit/openvino/pull/33040 after the author rebased their changes onto the master branch, I took `test_cache.lst` file generated in the failed run, and used them in my own PR without any code changes whatsoever, and the test failed: https://github.com/openvinotoolkit/openvino/actions/runs/20106444284/job/57693532915?pr=33192 Basically, if you unlucky enough, the test will fail in your PR. It's a time bomb, so let's disable it until it will be fixed. ### Tickets: - CVS-178036
Author
Parents
Loading