Enable disabling test cases on specific platforms (#61427)
Summary:
This adds functionality to our common_utils.py to allow disabling test cases for platforms Mac, Windows, and Linux.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/61427
Test Plan:
CI should not change as no issues currently have the line "Platforms:..."
I tested locally by making sure `test_async_script` is skipped while running `python test/test_jit.py -k TestAsync.test_async_script` with a cached modified `.pytorch-disabled-tests.json`:
```
{
"total_count": 32,
"incomplete_results": false,
"items": [
{
"url": "https://api.github.com/repos/pytorch/pytorch/issues/60652",
"repository_url": "https://api.github.com/repos/pytorch/pytorch",
"labels_url": "https://api.github.com/repos/pytorch/pytorch/issues/60652/labels{/name}",
"comments_url": "https://api.github.com/repos/pytorch/pytorch/issues/60652/comments",
"events_url": "https://api.github.com/repos/pytorch/pytorch/issues/60652/events",
"html_url": "https://github.com/pytorch/pytorch/issues/60652",
"id": 929288995,
"node_id": "MDU6SXNzdWU5MjkyODg5OTU=",
"number": 60652,
"title": "DISABLED test_async_script (jit.test_async.TestAsync)",
"user": {
"login": "ezyang",
"id": 13564,
"node_id": "MDQ6VXNlcjEzNTY0",
"avatar_url": "https://avatars.githubusercontent.com/u/13564?v=4",
"gravatar_id": "",
"url": "https://api.github.com/users/ezyang",
"html_url": "https://github.com/ezyang",
"followers_url": "https://api.github.com/users/ezyang/followers",
"following_url": "https://api.github.com/users/ezyang/following{/other_user}",
"gists_url": "https://api.github.com/users/ezyang/gists{/gist_id}",
"starred_url": "https://api.github.com/users/ezyang/starred{/owner}{/repo}",
"subscriptions_url": "https://api.github.com/users/ezyang/subscriptions",
"organizations_url": "https://api.github.com/users/ezyang/orgs",
"repos_url": "https://api.github.com/users/ezyang/repos",
"events_url": "https://api.github.com/users/ezyang/events{/privacy}",
"received_events_url": "https://api.github.com/users/ezyang/received_events",
"type": "User",
"site_admin": false
},
"labels": [
{
"id": 1301397902,
"node_id": "MDU6TGFiZWwxMzAxMzk3OTAy",
"url": "https://api.github.com/repos/pytorch/pytorch/labels/module:%20flaky-tests",
"name": "module: flaky-tests",
"color": "f7e101",
"default": false,
"description": "Problem is a flaky test in CI"
},
{
"id": 679953883,
"node_id": "MDU6TGFiZWw2Nzk5NTM4ODM=",
"url": "https://api.github.com/repos/pytorch/pytorch/labels/oncall:%20distributed",
"name": "oncall: distributed",
"color": "f7e101",
"default": false,
"description": "Add this issue/PR to distributed oncall triage queue"
}
],
"state": "open",
"locked": false,
"assignee": {
"login": "rohan-varma",
"id": 8039770,
"node_id": "MDQ6VXNlcjgwMzk3NzA=",
"avatar_url": "https://avatars.githubusercontent.com/u/8039770?v=4",
"gravatar_id": "",
"url": "https://api.github.com/users/rohan-varma",
"html_url": "https://github.com/rohan-varma",
"followers_url": "https://api.github.com/users/rohan-varma/followers",
"following_url": "https://api.github.com/users/rohan-varma/following{/other_user}",
"gists_url": "https://api.github.com/users/rohan-varma/gists{/gist_id}",
"starred_url": "https://api.github.com/users/rohan-varma/starred{/owner}{/repo}",
"subscriptions_url": "https://api.github.com/users/rohan-varma/subscriptions",
"organizations_url": "https://api.github.com/users/rohan-varma/orgs",
"repos_url": "https://api.github.com/users/rohan-varma/repos",
"events_url": "https://api.github.com/users/rohan-varma/events{/privacy}",
"received_events_url": "https://api.github.com/users/rohan-varma/received_events",
"type": "User",
"site_admin": false
},
"assignees": [
{
"login": "rohan-varma",
"id": 8039770,
"node_id": "MDQ6VXNlcjgwMzk3NzA=",
"avatar_url": "https://avatars.githubusercontent.com/u/8039770?v=4",
"gravatar_id": "",
"url": "https://api.github.com/users/rohan-varma",
"html_url": "https://github.com/rohan-varma",
"followers_url": "https://api.github.com/users/rohan-varma/followers",
"following_url": "https://api.github.com/users/rohan-varma/following{/other_user}",
"gists_url": "https://api.github.com/users/rohan-varma/gists{/gist_id}",
"starred_url": "https://api.github.com/users/rohan-varma/starred{/owner}{/repo}",
"subscriptions_url": "https://api.github.com/users/rohan-varma/subscriptions",
"organizations_url": "https://api.github.com/users/rohan-varma/orgs",
"repos_url": "https://api.github.com/users/rohan-varma/repos",
"events_url": "https://api.github.com/users/rohan-varma/events{/privacy}",
"received_events_url": "https://api.github.com/users/rohan-varma/received_events",
"type": "User",
"site_admin": false
}
],
"milestone": null,
"comments": 0,
"created_at": "2021-06-24T14:28:33Z",
"updated_at": "2021-06-24T16:40:42Z",
"closed_at": null,
"author_association": "CONTRIBUTOR",
"active_lock_reason": null,
"body": "Platforms:Mac, windows, Linux\r\n```\r\nJun 24 00:59:14 ======================================================================\r\nJun 24 00:59:14 ERROR [0.477s]: test_async_script (__main__.ProcessGroupGlooWrapperTest)\r\nJun 24 00:59:14 ----------------------------------------------------------------------\r\nJun 24 00:59:14 Traceback (most recent call last):\r\nJun 24 00:59:14 File \"/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_distributed.py\", line 398, in wrapper\r\nJun 24 00:59:14 self._join_processes(fn)\r\nJun 24 00:59:14 File \"/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_distributed.py\", line 590, in _join_processes\r\nJun 24 00:59:14 self._check_return_codes(elapsed_time)\r\nJun 24 00:59:14 File \"/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_distributed.py\", line 633, in _check_return_codes\r\nJun 24 00:59:14 raise RuntimeError(error)\r\nJun 24 00:59:14 RuntimeError: Process 0 exited with error code 10 and exception:\r\nJun 24 00:59:14 RuntimeError: [/var/lib/jenkins/workspace/third_party/gloo/gloo/transport/tcp/pair.cc:598] Connection closed by peer [172.17.0.2]:21400\r\nJun 24 00:59:14 \r\nJun 24 00:59:14 During handling of the above exception, another exception occurred:\r\nJun 24 00:59:14 \r\nJun 24 00:59:14 Traceback (most recent call last):\r\nJun 24 00:59:14 File \"/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_distributed.py\", line 516, in run_test\r\nJun 24 00:59:14 getattr(self, test_name)()\r\nJun 24 00:59:14 File \"/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_distributed.py\", line 400, in wrapper\r\nJun 24 00:59:14 fn()\r\nJun 24 00:59:14 File \"distributed/test_pg_wrapper.py\", line 270, in test_collective_hang\r\nJun 24 00:59:14 self._test_collective_hang(pg)\r\nJun 24 00:59:14 File \"distributed/test_pg_wrapper.py\", line 52, in _test_collective_hang\r\nJun 24 00:59:14 wrapper_pg.allreduce([tensor])\r\nJun 24 00:59:14 File \"/opt/conda/lib/python3.6/unittest/case.py\", line 217, in __exit__\r\nJun 24 00:59:14 expected_regex.pattern, str(exc_value)))\r\nJun 24 00:59:14 File \"/opt/conda/lib/python3.6/unittest/case.py\", line 135, in _raiseFailure\r\nJun 24 00:59:14 raise self.test_case.failureException(msg)\r\nJun 24 00:59:14 AssertionError: \"Ranks 1 failed to pass monitoredBarrier\" does not match \"[/var/lib/jenkins/workspace/third_party/gloo/gloo/transport/tcp/pair.cc:598] Connection closed by peer [172.17.0.2]:21400\"\r\n```\r\n\r\nhttps://www.internalfb.com/intern/opensource/ci/job/log/225221175921058/\n\ncc pietern mrshenli pritamdamania87 zhaojuanmao satgera rohan-varma gqchen aazzolini osalpekar jiayisuse agolynski SciPioneer H-Huang mrzzd cbalioglu gcramer23",
"performed_via_github_app": null,
"score": 0.0
}
]
}
```
Reviewed By: iramazanli
Differential Revision: D29627799
Pulled By: janeyx99
fbshipit-source-id: 5ef79127cbe0055c4f41766048e66f98cf80d2c4
