pytorch
6718e3ca - Cache the transformers model used in ONNX test (#96793)

Commit
1 year ago
Cache the transformers model used in ONNX test (#96793) Also updating merge_rule to allow ONNX exporter team to update the Docker script by themselves. By default, the model is cached at ~/.cache/huggingface/hub/ under CI jenkins user. The model is cached so that we don't need to re-download it every time in CI, which causes flaky [CI failures](https://hud.pytorch.org/failure/FAILED%20test%2Fonnx%2Ftest_fx_to_onnx_with_onnxruntime.py%3A%3ATestFxToOnnxWithOnnxRuntime%3A%3Atest_large_scale_exporter_with_tiny_gpt2%20-%20requests.exceptions.ReadTimeout%3A%20HTTPSConnectionPool(host%3D'huggingface.co'%2C%20port%3D443)%3A%20Read%20timed%20out.%20(read%20timeout%3D10.0)). This is the second part after https://github.com/pytorch/pytorch/pull/96590 ### Testing Confirm that the model is cached in the Docker image before running the test: ``` jenkins@dd0db85dd34f:~/workspace$ ls -la ~/.cache/huggingface/hub/models--sshleifer--tiny-gpt2/* /var/lib/jenkins/.cache/huggingface/hub/models--sshleifer--tiny-gpt2/blobs: total 2460 drwxrwxr-x 2 jenkins jenkins 126 Mar 15 05:48 . drwxrwxr-x 5 jenkins jenkins 48 Mar 15 05:48 .. -rw-rw-r-- 1 jenkins jenkins 662 Mar 15 05:48 2c81a6c4c984e95a45338c64a7445c1f0f88077f -rw-rw-r-- 1 jenkins jenkins 2514146 Mar 15 05:48 b706b24034032bdfe765ded5ab6403d201d295a995b790cb24c74becca5c04e6 /var/lib/jenkins/.cache/huggingface/hub/models--sshleifer--tiny-gpt2/refs: total 4 drwxrwxr-x 2 jenkins jenkins 18 Mar 15 05:48 . drwxrwxr-x 5 jenkins jenkins 48 Mar 15 05:48 .. -rw-rw-r-- 1 jenkins jenkins 40 Mar 15 05:48 main /var/lib/jenkins/.cache/huggingface/hub/models--sshleifer--tiny-gpt2/snapshots: total 0 drwxrwxr-x 3 jenkins jenkins 54 Mar 15 05:48 . drwxrwxr-x 5 jenkins jenkins 48 Mar 15 05:48 .. drwxrwxr-x 2 jenkins jenkins 50 Mar 15 05:48 5f91d94bd9cd7190a9f3216ff93cd1dd95f2c7be ``` Pull Request resolved: https://github.com/pytorch/pytorch/pull/96793 Approved by: https://github.com/titaiwangms, https://github.com/ZainRizvi
Author
Committer
Parents
Loading