pytorch
e63160d7 - Add squid proxy as egress cache (#62103)

Commit
3 years ago
Add squid proxy as egress cache (#62103) Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/62103 This PR adds a squid proxy that's deployed dedicated for PyTorch CI. Initially we only roll out to GHA, and if things are ok we will extend this to circleci tests if necessary. `http_proxy` and `https_proxy` are compatible with the following http clients: - curl - wget - python Existing cache policy: ``` refresh_pattern -i .(7z|deb|rpm|exe|zip|tar|tgz|gz|ram|rar|bin|tiff|bz2|run|csv|sh)$ 1440 80% 2880 ``` It uses the standard squid refresh_pattern for cache requests. In our setup, we tried to cache at least (1440 minutes - 1 day) and at max (2880 minutes - 2 days), with last-modified factor 80% ([squid doc](http://www.squid-cache.org/Doc/config/refresh_pattern/)). Please refer to [pytorch/test-infra](https://github.com/pytorch/test-infra/tree/master/aws/websites/squid-proxy) for details. Right now, it only applies to the `build` and `test` step, to limit the scope and make sure build and test are more reliable with egress cache. Test Plan: Imported from OSS Reviewed By: jbschlosser, malfet, seemethere, janeyx99 Differential Revision: D29892919 Pulled By: zhouzhuojie fbshipit-source-id: ac17227f2553ca62881711b3e9943488dfd8defd
Author
Parents
Loading