DeepSpeed
1a45bd8e - Lock cache file of HF model list (#6628)

Commit
344 days ago
Lock cache file of HF model list (#6628) The error in the following log suggests that the cache file for HF model list can be broken: https://github.com/microsoft/DeepSpeed/actions/runs/11343665365/job/31546708118?pr=6614 The actual cause of the above error is unclear, but `_hf_model_list` potentially breaks the cache file when it is concurrently called from multiple processes. This PR locks the cache file to ensure `_hf_model_list` safely reads and writes the file.
Author
Parents
Loading