script to monitor memory + cpu utilization (#82006)
Add a python script that runs in the background during test jobs to log cpu + gpu memory usage and cpu utilization of python tests (really any python process) to a file and upload the file as an artifact.
I plan on using the the gpu memory usage stats to better understand how to parallelize them, but it is easy to add on other stats if people want them.
In the future, we want to add the ability to track network usage to see if we can decrease it. GPU utilization will also likely need to be improved.
Click the hud link to see uploaded usage log artifacts
Pull Request resolved: https://github.com/pytorch/pytorch/pull/82006
Approved by: https://github.com/huydhn
