Treat non-existent site-packages as empty (#2413)
## Summary
It turns out this doesn't need to exist until something has been
installed into it. See, e.g., https://github.com/astral-sh/uv/pull/2402.
Closes https://github.com/astral-sh/uv/issues/2404.
