CI: Simplify Python CI (#3982)
* CI: use actions/setup-python's native caching
* CI: don't create virtualenvs (CI is run in an isolated context anyway)
* CI: clean up extra directory-creating step
* Changelog
* Update CHANGELOG.md
---------
Co-authored-by: Abubakar Abid <abubakar@huggingface.co>
