New git-aware cache file layout (#801)
* light typing
(cherry picked from commit b2c8f9b970c505cdf2c685e645e9e36cc472b0d3)
* remove this seminal comment
(cherry picked from commit 12a841a605c94733154f3b22e812c0f5e69ef37b)
* I don't understand why we don't early return here
cc @patrickvonplaten care to take a look? cc @LysandreJik
(cherry picked from commit 259ab36f03ab3eed6eeb4fc4984bc259619b442f)
* following last commit, unnest this
(cherry picked from commit 54957f3f049d887af21dd8f6950873a2823c4247)
* [BIG] This should work for all repo_types not just models!
(cherry picked from commit 9a3f96ccb2de6663cf4cf2d9a60dd7f415227c1b)
* one more
(cherry picked from commit b74871250616c44a2125b26d5de29b1189e82e12)
* forgot a repo_type and reorder code
(cherry picked from commit 3ef7d79a44087e971e10e35d3b9f5bea3474f297)
* also rename this cache folder
(cherry picked from commit 4c518b861723a6d28d59108403c37edf5208f2fe)
* Use `hf_hub_download`, will be simpler later
(cherry picked from commit c7478d58fe62da02625b8ca17796ad1419a048b1)
* in this new version, `force_filename` does not make sense anymore
(cherry picked from commit 9a674bc795d5c8a26aecf5429d391fff92e47e8d)
* Just inline everything inside `hf_hub_download` for now
(cherry picked from commit ee49f8f57ba4e7e66f237df8f64c804862fe3ee8)
* Big prototype! it works! :tada:
(cherry picked from commit 7fe19ec66a2c5a7386a956cb9b65616cb209608a)
* wip wip
* do not touch `cached_download`
* Prompt user to upgrade to `hf_hub_download`
* Add a `legacy_cache_layout=True` to preserve old behavior, just in case
* Create `relative symlinks` + add some doc
* Fix behavior when no network
* This test now is legacy
* Fix-ish conflict-ish
* minimize diff
* refactor `repo_folder_name`
* windows support + shortcut if user passes a commit hash
* Rewrite `snapshot_download` and make it more robust
* OOops
* Create example-transformers-tf.py
* Fix + add a way more complete example (running on Ubuntu)
* Apply suggestions from code review
Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* Update src/huggingface_hub/file_download.py
Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
* Update src/huggingface_hub/file_download.py
Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
* Only allow full revision hashes otherwise the `revision != commit_hash` test is not reliable
* add a little bit more doc + consistency
* Update src/huggingface_hub/snapshot_download.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* Update snapshot download
* First pass on tests
* Wrap up tests
* :wolf: Fix for bug reported by @thomwolf
see https://github.com/huggingface/huggingface_hub/pull/801#issuecomment-1134576435
* Special case for Windows
* Address comments and docs
* Clean up with ternary cc @julien-c
* Add argument to `cached_download`
* Opt-in for filename_to-url
* Opt-in for filename_to-url
* Pass the flag
* Update docs/source/package_reference/file_download.mdx
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update src/huggingface_hub/file_download.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Address review comments
Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
