PR #801 New git-aware cache file layout

New git-aware cache file layout #801

LysandreJik merged 47 commits into main from cache-file-layout

julien-c changed the title ~~[PoC] Prototype for a new cache file layout~~ [PoC] Prototype for a new git-aware cache file layout 4 years ago

julien-c commented on 2022-03-28

LysandreJik assigned

LysandreJik 4 years ago

Base automatically changed from hf-api-adds to main 4 years ago

julien-c force pushed from 7fe19ec6 to 1087e54f 4 years ago

julien-c changed the title ~~[PoC] Prototype for a new git-aware cache file layout~~ New git-aware cache file layout 4 years ago

patrickvonplaten commented on 2022-05-05

patrickvonplaten approved these changes on 2022-05-05

sgugger approved these changes on 2022-05-05

LysandreJik commented on 2022-03-31

julien-c commented on 2022-05-09

LysandreJik added this to the v0.7 milestone 4 years ago

julien-c commented on 2022-05-19

julien-c commented on 2022-05-24

light typing

6fa66c49

remove this seminal comment

29a8833a

I don't understand why we don't early return here

da920225

following last commit, unnest this

e4d36861

[BIG] This should work for all repo_types not just models!

8e4ccf45

one more

5ab8b742

forgot a repo_type and reorder code

b8376f84

also rename this cache folder

4cb1d631

Use `hf_hub_download`, will be simpler later

f7cbe183

in this new version, `force_filename` does not make sense anymore

ea94d43d

Just inline everything inside `hf_hub_download` for now

7b92719a

Big prototype! it works! :tada:

d29c90ef

wip wip

02480fd0

do not touch `cached_download`

f81885d5

Prompt user to upgrade to `hf_hub_download`

7fbd7a45

Add a `legacy_cache_layout=True` to preserve old behavior, just in case

d1e47a10

Create `relative symlinks` + add some doc

fc930b62

Fix behavior when no network

f5420ed9

This test now is legacy

9f1e0f6f

Fix-ish conflict-ish

973913e0

minimize diff

e9fb4d40

refactor `repo_folder_name`

adaca46d

windows support + shortcut if user passes a commit hash

fff7e9f7

Rewrite `snapshot_download` and make it more robust

06c0ca15

OOops

e3e2485b

Create example-transformers-tf.py

70516833

Fix + add a way more complete example (running on Ubuntu)

84ff68df

Apply suggestions from code review

11974d48

Update src/huggingface_hub/file_download.py

73cdb467

Update src/huggingface_hub/file_download.py

9f8b8769

Only allow full revision hashes otherwise the `revision != commit_has…

2c5c94b2

add a little bit more doc + consistency

35fff445

Update src/huggingface_hub/snapshot_download.py

383022fd

Update snapshot download

7112146b

First pass on tests

8f99d226

Wrap up tests

01e3a61d

:wolf: Fix for bug reported by @thomwolf

2d54dd17

Special case for Windows

8f71ad66

Address comments and docs

442b5b17

LysandreJik force pushed from 3042731a to 442b5b17 4 years ago

LysandreJik marked this pull request as ready for review 4 years ago

LysandreJik commented on 2022-05-24

julien-c commented on 2022-05-25

Clean up with ternary cc @julien-c

e1ff4eb9

LysandreJik commented on 2022-05-25

Add argument to `cached_download`

e69ef4fd

Opt-in for filename_to-url

0d06b557

Opt-in for filename_to-url

d6bb92e1

Pass the flag

42596427

sgugger approved these changes on 2022-05-25

Update docs/source/package_reference/file_download.mdx

a8348ab6

Update src/huggingface_hub/file_download.py

8c599ee1

Address review comments

3dfc00c6

LysandreJik merged f124f8be into main 4 years ago

LysandreJik deleted the cache-file-layout branch 4 years ago

Reviewers

sgugger

patrickvonplaten

LysandreJik

Assignees

LysandreJik

Labels

None yet

Milestone

v0.8

huggingface_hub New git-aware cache file layout #801 Merged

New git-aware cache file layout #801

huggingface_hub
New git-aware cache file layout
#801

Merged