transformers.js
[v4] Improve download progress tracking (model cache registry and define which files will be loaded for pipelines)
#1511
Merged
Merged
[v4] Improve download progress tracking (model cache registry and define which files will be loaded for pipelines) #1511
xenova
merged
62 commits into
main
from
v4-cache-handler
added progress_total progress callback status info
dfebb4a4
added get_file_metadata helper
92163263
some clean up
17f98553
improved get_file_metadata and get_files
4fbe9a1d
added functions to main export
ba4a4bab
removed dynamic import
6249f311
restructuring
be8d7bbf
refactored the pipeline tasks so I can have a get_pipeline_files func…
9f3e2249
updated doc
7b327a6a
Update packages/transformers/src/utils/core.js
32dad76a
added is_pipeline_cached and improved return object
46433d68
fixes after review
4eeed397
added ModelRegistry to doc
f19ccfd7
added clear_cache and clear_pipeline_cache
3430421f
xenova
changed the title
Update packages/transformers/src/utils/cache/clear_cache.js
be5a6b2b
small doc fix
563e872e
Merge branch 'v4-cache-handler' of github.com:huggingface/transformer…
01d5b23c
changed delete logic for cache
856d3024
fixed examples in cache utilitiy files
4cb293d7
fixed examples
83170d17
renamed type
b9d8c33b
refactoring get_file_metadata
056843f7
moved src/utils/pipeline-tasks.js to src/pipelines/index.js
4f38c6d7
fixed doc builder
138a6d9b
fixed doc builder
e45bbbb4
created shared getFetchHeaders function
ef6c1969
added case for DecoderOnlyWithoutHead and DecoderOnly
dfdfa41a
Merge branch 'main' into v4-cache-handler
ffc38705
improved console.warn
3a19ca9f
changed to modelType = MODEL_TYPES.EncoderOnly if not foundInMapping
ded0a380
removed full download from get_file_metadata
908e5bb6
solved merge conflicts
6bf83cf9
Merge branch 'main' into v4-cache-handler
e095ee8a
Merge branch 'main' into v4-cache-handler
6a890676
Merge branch 'main' into v4-cache-handler
eb8ae841
Remove duplicate module tag (already set in ModelRegistry.js)
3dbfa36b
Remove test file
c3761493
pnpm format
5b647582
Cleanup
333d8a77
use config from_pretrained logic for ensuring config is of correct type
842334c8
Reorder file acquisition
34d1299e
Add example JSDoc to file header
7ef10ad9
Add ModelRegistry tests
0248d904
FIXME: skip cache clearing tests
131d1df8
Formatting
3ba1c30d
Unify model-loader.js, get_model_files.js, and session.js
f8679af8
nico-martin
commented
on 2026-02-27
console.warn to logger.warn
1c34775e
Only resolve dtype once in session.js
8bbe2290
Use map + Promise.all
d794990d
map -> forEach
499ae00a
Use env.fetch instead of global fetch
e7cb64d1
Add comment to clear_cache for clarity
bbc0fec7
Add model_file_name support in cache operations
47ea1586
Update cache tests
14400e80
Fix TOCTOU race condition
e6416726
Remove dead code
59b2e645
cleanup
a49c0bfa
Cleanup pipeline import/exports
36bde74c
renamed folder cache to model_registry
9e10bf7a
changed doc title
bafc15cf
fix for unit tests on node 20
c004db39
standardize module name
3fa74fd3
xenova
approved these changes
on 2026-03-01
Login to write a write a comment.
Login via GitHub