lighteval
Initial proposal for model lazy loading
#497
Open

Initial proposal for model lazy loading #497

JoelNiklaus
JoelNiklaus181 days ago

This is a first try for issue #496.

However, we need the docs which currently in turn depend on the model being initialized. The model itself is not actually needed, but the tokenizer would be enough (in lighteval_task.create_requests_from_tasks).

JoelNiklaus Implemented the possibility to load predictions from details files an…
a95156e8
JoelNiklaus Run model as fallback when no details can be loaded.
66244528
JoelNiklaus Merge branch 'main' into load-details
ed676249
JoelNiklaus Merge branch 'main' into load-details
874541bd
JoelNiklaus Improved loading speed and added more useful error messages.
742a6724
JoelNiklaus Fixed typo.
eaedd04c
JoelNiklaus Fixed gnarly bug with details loading to prevent loading too many exa…
ca8331a9
JoelNiklaus Added initial proposal for lazy loading model initialization.
e22d4784
JoelNiklaus JoelNiklaus marked this pull request as draft 181 days ago

Login to write a write a comment.

Login via GitHub

Reviewers
No reviews
Assignees
No one assigned
Labels
Milestone