lighteval
1036929d - Registry refactorisation (#937)

Commit
142 days ago
Registry refactorisation (#937) * init, wip * unrelated but these tasks were buggy * better suite management: we don't load all optional deps all the time * upgrade * singleton + transformer sampling fix in config * incredible how much code was just pulled from ifeval * fix test 1 * fix test 2 * fix tests part 1 - also removes fewshot truncation in the task name because it's no longer used anywhere in the code logically * fix registry mockup * fixed last tests * remove ifbench * allow old task name but logs a warning * Update tests/utils.py * Update src/lighteval/tasks/registry.py * Update src/lighteval/tasks/registry.py Co-authored-by: Nathan Habib <30601243+NathanHB@users.noreply.github.com> * Update src/lighteval/tasks/registry.py * style --------- Co-authored-by: Nathan Habib <30601243+NathanHB@users.noreply.github.com>
Author
Parents
Loading