lighteval
0da128f5 - fix tasks list (#906)

Commit
116 days ago
fix tasks list (#906) * Update registry.py refix the tasks list command * Update registry.py feat(tasks): add suite filtering and fix community task discovery - Revert the community task path fix - Add --suites parameter to lighteval tasks list - Default to core suites only to prevent overwhelming output - Add dependency checking for multilingual tasks * Update main_tasks.py feat(tasks): add suite filtering and fix community task discovery - Revert the community task path fix - Add --suites parameter to lighteval tasks list - Default to core suites only to prevent overwhelming output - Add dependency checking for multilingual tasks * Update aimo_evals.py fix `community` tasks loading error due to an unexpected keyword argument 'metric' * Update german_rag_evals.py fix `community` tasks loading error due to an unexpected keyword argument 'metric' * Update oz_evals.py fix `community` tasks loading error due to an unexpected keyword argument 'metric' * Update serbian_eval.py fix `community` tasks loading error due to an unexpected keyword argument 'metric' * Update turkic_evals.py fix `community` tasks loading error due to an unexpected keyword argument 'metric' * Update turkic_evals.py Remove the import of `MetricUseCase` because it is given the following error: `Failed to load community tasks from turkic_evals: cannot import name 'MetricUseCase' from 'lighteval.metrics.utils.metric_utils' (/~/repos/lighteval-070825/lighteval/src/lighteval/metrics/utils/metric_utils.py) (registry.py:70)` * Update registry.py adding 'all' as one option in the `--suites` arg to simplify getting the list of both core and optional suites * Update main_tasks.py adding 'all' as one option in the `--suites` arg to simplify getting the list of both core and optional suites
Author
Parents
Loading