huggingface/lighteval

Pull Requests Commits

Merge branch 'main' into nathan-try-fix-vllm

clefourrier committed 95 days ago

Verified 52f8cf66

Fix 999: always provide parameters in the metric name to allow using several combinations (#1017)

clefourrier committed 95 days ago

Verified 70acb852

Fix nltk import failing (#1013)

clefourrier committed 95 days ago

Verified 3af89255

Fixing mixeval (#1006)

clefourrier committed 95 days ago

Verified 090630eb

Split up enhancement and features in release notes template (#984)

NathanHB committed 95 days ago

Verified c2b83e20

Fix broken link (#1014)

JoelNiklaus committed 96 days ago

Verified b63348d6

fix semaphore calls (#1012)

clefourrier committed 100 days ago

Verified c819e179

clefourrier committed 102 days ago

Verified ba607569

Added litellm model config options and improved `_prepare_max_new_tokens` (#967)

rolshoven committed 103 days ago

Verified e98e4639

Fix None Doc (#1008)

lewtun committed 104 days ago

Verified 7a456fd4

Revert extraction setting for `IndicesExtractionConfig` (#998)

cmpatino committed 109 days ago

Verified b1d45e36

Adding New Task SLR-Bench as a Community Task : Scalable Logical Reasoning Benchmark (#983)

Ahmad21Omar committed 115 days ago

Verified c7a063ae

Fix AvgAtK num_samples method (#987)

amstu2 committed 116 days ago

Verified 5137e03f

Fix gsm8k HF repo (#986)

amstu2 committed 116 days ago

Verified 3ec1992b

fx lcb metric (#981)

NathanHB committed 117 days ago

Verified 0b16f8a5

adds `enable_prefix_caching` option to VLLMModelConfig (#945)

GAD-cell committed 117 days ago

Verified 16c26303

NathanHB committed 117 days ago

ff886560

NathanHB committed 117 days ago

4222a212

Bump for release v0.11.0

NathanHB committed 117 days ago

5a0e8972

Sample to sample compare for integration tests (#977)

NathanHB committed 118 days ago

Verified 35ae2d15

[RFC] Rework the dependencies to be more versatile (#951)

LysandreJik committed 120 days ago

Verified 6ee4ff81

Refine acknowledgements in README.md (#976)

clefourrier committed 121 days ago

Verified e61b868a

Support local GGUF in VLLM and use HF tokenizer #943 (#972)

JIElite committed 121 days ago

Verified d6a65ca5

Add auto tests for metrics (#939)

NathanHB committed 123 days ago

Verified 16318bb4

Add IFBench (#944)

clefourrier committed 124 days ago

Verified 3a71f688

Added `backend_options` parameter to llm judges. (#963)

rolshoven committed 124 days ago

Verified 9ba430f1

Multilingual extractiveness (#956)

rolshoven committed 124 days ago

Verified 2dc17885

Fixed bug that prevented the metrics from being mixed (some batched and others not batched) (#958)

rolshoven committed 124 days ago

Verified 96a6882e

Update readme (#961)

NathanHB committed 124 days ago

Verified c409f3f9

set upper bound on vllm version (#964)

NathanHB committed 124 days ago

Verified fc50ca09

Older