huggingface/lighteval

Pull Requests Commits

Merge branch 'main' into mixeval_dive

clefourrier committed 151 days ago

Verified 904448fc

Fix broken link (#1014)

JoelNiklaus committed 151 days ago

Verified b63348d6

fix semaphore calls (#1012)

clefourrier committed 156 days ago

Verified c819e179

clefourrier committed 158 days ago

Verified ba607569

Added litellm model config options and improved `_prepare_max_new_tokens` (#967)

rolshoven committed 159 days ago

Verified e98e4639

Merge branch 'main' into mixeval_dive

clefourrier committed 159 days ago

Verified 5f291a00

Fix None Doc (#1008)

lewtun committed 159 days ago

Verified 7a456fd4

more memory for flow judge

clefourrier committed 161 days ago

32bb552d

stringify model judge + change max_length to what's actually passed instead of setting a bunch of overwrites

clefourrier committed 161 days ago

a83d84ab

likely fix for the GSM+ issue

clefourrier committed 161 days ago

257ff354

eval tracker fix 1

clefourrier committed 161 days ago

2ae58aaf

clefourrier committed 161 days ago

fa3ef629

also debugging the judge

clefourrier committed 161 days ago

213eda86

also debugging the judge

clefourrier committed 161 days ago

9d5bd304

clefourrier committed 161 days ago

e15db12c

Revert extraction setting for `IndicesExtractionConfig` (#998)

cmpatino committed 165 days ago

Verified b1d45e36

Adding New Task SLR-Bench as a Community Task : Scalable Logical Reasoning Benchmark (#983)

Ahmad21Omar committed 170 days ago

Verified c7a063ae

Fix AvgAtK num_samples method (#987)

amstu2 committed 172 days ago

Verified 5137e03f

Fix gsm8k HF repo (#986)

amstu2 committed 172 days ago

Verified 3ec1992b

fx lcb metric (#981)

NathanHB committed 172 days ago

Verified 0b16f8a5

adds `enable_prefix_caching` option to VLLMModelConfig (#945)

GAD-cell committed 172 days ago

Verified 16c26303

Bump for release v0.11.0

NathanHB committed 173 days ago

5a0e8972

Sample to sample compare for integration tests (#977)

NathanHB committed 173 days ago

Verified 35ae2d15

[RFC] Rework the dependencies to be more versatile (#951)

LysandreJik committed 176 days ago

Verified 6ee4ff81

Refine acknowledgements in README.md (#976)

clefourrier committed 177 days ago

Verified e61b868a

Support local GGUF in VLLM and use HF tokenizer #943 (#972)

JIElite committed 177 days ago

Verified d6a65ca5

Add auto tests for metrics (#939)

NathanHB committed 178 days ago

Verified 16318bb4

Add IFBench (#944)

clefourrier committed 179 days ago

Verified 3a71f688

Added `backend_options` parameter to llm judges. (#963)

rolshoven committed 179 days ago

Verified 9ba430f1

Multilingual extractiveness (#956)

rolshoven committed 179 days ago

Verified 2dc17885

Older