huggingface/lighteval

Pull Requests Commits

Merge branch 'main' into mixeval_dive

clefourrier committed 160 days ago

Verified 904448fc

Fix broken link (#1014)

JoelNiklaus committed 160 days ago

Verified b63348d6

fix semaphore calls (#1012)

clefourrier committed 165 days ago

Verified c819e179

clefourrier committed 167 days ago

Verified ba607569

Added litellm model config options and improved `_prepare_max_new_tokens` (#967)

rolshoven committed 168 days ago

Verified e98e4639

Merge branch 'main' into mixeval_dive

clefourrier committed 168 days ago

Verified 5f291a00

Fix None Doc (#1008)

lewtun committed 168 days ago

Verified 7a456fd4

more memory for flow judge

clefourrier committed 170 days ago

32bb552d

stringify model judge + change max_length to what's actually passed instead of setting a bunch of overwrites

clefourrier committed 170 days ago

a83d84ab

likely fix for the GSM+ issue

clefourrier committed 170 days ago

257ff354

eval tracker fix 1

clefourrier committed 170 days ago

2ae58aaf

clefourrier committed 170 days ago

fa3ef629

also debugging the judge

clefourrier committed 170 days ago

213eda86

also debugging the judge

clefourrier committed 170 days ago

9d5bd304

clefourrier committed 170 days ago

e15db12c

Revert extraction setting for `IndicesExtractionConfig` (#998)

cmpatino committed 174 days ago

Verified b1d45e36

Adding New Task SLR-Bench as a Community Task : Scalable Logical Reasoning Benchmark (#983)

Ahmad21Omar committed 179 days ago

Verified c7a063ae

Fix AvgAtK num_samples method (#987)

amstu2 committed 181 days ago

Verified 5137e03f

Fix gsm8k HF repo (#986)

amstu2 committed 181 days ago

Verified 3ec1992b

fx lcb metric (#981)

NathanHB committed 181 days ago

Verified 0b16f8a5

adds `enable_prefix_caching` option to VLLMModelConfig (#945)

GAD-cell committed 181 days ago

Verified 16c26303

Bump for release v0.11.0

NathanHB committed 182 days ago

5a0e8972

Sample to sample compare for integration tests (#977)

NathanHB committed 182 days ago

Verified 35ae2d15

[RFC] Rework the dependencies to be more versatile (#951)

LysandreJik committed 185 days ago

Verified 6ee4ff81

Refine acknowledgements in README.md (#976)

clefourrier committed 186 days ago

Verified e61b868a

Support local GGUF in VLLM and use HF tokenizer #943 (#972)

JIElite committed 186 days ago

Verified d6a65ca5

Add auto tests for metrics (#939)

NathanHB committed 187 days ago

Verified 16318bb4

Add IFBench (#944)

clefourrier committed 188 days ago

Verified 3a71f688

Added `backend_options` parameter to llm judges. (#963)

rolshoven committed 188 days ago

Verified 9ba430f1

Multilingual extractiveness (#956)

rolshoven committed 188 days ago

Verified 2dc17885

Older