huggingface/lighteval

Pull Requests Commits

NathanHB committed 115 days ago

a8fa3c55

NathanHB committed 115 days ago

b40fc032

NathanHB committed 115 days ago

90a9d9a7

NathanHB committed 115 days ago

9ec50d08

use another file for latest tests

NathanHB committed 115 days ago

cd67bdd2

use another file for latest tests

NathanHB committed 115 days ago

fe100f9b

Merge branch 'main' into nathan-run-against-main

NathanHB committed 115 days ago

e67f8a8b

fix vllm model with main modifs from vllm

NathanHB committed 116 days ago

12313677

NathanHB committed 117 days ago

ac4052d5

NathanHB committed 117 days ago

da51c945

run only when merged on main and every week

NathanHB committed 117 days ago

6516b15e

run only when merged on main and every week

NathanHB committed 117 days ago

b462a1a5

run only when merged on main and every week

NathanHB committed 117 days ago

aee72646

run slow tests aginst vllm and transformers main

NathanHB committed 117 days ago

ba17deaa

fx lcb metric (#981)

NathanHB committed 117 days ago

Verified 0b16f8a5

adds `enable_prefix_caching` option to VLLMModelConfig (#945)

GAD-cell committed 117 days ago

Verified 16c26303

Bump for release v0.11.0

NathanHB committed 118 days ago

5a0e8972

Sample to sample compare for integration tests (#977)

NathanHB committed 118 days ago

Verified 35ae2d15

[RFC] Rework the dependencies to be more versatile (#951)

LysandreJik committed 121 days ago

Verified 6ee4ff81

Refine acknowledgements in README.md (#976)

clefourrier committed 122 days ago

Verified e61b868a

Support local GGUF in VLLM and use HF tokenizer #943 (#972)

JIElite committed 122 days ago

Verified d6a65ca5

Add auto tests for metrics (#939)

NathanHB committed 123 days ago

Verified 16318bb4

Add IFBench (#944)

clefourrier committed 124 days ago

Verified 3a71f688

Added `backend_options` parameter to llm judges. (#963)

rolshoven committed 124 days ago

Verified 9ba430f1

Multilingual extractiveness (#956)

rolshoven committed 124 days ago

Verified 2dc17885

Fixed bug that prevented the metrics from being mixed (some batched and others not batched) (#958)

rolshoven committed 125 days ago

Verified 96a6882e

Update readme (#961)

NathanHB committed 125 days ago

Verified c409f3f9

set upper bound on vllm version (#964)

NathanHB committed 125 days ago

Verified fc50ca09

MMLU Redux and Fixing the caching (#883)

clefourrier committed 129 days ago

Verified eda7eed3

Update docs and enforces google's docstring style (#941)

NathanHB committed 132 days ago

Verified 7ed2636e

Older