huggingface/lighteval

Pull Requests Commits

Fixed bug that prevented the metrics from being mixed (some batched and others not batched) (#958)

rolshoven committed 94 days ago

80770da8

Update readme (#961)

NathanHB committed 94 days ago

2b0fffed

set upper bound on vllm version (#964)

NathanHB committed 94 days ago

8e9b0782

MMLU Redux and Fixing the caching (#883)

clefourrier committed 98 days ago

a757a23e

Update docs and enforces google's docstring style (#941)

NathanHB committed 101 days ago

7013452e

Registry refactorisation (#937)

clefourrier committed 108 days ago

f9e6bef4

reduces cli args redundancy (#932)

NathanHB committed 112 days ago

9ec9236f

Fixe Sampling Metrics and Evals (#938)

NathanHB committed 113 days ago

8d056971

Fix `TGI` (Text Generation Inference) Endpoint Inference and TGI JSON Grammar Generation (#502)

cpcdoy committed 115 days ago

5dd44eae

Parametrizing the sampling evals from the CLI (#926)

clefourrier committed 115 days ago

247336ba

Save reasoning_content from litellm as details (#929)

muupan committed 118 days ago

44dd7f89

Add Filipino Benchmark Community Tasks (#852)

ljvmiranda921 committed 118 days ago

4b02d2ef

Test for the xet hub team (#928)

clefourrier committed 118 days ago

ff0bc3db

fix tasks list (#906)

alielfilali01 committed 118 days ago

de332bb2

Fix #878: load tasks before models (#931)

clefourrier committed 118 days ago

82d73992

Trackio when available (#930)

NathanHB committed 120 days ago

ff9b7a6c

Upgrade to datasets 4.0 (#924)

NathanHB committed 120 days ago

0f98275e

fix context size check in sglang model (#787)

xgwang committed 120 days ago

03314e06

Update translation_literals.py - add romanian (#923)

dianaonutu committed 120 days ago

2d0f34e6

Reasoning tags removal is true by default (#925)

clefourrier committed 121 days ago

1e08477c

fixes from_model function and adds tests (#921)

NathanHB committed 122 days ago

3f2aaa51

Restore `avg@4` and `avg@8` for GPQA-D (#922)

lewtun committed 125 days ago

95646ba9

Added ability to configure concurrent_requests in litellm_model.py (#911)

dameikle committed 126 days ago

11ee72b4

Use lower bound for httpx (#920)

lewtun committed 127 days ago

ff6e1353

making inspect working again (#887)

uralik committed 127 days ago

c28177d7

Trying to fix xet bug in test suite (#917)

clefourrier committed 129 days ago

76a73177

Fix path in cache (#915)

clefourrier committed 129 days ago

0d1a2a83

Adds the option to override the chat template (#914)

clefourrier committed 129 days ago

2feb00d9

Update dep (#913)

clefourrier committed 129 days ago

9b733cba

Caching samples PR (#909)

clefourrier committed 129 days ago

6ceb2e60

Newer Older