huggingface/lighteval

Pull Requests Commits

raise instead of assert

NathanHB committed 123 days ago

2ebeddbf

revert registry.py

NathanHB committed 123 days ago

4f85560f

revert main_tasks.py

NathanHB committed 123 days ago

9dbe9ddf

NathanHB committed 123 days ago

00ce40a8

fixing custom task loading to be compatible with datasets 4.0

NathanHB committed 124 days ago

a0258c6f

fixing tasks for datasets 4.0

NathanHB committed 124 days ago

c65a4acf

remove trust remote code for datasets in multilingual tasks

NathanHB committed 128 days ago

6761528f

remove trust remote code for datasets in extended tasks

NathanHB committed 130 days ago

96bca064

remove trust remote code for datasets and bump datasets to 4.0

NathanHB committed 130 days ago

5765c740

making inspect working again (#887)

uralik committed 130 days ago

Verified af52de69

Trying to fix xet bug in test suite (#917)

clefourrier committed 132 days ago

Verified ecef2c66

Fix path in cache (#915)

clefourrier committed 132 days ago

Verified a88a5269

Adds the option to override the chat template (#914)

clefourrier committed 132 days ago

Verified 02bf1355

Update dep (#913)

clefourrier committed 132 days ago

Verified af211932

Caching samples PR (#909)

clefourrier committed 132 days ago

Verified df3a82d9

Fix vLLM doc (#912)

lewtun committed 132 days ago

Verified bfa60767

Fix log system prompt (#907)

clefourrier committed 135 days ago

Verified 7693a0fd

Moved some files (#905)

clefourrier committed 136 days ago

Verified d0cd4c91

Debug log likelihood evals which are broken for accelerate (#901)

clefourrier committed 137 days ago

Verified ea1dd184

Number of fixes to run accelerate evaluations (#898)

clefourrier committed 138 days ago

Verified 64f93b0e

Update transformers and vllm dependencies (#899)

clefourrier committed 138 days ago

Verified 865335e4

Added post processing (for reasoning tokens) to pipeline (#882)

clefourrier committed 139 days ago

Verified d7beacbc

fix path concatenation (#891)

clefourrier committed 139 days ago

Verified 994e9e07

Fix DP>1 & TP>1 evals with vllm (#841)

NouamaneTazi committed 142 days ago

Verified 24895519

Adds continuous batching (#850)

NathanHB committed 142 days ago

Verified 99bfd9f2

Fix typo: CONCURENT_CALLS -> CONCURRENT_CALLS (#884)

muupan committed 142 days ago

Verified 1404ba1d

Fixed typo in Python API documentation (#862)

dtung8068 committed 142 days ago

Verified 3b1126ad

Automatically infer whether to use a chat template or not instead of using kwargs (#885)

clefourrier committed 142 days ago

Verified e7d70902

Pin transforers version to fix broken test suite due to mismatch with vllm (#886)

clefourrier committed 143 days ago

Verified bc5e450a

Remove datasets 4 from pyproject (#863)

clefourrier committed 157 days ago

Verified d805f9fa

Older