huggingface/lighteval

Pull Requests Commits

adds enable thinkknig to model args

NathanHB committed 195 days ago

fb8d3770

adds enable thinkknig to model args

NathanHB committed 195 days ago

ce242ef3

Trackio when available (#930)

NathanHB committed 196 days ago

Verified 0da24778

Upgrade to datasets 4.0 (#924)

NathanHB committed 196 days ago

Verified e53587ef

fix context size check in sglang model (#787)

xgwang committed 196 days ago

Verified 63424f4e

Update translation_literals.py - add romanian (#923)

dianaonutu committed 197 days ago

Verified bab9f6db

Reasoning tags removal is true by default (#925)

clefourrier committed 198 days ago

Verified 4e1bb325

fixes from_model function and adds tests (#921)

NathanHB committed 198 days ago

Verified 4d2bf42c

Restore `avg@4` and `avg@8` for GPQA-D (#922)

lewtun committed 201 days ago

Verified 458a3e96

Added ability to configure concurrent_requests in litellm_model.py (#911)

dameikle committed 202 days ago

Verified b0edbc34

Use lower bound for httpx (#920)

lewtun committed 203 days ago

Verified 76e5affe

making inspect working again (#887)

uralik committed 203 days ago

Verified af52de69

Trying to fix xet bug in test suite (#917)

clefourrier committed 205 days ago

Verified ecef2c66

Fix path in cache (#915)

clefourrier committed 205 days ago

Verified a88a5269

Adds the option to override the chat template (#914)

clefourrier committed 205 days ago

Verified 02bf1355

Update dep (#913)

clefourrier committed 205 days ago

Verified af211932

Caching samples PR (#909)

clefourrier committed 205 days ago

Verified df3a82d9

Fix vLLM doc (#912)

lewtun committed 205 days ago

Verified bfa60767

Fix log system prompt (#907)

clefourrier committed 209 days ago

Verified 7693a0fd

Moved some files (#905)

clefourrier committed 210 days ago

Verified d0cd4c91

Debug log likelihood evals which are broken for accelerate (#901)

clefourrier committed 210 days ago

Verified ea1dd184

Number of fixes to run accelerate evaluations (#898)

clefourrier committed 211 days ago

Verified 64f93b0e

Update transformers and vllm dependencies (#899)

clefourrier committed 211 days ago

Verified 865335e4

Added post processing (for reasoning tokens) to pipeline (#882)

clefourrier committed 212 days ago

Verified d7beacbc

fix path concatenation (#891)

clefourrier committed 212 days ago

Verified 994e9e07

Fix DP>1 & TP>1 evals with vllm (#841)

NouamaneTazi committed 215 days ago

Verified 24895519

Adds continuous batching (#850)

NathanHB committed 215 days ago

Verified 99bfd9f2

Fix typo: CONCURENT_CALLS -> CONCURRENT_CALLS (#884)

muupan committed 215 days ago

Verified 1404ba1d

Fixed typo in Python API documentation (#862)

dtung8068 committed 215 days ago

Verified 3b1126ad

Automatically infer whether to use a chat template or not instead of using kwargs (#885)

clefourrier committed 216 days ago

Verified e7d70902

Older