huggingface/lighteval

Pull Requests Commits

Only the main process writes the sample cache under data-parallel (#1271)

shipbehaves committed 8 days ago

Verified 64f4f5ae

add move_trailing_context_space flag for answer-only bits-per-byte (#1264)

kashif committed 14 days ago

Verified 95ad3bcb

fix rolling perplexity on the transformers backend (#1252)

kashif committed 28 days ago

Verified 78dbee22

Add multilingual task loading to vLLM (#1248)

JoelNiklaus committed 39 days ago

Verified 4d470292

Add LEXam legal exam benchmark to Swiss legal evals (#1243)

JoelNiklaus committed 39 days ago

Verified 9e427838

Add Pillow dependency (#1242)

JoelNiklaus committed 43 days ago

Verified 89f9395b

Legal NLP tasks on Swiss data (#1032)

rolshoven committed 48 days ago

Verified 8d29839e

[CI] Bump style-bot to hardened TOCTOU-fix SHA (#1233)

paulinebm committed 63 days ago

Verified 3fd15266

Fix/bump style bot sha (#1232)

paulinebm committed 63 days ago

Verified 3e75d1f7

[CI] Bump style-bot to hardened TOCTOU-fix SHA (#1230)

paulinebm committed 63 days ago

Verified ba82a295

Fix/bump style bot sha (#1229)

paulinebm committed 63 days ago

Verified 8aac2b78

Fix/bump style bot sha (#1228)

paulinebm committed 63 days ago

Verified a67c5576

Fix/bump style bot sha (#1227)

paulinebm committed 63 days ago

Verified 4ea6bce1

Fix/bump style bot sha (#1226)

paulinebm committed 63 days ago

Verified 2c7d1c2e

Fix/bump style bot sha (#1225)

paulinebm committed 63 days ago

Verified 5cf8bd99

Fix/bump style bot sha (#1223)

paulinebm committed 63 days ago

Verified 6b9c415c

[CI] Bump style-bot-action to hardened SHA (TOCTOU fix) (#1222)

paulinebm committed 63 days ago

Verified 8e25dcfd

test: point style-bot to TOCTOU fix SHA (#1220)

paulinebm committed 64 days ago

Verified 726807db

chore: bump doc-builder SHA for PR upload workflow (#1213)

rtrompier committed 83 days ago

Verified 10b9104e

Add support for vllm >= 0.19.0 (#1211)

lewtun committed 85 days ago

Verified 34889df3

🔒 Pin GitHub Actions to commit SHAs (#1201)

paulinebm committed 91 days ago

Verified e274b378

Add test on main branch of vllm (#1175)

NathanHB committed 125 days ago

Verified 33acf35f

Upgrade vLLM from 0.10.1.1 to 0.14.1 (#1173)

NathanHB committed 125 days ago

Verified 0a74a170

add eval results tip (#1126)

burtenshaw committed 167 days ago

Verified 06aee5b2

Fix typo in few_shots_select option error message. ('fbalanced' -> 'balanced') (#1111)

jayminban committed 174 days ago

Verified 6ce93d53

NathanHB committed 174 days ago

Verified f8888587

long horizon execution (#1119)

NathanHB committed 174 days ago

Verified 61c547b8

Mathvista (#1118)

NathanHB committed 175 days ago

Verified e7048c30

refactor: add formatted response to litellm (#1116)

pjavanrood committed 175 days ago

Verified 845c9899

multi challenge (#1120)

NathanHB committed 175 days ago

Verified d9a9401c

Older