Go
Home
FAQ
Install
Home
FAQ
Install
Login
via GitHub
huggingface/lighteval
Pull Requests
Commits
Open
Closed
fix: don't expand an exact task name into its own subsets
#1487 opened 2026-11-08 18:28 by
KhalidKhan541
Add HAE-RAE Bench (five original subsets)
#1485 opened 2026-11-06 13:51 by
jhwanseok
fix(metrics): stop counting missing samples as failures in pass@k
#1484 opened 2026-11-06 11:05 by
aidless
fix: compute multiclass F1 averages over all classes
#1482 opened 2026-11-06 08:12 by
aidless
Add a Count column with the evaluated sample count to the markdown results summary
#1480 opened 2026-11-05 13:40 by
amkkoussay
Read numbers grouped with no-break, narrow no-break or thin spaces
#1478 opened 2026-11-05 00:27 by
ChrystianSchutz
Do not let an extracted expression end with an operator (bold answers)
#1477 opened 2026-11-05 00:27 by
ChrystianSchutz
fix(cache): persist regenerated responses after invalid cache entries
#1474 opened 2026-11-03 07:52 by
Cookie0103
Fill in the translation literals for Latvian, Lithuanian, Maltese and Slovenian
#1473 opened 2026-11-03 07:46 by
inchang-ing
fix(litellm): log and back off on non-filter BadRequestError
#1472 opened 2026-11-03 02:23 by
pcbeingused333
fix(hle): scale confidence to 0-1 before computing calibration error
#1470 opened 2026-11-02 12:42 by
damienen
Combine subtask stderrs in quadrature for the _average and all rows
#1468 opened 2026-11-01 22:31 by
Andreas-Hum
Fix double BOS when using chat templates (vllm, transformers, sglang backends)
#1467 opened 2026-11-01 22:13 by
EvanZhuang
fix(cache): stabilize solver task hashes
#1462 opened 2026-11-01 04:23 by
jovial-liu
Fix `Probability` metric to aggregate over gold choices, not all choices
#1461 opened 2026-10-31 06:23 by
Sivadheeraj5356
Limit the memory a generated LiveCodeBench solution can add
#1460 opened 2026-10-29 21:58 by
williamcaban
Fix typos in docs, docstrings, comments and log messages
#1458 opened 2026-10-29 18:53 by
sarankumar1325
Bump doc-builder to get the docs machine-translated
#1457 opened 2026-10-29 12:51 by
mishig25
Strip reasoning section when only the closing `</think>` tag is generated
#1454 opened 2026-10-28 22:56 by
selimfedakar
Fix aimo_progress_prize_1 exact match on integer gold answers
#1452 opened 2026-10-27 09:34 by
mottopanikeiku
fix(metrics): compare mathematical equivalence in Inspect math_scorer (#1430)
#1451 opened 2026-10-24 23:46 by
pfolex
Keep Recall@k tie-breaking consistent with Recall@1
#1450 opened 2026-10-24 22:38 by
yunyv
Fix source offsets in extracted fragment strings
#1449 opened 2026-10-24 05:31 by
yunyv
Skip batch size search when model parallel has several processes
#1447 opened 2026-10-23 05:24 by
chetana1410
Restore TriviaQA prediction normalization
#1446 opened 2026-10-22 16:20 by
mottopanikeiku
Use actual gold labels for TruthfulQA MC2
#1444 opened 2026-10-20 22:45 by
salma2vec
Count repeated tokens in F1 scores
#1443 opened 2026-10-20 22:45 by
salma2vec
Keep explicit model parallel on one multi-GPU machine
#1439 opened 2026-10-20 11:35 by
chetana1410
Fix invalid type annotations (callable, np.array) in the package
#1438 opened 2026-10-20 10:39 by
Michael-Duck
docs: point task guides at paths that exist
#1436 opened 2026-10-20 03:23 by
xlb3055
Newer
Older