Go
Home
FAQ
Install
Home
FAQ
Install
Login
via GitHub
huggingface/lighteval
Pull Requests
Commits
Open
Closed
drop metric: fix exact-match span-count comparison (self-comparison typo)
#1524 opened 2026-11-22 09:56 by
fayaz316
Add KorMedMCQA (Korean medical licensing exam MCQA, 3,009 questions)Team d/kormedmcqa
#1523 opened 2026-11-22 07:01 by
eotjd613
Read a Markdown-bold final answer instead of an earlier number
#1522 opened 2026-11-20 09:13 by
ishan-nerlekar
stop character logprob normalization from dropping choices
#1521 opened 2026-11-20 02:53 by
chessing234
keep math_normalizer from crashing on a truncated command
#1520 opened 2026-11-20 01:57 by
chessing234
Grade the post-processed text in the swiss_legal judge metrics
#1519 opened 2026-11-19 09:44 by
BahaaMattar
Remove capitalized articles in the multilingual normalizer
#1517 opened 2026-11-19 07:41 by
BahaaMattar
Harmonize mixed sampling detail schemas before saving evaluation details
#1515 opened 2026-11-18 20:47 by
GuilhermeLopesBertacini
Fix TypeError when stop_sequences is a tuple or None without chat template
#1514 opened 2026-11-18 20:46 by
GuilhermeLopesBertacini
Add HRM8K (Korean math reasoning, 5 subsets)
#1513 opened 2026-11-18 19:43 by
dw021213
fix(metrics): infer GPassAtK sample count from the largest k
#1512 opened 2026-11-18 18:38 by
linhongyu510
Score arc_agi_2 answers written in the prompt's grid format
#1511 opened 2026-11-18 07:22 by
BahaaMattar
Bump the actions group across 1 directory with 8 updates
dependencies
github_actions
#1509 opened 2026-11-18 00:35 by
dependabot[bot]
docs: fix dead IFEval and multilingual tasks links
#1508 opened 2026-11-17 22:58 by
farisbasha
Make aa_omniscience metrics match the AA-Omniscience paper
#1507 opened 2026-11-17 16:59 by
BahaaMattar
Keep long_horizon_execution turns aligned when a turn has no answer tag
#1505 opened 2026-11-17 15:35 by
BahaaMattar
Add IFEval-Ko (Korean IFEval, 342 prompts)
#1503 opened 2026-11-17 07:09 by
SubJeeLee
Add Kazakh translation literals for multilingual prompts
#1502 opened 2026-11-16 17:34 by
nazek52
Add CLIcK (Korean culture and language, 11 categories)
#1501 opened 2026-11-15 09:33 by
ojm6135
Fix or remove tasks whose Hub subsets don't exist (storycloze, xcopa:en, wikifact, mlmm_hellaswag is/no)
#1500 opened 2026-11-13 10:38 by
Zephyr4772
Fix Czech word tokenizer: spaCy language code is cs, not cz
#1499 opened 2026-11-12 20:17 by
Zephyr4772
Depend on langcodes[data]: multilingual tasks fail with ModuleNotFoundError: language_data
#1498 opened 2026-11-12 20:14 by
Zephyr4772
Fix Windows crash and lost details: percent-encode task names in cache and details paths
#1497 opened 2026-11-12 19:26 by
Zephyr4772
Add a file backend to score stored responses
#1496 opened 2026-11-12 16:34 by
Rutvij97
Read user config, task and fixture files as UTF-8 (non-ASCII system prompts crash on Windows)
#1495 opened 2026-11-12 09:55 by
Zephyr4772
Fix timeout decorator on Windows (math metrics scored every answer as wrong)
#1494 opened 2026-11-12 09:27 by
Zephyr4772
Keep the evaluated doc out of its own few-shot examples
#1492 opened 2026-11-11 04:43 by
David-Wu1119
Fix HLE judge metric crashing when metrics are computed
#1491 opened 2026-11-11 04:43 by
David-Wu1119
Allow selecting few-shot examples by explicit ids
#1489 opened 2026-11-10 22:23 by
amalsaidov
fix(ci): surface failures in the vLLM nightly workflow
#1488 opened 2026-11-09 12:02 by
voropaevv
Older