Go
Home
Pricing
FAQ
Install
Home
Pricing
FAQ
Install
Login
via GitHub
huggingface/lighteval
Pull Requests
Commits
Open
Closed
Fix CorpusLevelF1Score(None) to report positive-class F1, not max-per-class
#1275 opened 2026-06-25 20:05 by
iamsharduld
Fix MajAtN IndexError when the gold is not the first choice
#1274 opened 2026-06-25 20:03 by
iamsharduld
Fix helm_normalizer mis-scoring numbers (homogeneize before remove_punc)
#1273 opened 2026-06-25 20:01 by
iamsharduld
Add script-aware language tags for FLORES 200
#1272 opened 2026-06-23 14:44 by
goktugozkanmd
Fix sample-cache corruption under accelerate data-parallel
#1271 opened 2026-06-22 14:58 by
shipbehaves
fix(mmlu_pro): correct answer letters in prompt and choices
#1269 opened 2026-06-19 19:31 by
vineethsaivs
feat: add static linter command for custom tasks
#1268 opened 2026-06-19 18:40 by
Kesav2k04
feat: add ID-based few-shot example selection (#634)
#1263 opened 2026-06-16 06:15 by
sohammmmm10
Fix KeyError: 'default' for bigbench tasks with nonexistent split
#1262 opened 2026-06-14 18:56 by
nuthalapativarun
Document unconditioned_query requirement for PMI normalization
#1261 opened 2026-06-14 18:53 by
nuthalapativarun
Fix non-deterministic cache hash for MetricGrouping tasks
#1260 opened 2026-06-14 18:50 by
nuthalapativarun
Sync LightevalTaskConfig docstring with current fields
#1259 opened 2026-06-14 18:44 by
nuthalapativarun
Fix broken default_tasks.py link in quicktour docs
#1258 opened 2026-06-14 18:42 by
nuthalapativarun
Fix LCB grader failing solutions that read sys.stdin.buffer
#1257 opened 2026-06-13 09:37 by
vineethsaivs
fix(lcb): expose sys.stdin.buffer so binary-stdin solutions are graded correctly
#1256 opened 2026-06-13 09:22 by
avalyset
Add AIME26 task
#1254 opened 2026-06-11 15:01 by
Vedant-Agarwal
Enhance model loading and error handling
#1251 opened 2026-06-03 18:54 by
nandeanie
docs: fix reference links
#1250 opened 2026-05-31 11:51 by
charles-openclaw
Add optional juryeval integration for LLM-as-Judge metrics
#1249 opened 2026-05-31 11:11 by
py-ai-dev
Fix flatten_dict crash/wrong key for bare numpy array values
#1247 opened 2026-05-28 21:37 by
Kymi808
chore: enable Dependabot weekly GitHub Actions bumps
dependabot
#1246 opened 2026-05-26 10:32 by
hf-dependantbot-rollout[bot]
Add ArxivRollBench tasks
#1245 opened 2026-05-24 05:34 by
liangzid
feat: implement loglikelihood and loglikelihood_rolling for LiteLLMClient (closes #1093)
#1244 opened 2026-05-21 19:47 by
ALI-AL-MARJANI
Fix callable type hint in parallelism helper
#1239 opened 2026-05-20 04:20 by
GoparapukethaN
fix: guard choices[0] and message=None before content access in llm_as_judge
#1238 opened 2026-05-17 23:34 by
qizwiz
docs: fix custom model examples
#1237 opened 2026-05-15 15:20 by
MukundaKatta
fix: prevent IndexError in Doc.get_golds() for out-of-bounds gold_index
#1236 opened 2026-05-13 02:41 by
AmSach
fix typo
#1235 opened 2026-05-08 11:16 by
fpetrakov
fix: transpose references before passing to sacrebleu in CorpusLevelTranslationMetric
#1234 opened 2026-05-08 01:04 by
jaydenC88
test: style-bot trigger
#1221 opened 2026-05-04 12:51 by
paulinebm
Older