huggingface/lighteval

Pull Requests Commits

move prompt function to task files

NathanHB committed 37 days ago

43db078d

rename mmlu prompt functoin

NathanHB committed 37 days ago

8b8b9489

move prompts to respective eval file

NathanHB committed 37 days ago

0d0c47fe

add ifeval french

NathanHB committed 37 days ago

01e73a4d

NathanHB committed 37 days ago

46e6883f

NathanHB committed 37 days ago

70184eab

Fix set using wrong syntax (#1057)

f14-bertolotti committed 37 days ago

Verified cb97d5c8

add a task dump in registry for better documentation of tasks (#1052)

NathanHB committed 38 days ago

Verified b8ccd20c

Remove suites in task configs example and fix task with hf_filters (#1051)

NathanHB committed 38 days ago

Verified 35babcb1

wrong attribute self.k -> self.n (#1049)

f14-bertolotti committed 38 days ago

Verified 6524c6ad

not forcing use_logits at True (#1050)

f14-bertolotti committed 38 days ago

Verified 2247df76

neater bundle and logdir (#1043)

NathanHB committed 38 days ago

Verified cd91dde7

remove suites from serbian_eval.py (#1044)

S-Y-A-N committed 38 days ago

Verified d04e4f9b

put lower bound on typer to use literal type (#1042)

NathanHB committed 44 days ago

Verified 566a7be3

adding lcb to inspect-ai

NathanHB committed 44 days ago

307c10be

remove suites and make fewshot optional (#1038)

NathanHB committed 45 days ago

Verified 31433cc9

run all hf providers with `:all` (#1039)

NathanHB committed 45 days ago

Verified 5b7ca62f

fix `lighteval task inspect` command and tiny bench task (#992)

NathanHB committed 45 days ago

Verified b5cbd91a

add openai and inspect ai lower bound (#1035)

NathanHB committed 46 days ago

Verified 5aa09c5a

Update huggingface-cli login to use newer hf auth login (#1034)

xeophon committed 46 days ago

Verified 97303acf

Fix inspect reasoning effrot (#1033)

NathanHB committed 46 days ago

Verified 17e024bc

adds mmlu-pro (#1031)

NathanHB committed 46 days ago

Verified fa4860f9

Adds inspectai (#1022)

NathanHB committed 47 days ago

Verified 880bebef

Move tasks to individual files (#1016)

NathanHB committed 52 days ago

Verified 3cd31fd1

add translation literals indic (#1015)

rpm000 committed 60 days ago

Verified bf8b5471

Fixing naming for sample evals + adding reqs in aime24 (#989)

clefourrier committed 67 days ago

Verified 161d47cc

added fallback for incomplete configs for vlm models launched as llms (#828)

clefourrier committed 67 days ago

Verified e7d885c3

Fix 999: always provide parameters in the metric name to allow using several combinations (#1017)

clefourrier committed 67 days ago

Verified 70acb852

Fix nltk import failing (#1013)

clefourrier committed 67 days ago

Verified 3af89255

Fixing mixeval (#1006)

clefourrier committed 67 days ago

Verified 090630eb

Newer Older