huggingface/lighteval

Pull Requests Commits

Merge branch 'main' into nathan-convert-to-inspect

NathanHB committed 34 days ago

Verified 0b81dc90

NathanHB committed 34 days ago

6c00e572

NathanHB committed 34 days ago

5f1cbc47

remove LETTER_INDEX

NathanHB committed 34 days ago

55c1e77e

use correct hf subset for ifbench multiturn (#1061)

sam-paech-liquid committed 34 days ago

Verified ad58fedd

prompt function names end with _prompt

NathanHB committed 37 days ago

97d9680d

NathanHB committed 37 days ago

700524be

prompt function names end with _prompt

NathanHB committed 37 days ago

a898375d

Update LiteLLM configuration for hosted_vllm provider (#1060)

abhiram1809 committed 37 days ago

Verified af6b5b4f

Fix: correct argument order in MajAtN.compute (#1058)

WeiKangda committed 37 days ago

Verified 391d5b40

move prompt function to task files

NathanHB committed 37 days ago

0f49df68

move prompt function to task files

NathanHB committed 38 days ago

43db078d

rename mmlu prompt functoin

NathanHB committed 38 days ago

8b8b9489

move prompts to respective eval file

NathanHB committed 38 days ago

0d0c47fe

add ifeval french

NathanHB committed 38 days ago

01e73a4d

NathanHB committed 38 days ago

46e6883f

NathanHB committed 38 days ago

70184eab

Fix set using wrong syntax (#1057)

f14-bertolotti committed 38 days ago

Verified cb97d5c8

add a task dump in registry for better documentation of tasks (#1052)

NathanHB committed 39 days ago

Verified b8ccd20c

Remove suites in task configs example and fix task with hf_filters (#1051)

NathanHB committed 39 days ago

Verified 35babcb1

wrong attribute self.k -> self.n (#1049)

f14-bertolotti committed 39 days ago

Verified 6524c6ad

not forcing use_logits at True (#1050)

f14-bertolotti committed 39 days ago

Verified 2247df76

neater bundle and logdir (#1043)

NathanHB committed 39 days ago

Verified cd91dde7

remove suites from serbian_eval.py (#1044)

S-Y-A-N committed 39 days ago

Verified d04e4f9b

put lower bound on typer to use literal type (#1042)

NathanHB committed 45 days ago

Verified 566a7be3

adding lcb to inspect-ai

NathanHB committed 45 days ago

307c10be

remove suites and make fewshot optional (#1038)

NathanHB committed 46 days ago

Verified 31433cc9

run all hf providers with `:all` (#1039)

NathanHB committed 46 days ago

Verified 5b7ca62f

fix `lighteval task inspect` command and tiny bench task (#992)

NathanHB committed 46 days ago

Verified b5cbd91a

add openai and inspect ai lower bound (#1035)

NathanHB committed 47 days ago

Verified 5aa09c5a

Older