huggingface/lighteval

Pull Requests Commits

Apply suggestion from @NathanHB

NathanHB committed 45 days ago

Verified 2a7c7d39

Apply suggestion from @NathanHB

NathanHB committed 45 days ago

Verified a2a8af74

fixing the prompt

NathanHB committed 45 days ago

e4b99dc0

jgyasu committed 85 days ago

Verified 6f57f2cb

[EVAL] BIG-Bench Extra Hard

jgyasu committed 85 days ago

Verified d14da9cb

update hf_revision hash for multilingual hellaswag (#1084)

rpm000 committed 86 days ago

Verified 557b8d55

[DOC] Fix dev dependencies install command (#1085)

jgyasu committed 86 days ago

Verified 6889901c

Update available-tasks.mdx (#1088)

bram-pramono committed 86 days ago

Verified 48dcd834

Add starred attribute to gpqa.py metadata

NathanHB committed 86 days ago

Verified 98ac1bf7

Fix the quickstart description? (#1091)

julien-c committed 92 days ago

Verified 99ef5b98

bumping version

NathanHB committed 96 days ago

58038182

add to inspect (#1065)

NathanHB committed 96 days ago

Verified 9009723b

batched metric was not aggregated properly (#1067)

f14-bertolotti committed 96 days ago

Verified 943c4c37

Add style bot (#1079)

NathanHB committed 99 days ago

Verified 66ce47e3

Add style bot (#1078)

NathanHB committed 99 days ago

Verified a64541c6

Add style bot (#1077)

NathanHB committed 99 days ago

Verified d59ce259

Add style bot and other quality of life (#1076)

NathanHB committed 99 days ago

Verified 7f50228f

[FT] Add `py.typed` to `lighteval` (#1071)

akshathmangudi committed 100 days ago

Verified 14963552

removed duplicate code, useless function, added stronger deletion of items, plus updated the logic in generation size to respect what the user asks (#1073)

clefourrier committed 100 days ago

Verified 2236e17f

Fix task config metric typing to accept Metric enums (#1018)

emmanuel-ferdman committed 100 days ago

Verified 99162f17

Adding AA Omniscience task (#1066)

NathanHB committed 102 days ago

Verified 5425c337

adding satrred tag for frontend

NathanHB committed 102 days ago

d9ea404f

One file one task definition (#1059)

NathanHB committed 103 days ago

Verified babeec92

use correct hf subset for ifbench multiturn (#1061)

sam-paech-liquid committed 103 days ago

Verified ad58fedd

Update LiteLLM configuration for hosted_vllm provider (#1060)

abhiram1809 committed 106 days ago

Verified af6b5b4f

Fix: correct argument order in MajAtN.compute (#1058)

WeiKangda committed 106 days ago

Verified 391d5b40

Fix set using wrong syntax (#1057)

f14-bertolotti committed 107 days ago

Verified cb97d5c8

add a task dump in registry for better documentation of tasks (#1052)

NathanHB committed 108 days ago

Verified b8ccd20c

Remove suites in task configs example and fix task with hf_filters (#1051)

NathanHB committed 108 days ago

Verified 35babcb1

wrong attribute self.k -> self.n (#1049)

f14-bertolotti committed 108 days ago

Verified 6524c6ad

Older