huggingface/lighteval

Pull Requests Commits

fixed the maj@n qem being run on the sampe samples. needed to manage the sort and split

clefourrier committed 1 year ago

1e648c41

Merge branch 'main' into quick_fix_vllm

clefourrier committed 1 year ago

e4322f07

clefourrier committed 1 year ago

00bf6867

fix max len management in vllm

clefourrier committed 1 year ago

999b7a68

Misc-multilingual tasks (#339)

hynky1999 committed 1 year ago

Verified b018c9c7

Fix Tokenization + misc fixes (#354)

hynky1999 committed 1 year ago

Verified 1dfd77d1

Selecting tasks using their superset (#308)

hynky1999 committed 1 year ago

Verified 78cda93f

Nathan llm judge quickfix (#350)

NathanHB committed 1 year ago

Verified b8923aa7

Merge branch 'main' into fix341

clefourrier committed 1 year ago

Verified 1694d779

Nathan llm judge quickfix (#348)

NathanHB committed 1 year ago

Verified a6cf5cb3

NathanHB committed 1 year ago

Verified cf40366d

removing helm hellaswag since it was removed from the suite

clefourrier committed 1 year ago

5bc116f7

Merge branch 'main' into fix341

clefourrier committed 1 year ago

Verified 6a50e085

clefourrier committed 1 year ago

0aaf4ee3

clefourrier committed 1 year ago

a441c795

add do_sample to generative tas criteria

clefourrier committed 1 year ago

ffca3eaf

bump lighteval versoin (#328)

NathanHB committed 1 year ago

Verified f89ae20c

split greedy and sampling generative + remove small old helm mechanism

clefourrier committed 1 year ago

06a7c044

refacto judge and add mixeval (#337)

NathanHB committed 1 year ago

Verified 0483bef8

Fixes sampling for vllm when num_samples==1 (#343)

edbeeching committed 1 year ago

Verified 230b31bf

Multilingual General Knowledge tasks (#338)

hynky1999 committed 1 year ago

Verified 9cd26bc7

Multilingual Reading Comprehension tasks (#333)

hynky1999 committed 1 year ago

Verified dd2927aa

Multilingual Hellaswag tasks (#332)

hynky1999 committed 1 year ago

Verified 994fe73c

Fix Metrics import path in community task template file. (#309)

chuandudx committed 1 year ago

Verified 01e7240a

Multilingual COPA tasks (#330)

hynky1999 committed 1 year ago

Verified 7b94a180

Multilingual NLI Tasks (#329)

hynky1999 committed 1 year ago

Verified 551572ae

Allow kwargs for BERTScore compute function and remove unused var (#311)

chuandudx committed 1 year ago

Verified 170ed872

Adds tasks templating (#335)

hynky1999 committed 1 year ago

Verified b5d8fdf4

Class implementations of faithfulness and extractiveness metrics (#323)

chuandudx committed 1 year ago

Verified 5584e23c

Skipping push to hub test (#334)

clefourrier committed 1 year ago

Verified 1f303977

Older