huggingface/lighteval

Pull Requests Commits

pair_wise -> pairwise

hynky1999 committed 1 year ago

2a00c533

finally do the tokenization correctly

hynky1999 committed 1 year ago

a3edfb3f

Merge branch 'main' into tokenization_fixes

NathanHB committed 1 year ago

Verified fd306e95

Selecting tasks using their superset (#308)

hynky1999 committed 1 year ago

Verified 78cda93f

fix test + implementation

hynky1999 committed 1 year ago

bb36ef03

fix loggers with multimetric, fix pariwise tokenization + fallback for non-pairwise, add pairwise to vllm, use multiprocess fordataset loading

hynky1999 committed 1 year ago

814d06f4

Nathan llm judge quickfix (#350)

NathanHB committed 1 year ago

Verified b8923aa7

Nathan llm judge quickfix (#348)

NathanHB committed 1 year ago

Verified a6cf5cb3

NathanHB committed 1 year ago

Verified cf40366d

bump lighteval versoin (#328)

NathanHB committed 1 year ago

Verified f89ae20c

refacto judge and add mixeval (#337)

NathanHB committed 1 year ago

Verified 0483bef8

Fixes sampling for vllm when num_samples==1 (#343)

edbeeching committed 1 year ago

Verified 230b31bf

Multilingual General Knowledge tasks (#338)

hynky1999 committed 1 year ago

Verified 9cd26bc7

Multilingual Reading Comprehension tasks (#333)

hynky1999 committed 1 year ago

Verified dd2927aa

Multilingual Hellaswag tasks (#332)

hynky1999 committed 1 year ago

Verified 994fe73c

Fix Metrics import path in community task template file. (#309)

chuandudx committed 1 year ago

Verified 01e7240a

Multilingual COPA tasks (#330)

hynky1999 committed 1 year ago

Verified 7b94a180

Multilingual NLI Tasks (#329)

hynky1999 committed 1 year ago

Verified 551572ae

Allow kwargs for BERTScore compute function and remove unused var (#311)

chuandudx committed 1 year ago

Verified 170ed872

Adds tasks templating (#335)

hynky1999 committed 1 year ago

Verified b5d8fdf4

Class implementations of faithfulness and extractiveness metrics (#323)

chuandudx committed 1 year ago

Verified 5584e23c

Skipping push to hub test (#334)

clefourrier committed 1 year ago

Verified 1f303977

Support for multilingual generative metrics (#293)

hynky1999 committed 1 year ago

Verified 1bb0dff1

Math normalization: do not crash on invalid format (#331)

guipenedo committed 1 year ago

Verified 7e80aaa1

readme rewrite (#327)

NathanHB committed 1 year ago

Verified ba202480

[FIX] Fixes vllm backend (#317)

NathanHB committed 1 year ago

Verified 4b06b944

fix doc in readme (#303)

NathanHB committed 1 year ago

Verified 7295c78f

skip tests if secrets not provided (#304)

hynky1999 committed 1 year ago

Verified 6b943ecb

fix readme doc about custom tasks (#300)

NathanHB committed 1 year ago

Verified 3dd7d500

Fixes bug: You can't create a model without either a list of model_args or a model_config_path when model_config_path was submited (#298)

NathanHB committed 1 year ago

Verified 8d833565

Older