huggingface/lighteval

Pull Requests Commits

sync with selected tasks

Hynek Kydlicek committed 1 year ago

5843948a

lock beleble + add tasks to early signals for fr,sw

Hynek Kydlicek committed 1 year ago

8717de3b

adjust max lengths to new tokenizer

hynky1999 committed 1 year ago

727997db

update generative tasks so taht context comfortably fit with gemma tokenizer

hynky1999 committed 1 year ago

7e7c1192

fix hindi + belebele

hynky1999 committed 1 year ago

8c67560b

correctly tokenize thai

Hynek Kydlicek committed 1 year ago

8992d86f

update early signals based on gemma

Hynek Kydlicek committed 1 year ago

24cc2142

add max length to ensure the prefix fix into 2024 tokens

hynky1999 committed 1 year ago

f2d5318e

Hynek Kydlicek committed 1 year ago

cc04e71a

cmnli + hindi tasks

hynky1999 committed 1 year ago

18ae2808

add swahili early signals

hynky1999 committed 1 year ago

9e64fda4

Hynek Kydlicek committed 1 year ago

bdfaeb45

fix incorrect pair tokenziation

Hynek Kydlicek committed 1 year ago

8ccb8297

remove unused tasks

Hynek Kydlicek committed 1 year ago

f65b9434

codah fix prompt

hynky1999 committed 1 year ago

6eaf7eea

reduce comp intenstity of generational tasks

hynky1999 committed 1 year ago

c54ebc54

- hellaswag for te/th

hynky1999 committed 1 year ago

dccc485b

cosnsitent choices for meta mmlu

hynky1999 committed 1 year ago

f6a12990

add support for native arabic mmlu

hynky1999 committed 1 year ago

7afa54bb

add mkqa to arabic + fix exams

Hynek Kydlicek committed 1 year ago

922deee3

add xcodah + fix ceval with just one answer

Hynek Kydlicek committed 1 year ago

f2ddd940

add ocnli + mkqa

hynky1999 committed 1 year ago

73e7d904

- Add Arabic Belebele

hynky1999 committed 1 year ago

d44b2b89

fix m3exam + ceval, more safe lighteval docs conversion

hynky1999 committed 1 year ago

b36e8f52

swahili m3exam invalid answer fixed

hynky1999 committed 1 year ago

0cd02d2a

add probabilistic metrics

Hynek Kydlicek committed 1 year ago

a2a22df2

add second version of xnli

hynky1999 committed 1 year ago

8a696ca7

fix ceval + agieval prompts with choices

hynky1999 committed 1 year ago

832acf2c

Hynek Kydlicek committed 1 year ago

bbd1e861

support old fsspec

Hynek Kydlicek committed 1 year ago

7f2a6368

Older