huggingface/lighteval

Pull Requests Commits

Merge branch 'main' into clem_details

clefourrier committed 1 year ago

Verified 746208cb

Fix inference endpoint config (#244)

clefourrier committed 1 year ago

Verified f047874a

uses torch compile if provided (#248)

clefourrier committed 1 year ago

Verified 4751027a

Merge branch 'main' into clem_details

clefourrier committed 1 year ago

Verified cbb57db6

clefourrier committed 1 year ago

5dc3273e

udpated piqa (#222)

clefourrier committed 1 year ago

Verified 506abda3

clefourrier committed 1 year ago

Verified db502dde

Merge branch 'main' into clem_details

NathanHB committed 1 year ago

Verified e816cdc3

Removes default bert scorer init (#234)

clefourrier committed 1 year ago

Verified 2b4b637e

Merge branch 'main' into clem_details

clefourrier committed 1 year ago

Verified 65fef954

remove latex writer since we don't use it (#231)

clefourrier committed 1 year ago

Verified 86fbe641

Update issue templates (#235)

clefourrier committed 1 year ago

Verified 003f05e2

Merge branch 'main' into clem_details

clefourrier committed 1 year ago

Verified 8038c8cb

Fix a tiny bug in DROP metric (#229)

sadra-barikbin committed 1 year ago

Verified 66ed7a28

Quantization related issues (#224)

clefourrier committed 1 year ago

Verified 44f9a461

Make evaluator invariant of input request type order (#215)

sadra-barikbin committed 1 year ago

Verified 951cd5b5

added nice printing for the config and more options for task display

clefourrier committed 1 year ago

f892248b

Fix _init_max_length in base_model.py (#185)

gucci-j committed 1 year ago

Verified d43c9a32

added cache at all levels

clefourrier committed 1 year ago

3a288953

add examples of samples (does not include few shot), and add robustness to logger

clefourrier committed 1 year ago

2a6da982

launch lighteval using `lighteval --args` (#152)

NathanHB committed 1 year ago

Verified 4550cb7e

Add metrics as functions (#214)

clefourrier committed 1 year ago

Verified aaf7e8af

should fix most inference endpoints issues of version config (#226)

clefourrier committed 1 year ago

Verified 733257ff

Data split depending on eval params (#169)

clefourrier committed 1 year ago

Verified 66e6aaeb

Now only uses functions for prompt definition (#213)

clefourrier committed 1 year ago

Verified 4651531e

Use only dataclasses for task init (#212)

clefourrier committed 1 year ago

Verified 3aaec227

Fix a few typos in `metrics.py` (#218)

sadra-barikbin committed 1 year ago

Verified 3f909509

Homogeneize logging system (#150)

clefourrier committed 1 year ago

Verified ac57b78e

Adds a dummy/random model for baseline init (#220)

guipenedo committed 1 year ago

Verified 70f7fc6d

Fix the bug (#216)

sadra-barikbin committed 1 year ago

Verified 0528f297

Older