huggingface/lighteval

Pull Requests Commits

Merge branch 'main' into clem_metric_hynek

NathanHB committed 1 year ago

Verified 7fb031b3

Refactoring the few shot management (#272)

clefourrier committed 1 year ago

Verified 24adaa20

Example Configs and Docs (#255)

RohitMidha23 committed 1 year ago

Verified 8883a77f

Remove expensive prediction run during test collection (#279)

hynky1999 committed 1 year ago

Verified 357c75bf

make info loggers dataclass, so that their properties have expected lifetime (#280)

hynky1999 committed 1 year ago

Verified 52a12f98

Small file reorg (only renames/moves) (#271)

clefourrier committed 1 year ago

Verified e6b599a1

Programmatic interface + cleaner management of requests (#269)

clefourrier committed 1 year ago

Verified 9c82fdb8

Add grammar option to generation (#242)

sadra-barikbin committed 1 year ago

Verified e43c9490

comments from review

clefourrier committed 1 year ago

d132b557

Merge branch 'main' into clem_metric_hynek

NathanHB committed 1 year ago

Verified 8c27daa7

returns the results directly because the samples used for multichoices metrics have been popped out

NathanHB committed 1 year ago

32326105

updates ifeval repo (#268)

NathanHB committed 1 year ago

Verified cf87de99

fix the location of tasks list in the readme (#267)

NathanHB committed 1 year ago

Verified 194e742f

Merge branch 'main' into clem_metric_hynek

NathanHB committed 1 year ago

Verified 8646478e

change priority when choosing model dtype (#263)

NathanHB committed 1 year ago

Verified 5c1ae855

Merge branch 'main' into clem_metric_hynek

NathanHB committed 1 year ago

Verified 139426ee

Fix missing json file (#264)

NathanHB committed 1 year ago

Verified 01a5129f

A small improvement in `metrics_sample.py::ROUGE` (#217)

sadra-barikbin committed 1 year ago

Verified b54c1896

Merge branch 'main' into clem_metric_hynek

clefourrier committed 1 year ago

Verified 5409da44

adds llm as judge using transformers (#223)

NathanHB committed 1 year ago

Verified cb32024d

Fix IFEval metric (#259)

lewtun committed 1 year ago

Verified 38c03ad2

clefourrier committed 1 year ago

c1ba8699

clefourrier committed 1 year ago

c05826ba

Fixing issues with multichoice_continuations_start_space - was not parsed properly (#232)

clefourrier committed 1 year ago

Verified 6041e36e

Expose samples via the CLI (#228)

clefourrier committed 1 year ago

Verified cbae17dc

Fix inference endpoint config (#244)

clefourrier committed 1 year ago

Verified f047874a

uses torch compile if provided (#248)

clefourrier committed 1 year ago

Verified 4751027a

udpated piqa (#222)

clefourrier committed 1 year ago

Verified 506abda3

clefourrier committed 1 year ago

Verified db502dde

Removes default bert scorer init (#234)

clefourrier committed 1 year ago

Verified 2b4b637e

Older