huggingface/lighteval

Pull Requests Commits

Merge branch 'main' into prob_metrics_and_more_norms

hynky1999 committed 1 year ago

Verified 8b35ca79

hynky1999 committed 1 year ago

9de0a39e

fix nanotron (#283)

NathanHB committed 1 year ago

Verified cdeb6c2d

Update src/lighteval/tasks/lighteval_task.py

hynky1999 committed 1 year ago

Verified d99b64b6

Update src/lighteval/metrics/sample_preparator.py

NathanHB committed 1 year ago

Verified d98abebd

hynky1999 committed 1 year ago

3f4ae478

rework perplexity routing, move probabilities to target_perplexity

hynky1999 committed 1 year ago

0dd7e6c8

Update tests/metrics/test_metric_requests.py

hynky1999 committed 1 year ago

20513e23

Update src/lighteval/tasks/requests.py

hynky1999 committed 1 year ago

00bd2ecf

Update src/lighteval/metrics/__init__.py

hynky1999 committed 1 year ago

9790c73e

Update src/lighteval/logging/info_loggers.py

NathanHB committed 1 year ago

8910ad53

fix metric category separation of requests

hynky1999 committed 1 year ago

49585246

hynky1999 committed 1 year ago

5cb13dbb

nits + fix tests

hynky1999 committed 1 year ago

659907ca

resolving gh comments

hynky1999 committed 1 year ago

9443be86

better docs for uncoditioned_query

hynky1999 committed 1 year ago

3c40051b

simplify normalization name, fix bug with ignore space

hynky1999 committed 1 year ago

78e8e7f4

Hynek Kydlicek committed 1 year ago

80b4f012

formatting + nits

hynky1999 committed 1 year ago

974e0dc0

Add logic for handli PMI + add tests for requests

hynky1999 committed 1 year ago

c11b4229

add new acci + prob metrics

hynky1999 committed 1 year ago

53b56c39

Refactoring the few shot management (#272)

clefourrier committed 1 year ago

Verified 24adaa20

Example Configs and Docs (#255)

RohitMidha23 committed 1 year ago

Verified 8883a77f

Remove expensive prediction run during test collection (#279)

hynky1999 committed 1 year ago

Verified 357c75bf

make info loggers dataclass, so that their properties have expected lifetime (#280)

hynky1999 committed 1 year ago

Verified 52a12f98

Small file reorg (only renames/moves) (#271)

clefourrier committed 1 year ago

Verified e6b599a1

Programmatic interface + cleaner management of requests (#269)

clefourrier committed 1 year ago

Verified 9c82fdb8

Add grammar option to generation (#242)

sadra-barikbin committed 1 year ago

Verified e43c9490

updates ifeval repo (#268)

NathanHB committed 1 year ago

Verified cf87de99

fix the location of tasks list in the readme (#267)

NathanHB committed 1 year ago

Verified 194e742f

Older