huggingface/lighteval

Pull Requests Commits

fix vlm details

NathanHB committed 1 year ago

5eba9f33

dont log to cli when doing slow tests and log nvidia smi

NathanHB committed 1 year ago

b58b0635

Merge branch 'nathan-add-integration-tests' of github.com:huggingface/lighteval into nathan-add-integration-tests

NathanHB committed 1 year ago

2f683f4a

use math.iscloze for metrics and fix path to vlm details

NathanHB committed 1 year ago

c1af85a3

Update tests/slow_tests/sample_comparison.py

NathanHB committed 1 year ago

4066608f

Update tests/slow_tests/sample_comparison.py

NathanHB committed 1 year ago

d1d556d7

Apply suggestion from @Copilot

NathanHB committed 1 year ago

e6f86b5f

Apply suggestion from @Copilot

NathanHB committed 1 year ago

4dfcc18e

Apply suggestion from @Copilot

NathanHB committed 1 year ago

291178df

Apply suggestion from @Copilot

NathanHB committed 1 year ago

3dd4338e

only compare the text results

NathanHB committed 1 year ago

6b6af7ad

add samples compare for vlm

NathanHB committed 1 year ago

b2cce05f

compare logprobs ranking instead of values

NathanHB committed 1 year ago

24a20c38

modify sample to have temp = 0

NathanHB committed 1 year ago

1b39fcd1

get actual samples

NathanHB committed 1 year ago

ecf14b9a

get actual samples

NathanHB committed 1 year ago

a2d4267b

revert undeed changes

NathanHB committed 1 year ago

88d03926

revert undeed changes

NathanHB committed 1 year ago

cdc9f45d

fix logprobs compares for different harware

NathanHB committed 1 year ago

6068ec89

fix logprobs compares for different harware

NathanHB committed 1 year ago

6eb013c3

Merge branch 'nathan-add-integration-tests' of github.com:huggingface/lighteval into nathan-add-integration-tests

NathanHB committed 1 year ago

8955542d

adding reference details

NathanHB committed 1 year ago

3ae1c4be

NathanHB committed 1 year ago

70eeb9e8

Merge branch 'main' into nathan-add-integration-tests

NathanHB committed 1 year ago

0a0f3cb9

NathanHB committed 1 year ago

3aefbcec

Add auto tests for metrics (#939)

NathanHB committed 1 year ago

9d6b9126

compares sample to sample when doing slow tests

NathanHB committed 1 year ago

0103a989

Add IFBench (#944)

clefourrier committed 1 year ago

46663377

Added `backend_options` parameter to llm judges. (#963)

rolshoven committed 1 year ago

d90e3a5f

Multilingual extractiveness (#956)

rolshoven committed 1 year ago

96c2a4a5

Older