huggingface/lighteval

Pull Requests Commits

fix vlm details

NathanHB committed 90 days ago

5eba9f33

dont log to cli when doing slow tests and log nvidia smi

NathanHB committed 90 days ago

b58b0635

Merge branch 'nathan-add-integration-tests' of github.com:huggingface/lighteval into nathan-add-integration-tests

NathanHB committed 90 days ago

2f683f4a

use math.iscloze for metrics and fix path to vlm details

NathanHB committed 90 days ago

c1af85a3

Update tests/slow_tests/sample_comparison.py

NathanHB committed 90 days ago

4066608f

Update tests/slow_tests/sample_comparison.py

NathanHB committed 90 days ago

d1d556d7

Apply suggestion from @Copilot

NathanHB committed 90 days ago

e6f86b5f

Apply suggestion from @Copilot

NathanHB committed 90 days ago

4dfcc18e

Apply suggestion from @Copilot

NathanHB committed 90 days ago

291178df

Apply suggestion from @Copilot

NathanHB committed 90 days ago

3dd4338e

only compare the text results

NathanHB committed 90 days ago

6b6af7ad

add samples compare for vlm

NathanHB committed 90 days ago

b2cce05f

compare logprobs ranking instead of values

NathanHB committed 90 days ago

24a20c38

modify sample to have temp = 0

NathanHB committed 90 days ago

1b39fcd1

get actual samples

NathanHB committed 90 days ago

ecf14b9a

get actual samples

NathanHB committed 90 days ago

a2d4267b

revert undeed changes

NathanHB committed 91 days ago

88d03926

revert undeed changes

NathanHB committed 91 days ago

cdc9f45d

fix logprobs compares for different harware

NathanHB committed 91 days ago

6068ec89

fix logprobs compares for different harware

NathanHB committed 92 days ago

6eb013c3

Merge branch 'nathan-add-integration-tests' of github.com:huggingface/lighteval into nathan-add-integration-tests

NathanHB committed 92 days ago

8955542d

adding reference details

NathanHB committed 92 days ago

3ae1c4be

NathanHB committed 92 days ago

70eeb9e8

Merge branch 'main' into nathan-add-integration-tests

NathanHB committed 92 days ago

0a0f3cb9

NathanHB committed 92 days ago

3aefbcec

Add auto tests for metrics (#939)

NathanHB committed 92 days ago

9d6b9126

compares sample to sample when doing slow tests

NathanHB committed 93 days ago

0103a989

Add IFBench (#944)

clefourrier committed 93 days ago

46663377

Added `backend_options` parameter to llm judges. (#963)

rolshoven committed 93 days ago

d90e3a5f

Multilingual extractiveness (#956)

rolshoven committed 93 days ago

96c2a4a5

Older