huggingface/lighteval

Pull Requests Commits

fix main_tasks and ultilingual tasks

NathanHB committed 196 days ago

9a78a820

remove gpqa from tests :)

NathanHB committed 196 days ago

a58ae419

add gpqa to tests

NathanHB committed 196 days ago

e1584e59

fix tests for prompt manager

NathanHB committed 196 days ago

a5400a3f

fix gpqa metric

NathanHB committed 196 days ago

655b9e3a

Merge branch 'nathan-refactor-prompt-building' of github.com:huggingface/lighteval into nathan-refactor-prompt-building

NathanHB committed 197 days ago

2cf7219d

fix gpqa metric

NathanHB committed 197 days ago

c688ebe8

fix chat template, notably with few shots, for apis

clefourrier committed 197 days ago

bd5bab6c

few shot management with instruction out of system prompt

clefourrier committed 197 days ago

3e743366

gpqa extractive match bug

NathanHB committed 197 days ago

5c3be1cc

Fix system prompt concatenated with instruction in chat template (#830)

clefourrier committed 197 days ago

Verified 0ba8812f

Merge branch 'nathan-refactor-prompt-building' of github.com:huggingface/lighteval into nathan-refactor-prompt-building

NathanHB committed 197 days ago

3cd4b51e

NathanHB committed 197 days ago

67cbbd82

Merge branch 'main' into nathan-refactor-prompt-building

NathanHB committed 197 days ago

Verified d88ec32d

Merge branch 'nathan-refactor-prompt-building' of github.com:huggingface/lighteval into nathan-refactor-prompt-building

NathanHB committed 197 days ago

464dcf78

fixes from review

NathanHB committed 197 days ago

6b32de51

fixes from review

NathanHB committed 197 days ago

084d35cd

add TUMLU-mini benchmark, solves #577 (#811)

ceferisbarov committed 197 days ago

Verified 936612cb

Add TranslationLiterals for Language.DANISH (#770)

spyysalo committed 197 days ago

Verified b0c092a1

Fix Typos in Documentation and Task Description (#810)

leopardracer committed 197 days ago

Verified a3725503

Catch ROCM/HIP/AMD oom in `should_reduce_batch_size` (#812)

mcleish7 committed 197 days ago

Verified 95ec6691

Update tests/utils.py

NathanHB committed 197 days ago

Verified 75d00051

NathanHB committed 197 days ago

a74740ed

NathanHB committed 198 days ago

093f465c

NathanHB committed 198 days ago

288f9999

NathanHB committed 198 days ago

566725d9

NathanHB committed 198 days ago

9fe5efba

fix end to end tests to reflect changes in prompt manager

NathanHB committed 198 days ago

1d7a2bbb

NathanHB committed 198 days ago

7b828d45

NathanHB committed 198 days ago

79dc6415

Older