huggingface/lighteval

Pull Requests Commits

Fix vLLM doc (#912)

lewtun committed 130 days ago

e7d5cf2c

Fix log system prompt (#907)

clefourrier committed 133 days ago

2020bf63

Moved some files (#905)

clefourrier committed 134 days ago

1a3061be

Debug log likelihood evals which are broken for accelerate (#901)

clefourrier committed 135 days ago

f470f75e

Number of fixes to run accelerate evaluations (#898)

clefourrier committed 136 days ago

a23bf95e

Update transformers and vllm dependencies (#899)

clefourrier committed 136 days ago

d70aa141

Added post processing (for reasoning tokens) to pipeline (#882)

clefourrier committed 137 days ago

0f82f65f

fix path concatenation (#891)

clefourrier committed 137 days ago

71f22fe8

Fix DP>1 & TP>1 evals with vllm (#841)

NouamaneTazi committed 140 days ago

c7fcf3a3

Adds continuous batching (#850)

NathanHB committed 140 days ago

7ee7d421

Fix typo: CONCURENT_CALLS -> CONCURRENT_CALLS (#884)

muupan committed 140 days ago

28d76538

Fixed typo in Python API documentation (#862)

dtung8068 committed 140 days ago

c6c81839

Automatically infer whether to use a chat template or not instead of using kwargs (#885)

clefourrier committed 140 days ago

db13bf08

Pin transforers version to fix broken test suite due to mismatch with vllm (#886)

clefourrier committed 141 days ago

0432812f

Remove datasets 4 from pyproject (#863)

clefourrier committed 155 days ago

6fdfa49b

fix: update python api user docs (#784)

ryantzr1 committed 172 days ago

57fd4a40

Update extractive_match_utils.py for words where `:` is preceded by a space (#831)

clefourrier committed 176 days ago

14bd119e

Complete TranslationLiterals for Language.ESTONIAN (#779)

spyysalo committed 176 days ago

9ec1ae15

Fix GPQA and index extractive metric (#829)

clefourrier committed 176 days ago

7fe173af

Icelandic addition (#775)

joenaess committed 177 days ago

94af3c06

Refacto and remove bloated code (#709)

NathanHB committed 177 days ago

cfaea69b

Fix system prompt concatenated with instruction in chat template (#830)

clefourrier committed 178 days ago

2ca8b772

add TUMLU-mini benchmark, solves #577 (#811)

ceferisbarov committed 178 days ago

44c8d900

Add TranslationLiterals for Language.DANISH (#770)

spyysalo committed 178 days ago

39a7ae1e

Fix Typos in Documentation and Task Description (#810)

leopardracer committed 178 days ago

a0aac004

Catch ROCM/HIP/AMD oom in `should_reduce_batch_size` (#812)

mcleish7 committed 179 days ago

e0202529

Flores-200 combinations->permutations (#820)

anton-l committed 179 days ago

8d78817a

set default temperature to 0 in generation config (#814)

NathanHB committed 182 days ago

b64db65a

Add org_to_bill parameter to documentation (#781)

tfrere committed 185 days ago

a8f22515

Add Bulgarian and Macedonian literals (#769)

dianaonutu committed 196 days ago

e23edd31

Newer Older