huggingface/lighteval

Pull Requests Commits

suggestion from copilot

NathanHB committed 219 days ago

6ff02c65

Update src/lighteval/models/transformers/transformers_model.py

NathanHB committed 219 days ago

Verified 5eeb8fa5

add way to pass kwargs when loading transformers model

NathanHB committed 219 days ago

a2c4dc79

add way to pass kwargs when loading transformers model

NathanHB committed 219 days ago

31b59c0b

Add MCQ support to Yourbench evaluation (#734)

alozowski committed 219 days ago

Verified 317cb504

Fix task metric type mismatch (#743)

Abelgurung committed 219 days ago

Verified 3bb8a50f

Adds multimodal support and MMMU pro (#675)

NathanHB committed 220 days ago

Verified 1607dc10

Added Flores 200 (#717)

clefourrier committed 220 days ago

Verified 63be4b0f

Update main_endpoint.py (#739)

clefourrier committed 220 days ago

Verified d18f11ab

fix litellm (#736)

NathanHB committed 223 days ago

Verified a5903760

Adds More Generative tasks (#694)

hynky1999 committed 223 days ago

Verified c6d1231f

Update README.md (#733)

clefourrier committed 224 days ago

Verified f684d359

Fix revision arg for vLLM tokenizer (#721)

lewtun committed 224 days ago

Verified d3da6b9b

Added support for quantization in vLLM backend (#690)

SulRash committed 227 days ago

Verified 04a74a28

Fix tqdm logging (#711)

Vectorrent committed 227 days ago

Verified f7392fa0

add livecodebench v6 (#712)

Cppowboy committed 227 days ago

Verified 7477de09

Patch 0.9.1 (#708)

NathanHB committed 232 days ago

Verified c8714563

Bump dev version0.9.1.dev0 (#705)

NathanHB committed 234 days ago

Verified 20cff959

better release notes (#704)

NathanHB committed 234 days ago

Verified 42c9c61a

fix typos (#702)

omahs committed 234 days ago

Verified 77efde06

Update README.md (#703)

NathanHB committed 234 days ago

Verified 9bf210c0

[FIX] Inference providers (#701)

clefourrier committed 234 days ago

Verified 039debce

docs: improve consistency in punctuation of metric list (#605)

mariagrandury committed 234 days ago

Verified 40626e7a

Update split iteration for DynamicBatchingDataset (#684)

qubvel committed 234 days ago

Verified d146f9b1

Fix TransformersModel.from_model() method (#691)

Vectorrent committed 234 days ago

Verified ea729318

Added custom model inference. (#437)

JoelNiklaus committed 234 days ago

Verified e67ed9c9

reorder authors (#699)

NathanHB committed 234 days ago

Verified dfeb2349

Add pass@1 for GPQA-D and MATH-500 (#698)

lewtun committed 235 days ago

Verified d50bc307

Nathan adds wanddb logging (#685)

NathanHB committed 239 days ago

Verified 96e885d4

fix gsm8k metric (#688)

NathanHB committed 239 days ago

Verified c776694a

Older