huggingface/lighteval

Pull Requests Commits

suggestion from copilot

NathanHB committed 241 days ago

6ff02c65

Update src/lighteval/models/transformers/transformers_model.py

NathanHB committed 241 days ago

Verified 5eeb8fa5

add way to pass kwargs when loading transformers model

NathanHB committed 241 days ago

a2c4dc79

add way to pass kwargs when loading transformers model

NathanHB committed 241 days ago

31b59c0b

Add MCQ support to Yourbench evaluation (#734)

alozowski committed 241 days ago

Verified 317cb504

Fix task metric type mismatch (#743)

Abelgurung committed 241 days ago

Verified 3bb8a50f

Adds multimodal support and MMMU pro (#675)

NathanHB committed 242 days ago

Verified 1607dc10

Added Flores 200 (#717)

clefourrier committed 242 days ago

Verified 63be4b0f

Update main_endpoint.py (#739)

clefourrier committed 242 days ago

Verified d18f11ab

fix litellm (#736)

NathanHB committed 245 days ago

Verified a5903760

Adds More Generative tasks (#694)

hynky1999 committed 245 days ago

Verified c6d1231f

Update README.md (#733)

clefourrier committed 246 days ago

Verified f684d359

Fix revision arg for vLLM tokenizer (#721)

lewtun committed 246 days ago

Verified d3da6b9b

Added support for quantization in vLLM backend (#690)

SulRash committed 249 days ago

Verified 04a74a28

Fix tqdm logging (#711)

Vectorrent committed 249 days ago

Verified f7392fa0

add livecodebench v6 (#712)

Cppowboy committed 249 days ago

Verified 7477de09

Patch 0.9.1 (#708)

NathanHB committed 254 days ago

Verified c8714563

Bump dev version0.9.1.dev0 (#705)

NathanHB committed 255 days ago

Verified 20cff959

better release notes (#704)

NathanHB committed 255 days ago

Verified 42c9c61a

fix typos (#702)

omahs committed 255 days ago

Verified 77efde06

Update README.md (#703)

NathanHB committed 255 days ago

Verified 9bf210c0

[FIX] Inference providers (#701)

clefourrier committed 255 days ago

Verified 039debce

docs: improve consistency in punctuation of metric list (#605)

mariagrandury committed 255 days ago

Verified 40626e7a

Update split iteration for DynamicBatchingDataset (#684)

qubvel committed 256 days ago

Verified d146f9b1

Fix TransformersModel.from_model() method (#691)

Vectorrent committed 256 days ago

Verified ea729318

Added custom model inference. (#437)

JoelNiklaus committed 256 days ago

Verified e67ed9c9

reorder authors (#699)

NathanHB committed 256 days ago

Verified dfeb2349

Add pass@1 for GPQA-D and MATH-500 (#698)

lewtun committed 256 days ago

Verified d50bc307

Nathan adds wanddb logging (#685)

NathanHB committed 261 days ago

Verified 96e885d4

fix gsm8k metric (#688)

NathanHB committed 261 days ago

Verified c776694a

Older