huggingface/lighteval

Pull Requests Commits

lewtun committed 221 days ago

61f62f4e

lewtun committed 221 days ago

c8425d04

lewtun committed 221 days ago

a2e522c0

Fix revision arg for vLLM tokenizer

lewtun committed 221 days ago

fb364804

Added support for quantization in vLLM backend (#690)

SulRash committed 224 days ago

Verified 04a74a28

Fix tqdm logging (#711)

Vectorrent committed 224 days ago

Verified f7392fa0

add livecodebench v6 (#712)

Cppowboy committed 224 days ago

Verified 7477de09

Patch 0.9.1 (#708)

NathanHB committed 229 days ago

Verified c8714563

Bump dev version0.9.1.dev0 (#705)

NathanHB committed 230 days ago

Verified 20cff959

better release notes (#704)

NathanHB committed 230 days ago

Verified 42c9c61a

fix typos (#702)

omahs committed 230 days ago

Verified 77efde06

Update README.md (#703)

NathanHB committed 230 days ago

Verified 9bf210c0

[FIX] Inference providers (#701)

clefourrier committed 230 days ago

Verified 039debce

docs: improve consistency in punctuation of metric list (#605)

mariagrandury committed 230 days ago

Verified 40626e7a

Update split iteration for DynamicBatchingDataset (#684)

qubvel committed 231 days ago

Verified d146f9b1

Fix TransformersModel.from_model() method (#691)

Vectorrent committed 231 days ago

Verified ea729318

Added custom model inference. (#437)

JoelNiklaus committed 231 days ago

Verified e67ed9c9

reorder authors (#699)

NathanHB committed 231 days ago

Verified dfeb2349

Add pass@1 for GPQA-D and MATH-500 (#698)

lewtun committed 231 days ago

Verified d50bc307

Nathan adds wanddb logging (#685)

NathanHB committed 236 days ago

Verified 96e885d4

fix gsm8k metric (#688)

NathanHB committed 236 days ago

Verified c776694a

Lift protobuf restriction (#683)

Vectorrent committed 236 days ago

Verified b07d4189

Fix yourbench custom task (#695)

alozowski committed 236 days ago

Verified 71fec806

fix wrong 'custom_task_directory' in python api doc (#671)

xgwang committed 238 days ago

Verified 534a5b8c

fix slow tests (#689)

NathanHB committed 238 days ago

Verified 818a2cf2

adds wandb loging of metrics (#676)

NathanHB committed 243 days ago

Verified 989f5f55

adds simpleqa (#680)

NathanHB committed 244 days ago

Verified 69931dfd

Add G-Pass@k Metric (#589)

jnanliu committed 244 days ago

Verified 89ba8807

change tokenizer to pad to 'longest' sequence, instead of 'max_length' (#669)

xgwang committed 244 days ago

Verified 88e3a3bc

change inference provider model config to inherit from ModelConfig base class (#678)

NathanHB committed 244 days ago

Verified 5120c586

Older