huggingface/lighteval

Pull Requests Commits

removed tokenizer logging

clefourrier committed 230 days ago

14ac6904

added option to bill to org in inference providers

clefourrier committed 230 days ago

90cb89ad

Update split iteration for DynamicBatchingDataset (#684)

qubvel committed 231 days ago

Verified d146f9b1

Fix TransformersModel.from_model() method (#691)

Vectorrent committed 231 days ago

Verified ea729318

Added custom model inference. (#437)

JoelNiklaus committed 231 days ago

Verified e67ed9c9

reorder authors (#699)

NathanHB committed 231 days ago

Verified dfeb2349

Add pass@1 for GPQA-D and MATH-500 (#698)

lewtun committed 231 days ago

Verified d50bc307

Nathan adds wanddb logging (#685)

NathanHB committed 236 days ago

Verified 96e885d4

fix gsm8k metric (#688)

NathanHB committed 236 days ago

Verified c776694a

Lift protobuf restriction (#683)

Vectorrent committed 236 days ago

Verified b07d4189

Fix yourbench custom task (#695)

alozowski committed 236 days ago

Verified 71fec806

fix wrong 'custom_task_directory' in python api doc (#671)

xgwang committed 238 days ago

Verified 534a5b8c

fix slow tests (#689)

NathanHB committed 238 days ago

Verified 818a2cf2

adds wandb loging of metrics (#676)

NathanHB committed 243 days ago

Verified 989f5f55

adds simpleqa (#680)

NathanHB committed 244 days ago

Verified 69931dfd

Add G-Pass@k Metric (#589)

jnanliu committed 244 days ago

Verified 89ba8807

change tokenizer to pad to 'longest' sequence, instead of 'max_length' (#669)

xgwang committed 244 days ago

Verified 88e3a3bc

change inference provider model config to inherit from ModelConfig base class (#678)

NathanHB committed 244 days ago

Verified 5120c586

hardcode configs to not make lighteval crash if lcb repo unavailable (#677)

NathanHB committed 244 days ago

Verified 8e84e1ba

Update quicktour command (#679)

qubvel committed 244 days ago

Verified 1ae06841

Improve test (#674)

qubvel committed 250 days ago

Verified 2ef1bc53

Update README.md

NathanHB committed 252 days ago

Verified 4ab19e80

Nathan unify modelargs (#609)

NathanHB committed 252 days ago

Verified feb8b706

Parse string values for add_special_tokens in vLLM (#598)

eldarkurtic committed 257 days ago

Verified cc95ff27

update LCB prompt, fix newlines (#645)

rawsh committed 257 days ago

Verified d5efccf3

Lighteval math (#630)

HERIUN committed 257 days ago

Verified 2c3fdf36

Parse seed for vLLM (#602)

eldarkurtic committed 257 days ago

Verified 9ea7d6af

fix slow tests by removing HF_TOKEN (#667)

NathanHB committed 257 days ago

Verified 582d927f

Align AIME pass@1 with literature (#666)

lewtun committed 258 days ago

Verified bb14995c

Fix passing of generation config to main_accelerate (#659)

algo-home committed 258 days ago

Verified 632b69b8

Older