huggingface/lighteval

Pull Requests Commits

lewtun committed 356 days ago

61f62f4e

lewtun committed 356 days ago

c8425d04

lewtun committed 356 days ago

a2e522c0

Fix revision arg for vLLM tokenizer

lewtun committed 356 days ago

fb364804

Added support for quantization in vLLM backend (#690)

SulRash committed 359 days ago

Verified 04a74a28

Fix tqdm logging (#711)

Vectorrent committed 359 days ago

Verified f7392fa0

add livecodebench v6 (#712)

Cppowboy committed 359 days ago

Verified 7477de09

Patch 0.9.1 (#708)

NathanHB committed 364 days ago

Verified c8714563

Bump dev version0.9.1.dev0 (#705)

NathanHB committed 1 year ago

Verified 20cff959

better release notes (#704)

NathanHB committed 1 year ago

Verified 42c9c61a

fix typos (#702)

omahs committed 1 year ago

Verified 77efde06

Update README.md (#703)

NathanHB committed 1 year ago

Verified 9bf210c0

[FIX] Inference providers (#701)

clefourrier committed 1 year ago

Verified 039debce

docs: improve consistency in punctuation of metric list (#605)

mariagrandury committed 1 year ago

Verified 40626e7a

Update split iteration for DynamicBatchingDataset (#684)

qubvel committed 1 year ago

Verified d146f9b1

Fix TransformersModel.from_model() method (#691)

Vectorrent committed 1 year ago

Verified ea729318

Added custom model inference. (#437)

JoelNiklaus committed 1 year ago

Verified e67ed9c9

reorder authors (#699)

NathanHB committed 1 year ago

Verified dfeb2349

Add pass@1 for GPQA-D and MATH-500 (#698)

lewtun committed 1 year ago

Verified d50bc307

Nathan adds wanddb logging (#685)

NathanHB committed 1 year ago

Verified 96e885d4

fix gsm8k metric (#688)

NathanHB committed 1 year ago

Verified c776694a

Lift protobuf restriction (#683)

Vectorrent committed 1 year ago

Verified b07d4189

Fix yourbench custom task (#695)

alozowski committed 1 year ago

Verified 71fec806

fix wrong 'custom_task_directory' in python api doc (#671)

xgwang committed 1 year ago

Verified 534a5b8c

fix slow tests (#689)

NathanHB committed 1 year ago

Verified 818a2cf2

adds wandb loging of metrics (#676)

NathanHB committed 1 year ago

Verified 989f5f55

adds simpleqa (#680)

NathanHB committed 1 year ago

Verified 69931dfd

Add G-Pass@k Metric (#589)

jnanliu committed 1 year ago

Verified 89ba8807

change tokenizer to pad to 'longest' sequence, instead of 'max_length' (#669)

xgwang committed 1 year ago

Verified 88e3a3bc

change inference provider model config to inherit from ModelConfig base class (#678)

NathanHB committed 1 year ago

Verified 5120c586

Older