huggingface/lighteval

Pull Requests Commits

reorder authors (#699)

NathanHB committed 230 days ago

75fa5a3b

Add pass@1 for GPQA-D and MATH-500 (#698)

lewtun committed 230 days ago

743179a4

Nathan adds wanddb logging (#685)

NathanHB committed 235 days ago

9d329193

fix gsm8k metric (#688)

NathanHB committed 235 days ago

f135563e

Lift protobuf restriction (#683)

Vectorrent committed 235 days ago

758abd6b

Fix yourbench custom task (#695)

alozowski committed 235 days ago

9b0ba57f

fix wrong 'custom_task_directory' in python api doc (#671)

xgwang committed 237 days ago

864fd9a3

fix slow tests (#689)

NathanHB committed 237 days ago

49b3abe0

adds wandb loging of metrics (#676)

NathanHB committed 242 days ago

91244dc1

adds simpleqa (#680)

NathanHB committed 243 days ago

4039557f

Add G-Pass@k Metric (#589)

jnanliu committed 243 days ago

70cf604b

change tokenizer to pad to 'longest' sequence, instead of 'max_length' (#669)

xgwang committed 243 days ago

91f753ae

change inference provider model config to inherit from ModelConfig base class (#678)

NathanHB committed 243 days ago

1c0320cb

hardcode configs to not make lighteval crash if lcb repo unavailable (#677)

NathanHB committed 243 days ago

50e72171

Update quicktour command (#679)

qubvel committed 243 days ago

f96c54a3

Improve test (#674)

qubvel committed 249 days ago

a0240715

Update README.md

NathanHB committed 251 days ago

73bdf062

Nathan unify modelargs (#609)

NathanHB committed 251 days ago

c93da24e

Parse string values for add_special_tokens in vLLM (#598)

eldarkurtic committed 257 days ago

d8bc75eb

update LCB prompt, fix newlines (#645)

rawsh committed 257 days ago

b3cbf3c4

Lighteval math (#630)

HERIUN committed 257 days ago

4fb416a0

Parse seed for vLLM (#602)

eldarkurtic committed 257 days ago

10d9c5c5

fix slow tests by removing HF_TOKEN (#667)

NathanHB committed 257 days ago

8a0df246

Align AIME pass@1 with literature (#666)

lewtun committed 257 days ago

39469877

Fix passing of generation config to main_accelerate (#659)

algo-home committed 257 days ago

3662bd4f

add cot_prompt in vllm (#654)

HERIUN committed 257 days ago

53834914

add uzbek literals (#664)

murodbecks committed 257 days ago

1e91b491

Better CI and tests (#655)

NathanHB committed 257 days ago

c2ff6b2e

add arc agi 2 (#642)

NathanHB committed 258 days ago

c665431d

Use `n=16` samples to estimate `pass@1` for AIME benchmarks (#661)

lewtun committed 258 days ago

8b66e16d

Newer Older