huggingface/lighteval

Pull Requests Commits

Merge branch 'main' into nathan-adds-wanddb-logging

NathanHB committed 266 days ago

Verified 73f04f67

fix slow tests (#689)

NathanHB committed 266 days ago

Verified 818a2cf2

Merge branch 'main' into nathan-adds-wanddb-logging

NathanHB committed 271 days ago

Verified e417509f

adds wandb loging of metrics

NathanHB committed 271 days ago

03156fa4

adds wandb loging of metrics (#676)

NathanHB committed 271 days ago

Verified 989f5f55

adds simpleqa (#680)

NathanHB committed 272 days ago

Verified 69931dfd

Add G-Pass@k Metric (#589)

jnanliu committed 272 days ago

Verified 89ba8807

change tokenizer to pad to 'longest' sequence, instead of 'max_length' (#669)

xgwang committed 272 days ago

Verified 88e3a3bc

change inference provider model config to inherit from ModelConfig base class (#678)

NathanHB committed 272 days ago

Verified 5120c586

hardcode configs to not make lighteval crash if lcb repo unavailable (#677)

NathanHB committed 272 days ago

Verified 8e84e1ba

Update quicktour command (#679)

qubvel committed 272 days ago

Verified 1ae06841

adds wandb loging of metrics

NathanHB committed 277 days ago

01c28ffd

adds wandb loging of metrics

NathanHB committed 277 days ago

75481b5d

Improve test (#674)

qubvel committed 278 days ago

Verified 2ef1bc53

adds wandb loging of metrics

NathanHB committed 279 days ago

39c84f5d

adds wandb login of metrics

NathanHB committed 279 days ago

a0323027

Update README.md

NathanHB committed 280 days ago

Verified 4ab19e80

Nathan unify modelargs (#609)

NathanHB committed 280 days ago

Verified feb8b706

Parse string values for add_special_tokens in vLLM (#598)

eldarkurtic committed 286 days ago

Verified cc95ff27

update LCB prompt, fix newlines (#645)

rawsh committed 286 days ago

Verified d5efccf3

Lighteval math (#630)

HERIUN committed 286 days ago

Verified 2c3fdf36

Parse seed for vLLM (#602)

eldarkurtic committed 286 days ago

Verified 9ea7d6af

fix slow tests by removing HF_TOKEN (#667)

NathanHB committed 286 days ago

Verified 582d927f

Align AIME pass@1 with literature (#666)

lewtun committed 286 days ago

Verified bb14995c

Fix passing of generation config to main_accelerate (#659)

algo-home committed 286 days ago

Verified 632b69b8

add cot_prompt in vllm (#654)

HERIUN committed 286 days ago

Verified f8ebe644

add uzbek literals (#664)

murodbecks committed 286 days ago

Verified dceb39d6

Better CI and tests (#655)

NathanHB committed 286 days ago

Verified 840a6a31

add arc agi 2 (#642)

NathanHB committed 287 days ago

Verified 75f44298

Use `n=16` samples to estimate `pass@1` for AIME benchmarks (#661)

lewtun committed 287 days ago

Verified f3639c67

Older