huggingface/lighteval

Pull Requests Commits

Update split iteration for DynamicBatchingDataset (#684)

qubvel committed 226 days ago

Verified d146f9b1

Fix TransformersModel.from_model() method (#691)

Vectorrent committed 226 days ago

Verified ea729318

Added custom model inference. (#437)

JoelNiklaus committed 226 days ago

Verified e67ed9c9

reorder authors (#699)

NathanHB committed 226 days ago

Verified dfeb2349

Add pass@1 for GPQA-D and MATH-500 (#698)

lewtun committed 227 days ago

Verified d50bc307

remove dead code

clefourrier committed 231 days ago

c0c9e3a4

lol you need to iterate on the generator to actually get it to generate

clefourrier committed 231 days ago

17969d47

Nathan adds wanddb logging (#685)

NathanHB committed 231 days ago

Verified 96e885d4

fix gsm8k metric (#688)

NathanHB committed 231 days ago

Verified c776694a

Lift protobuf restriction (#683)

Vectorrent committed 231 days ago

Verified b07d4189

updated docstring

clefourrier committed 232 days ago

d3dddb5d

removing dead code from PR

clefourrier committed 232 days ago

35a9fd53

Update src/lighteval/models/vllm/vllm_model.py

clefourrier committed 232 days ago

Verified d4c0ece2

Fix yourbench custom task (#695)

alozowski committed 232 days ago

Verified 71fec806

now we manage async models at the pipeline level to have a single event loop

clefourrier committed 232 days ago

94ad7123

remove useless waits

clefourrier committed 232 days ago

50d9338e

cleaner shutdown

clefourrier committed 232 days ago

6e6c8e41

moved adapter/delta cleanup to avoid messing up the pipeline

clefourrier committed 233 days ago

bc577b50

clefourrier committed 233 days ago

b184f522

clefourrier committed 233 days ago

8be711f3

clefourrier committed 233 days ago

5950eb0f

fix wrong 'custom_task_directory' in python api doc (#671)

xgwang committed 234 days ago

Verified 534a5b8c

fix slow tests (#689)

NathanHB committed 234 days ago

Verified 818a2cf2

adds wandb loging of metrics (#676)

NathanHB committed 238 days ago

Verified 989f5f55

adds simpleqa (#680)

NathanHB committed 239 days ago

Verified 69931dfd

Add G-Pass@k Metric (#589)

jnanliu committed 240 days ago

Verified 89ba8807

change tokenizer to pad to 'longest' sequence, instead of 'max_length' (#669)

xgwang committed 240 days ago

Verified 88e3a3bc

change inference provider model config to inherit from ModelConfig base class (#678)

NathanHB committed 240 days ago

Verified 5120c586

hardcode configs to not make lighteval crash if lcb repo unavailable (#677)

NathanHB committed 240 days ago

Verified 8e84e1ba

Update quicktour command (#679)

qubvel committed 240 days ago

Verified 1ae06841

Newer Older