huggingface/lighteval

Pull Requests Commits

Merge branch 'main' into add_maj_at_k

NathanHB committed 1 year ago

Verified 1431283a

add 'cite as' section in readme (#178)

NathanHB committed 1 year ago

Verified 98060932

Fix prompt format german rag community task (#171)

jphme committed 1 year ago

Verified af35e886

added review change

clefourrier committed 1 year ago

5a6988ef

Update src/lighteval/metrics/metrics_sample.py

clefourrier committed 1 year ago

Verified 39cf6765

Merge branch 'main' into add_maj_at_k

NathanHB committed 1 year ago

Verified e35a6f49

no function we actually use should be named prompt_fn (#168)

clefourrier committed 1 year ago

Verified 11b48333

clefourrier committed 1 year ago

26c7868d

neither nanotron nor endpoints models cover sampling atm

clefourrier committed 1 year ago

4ef4c997

Merge branch 'main' into add_maj_at_k

clefourrier committed 1 year ago

Verified e1ea3e2a

clefourrier committed 1 year ago

e36d6e09

edit to prevent sampling for providing too many answers to some metrics

clefourrier committed 1 year ago

71aa2b88

Uses a homogeneized system for all greedy evaluations - we can do evals in one single step

clefourrier committed 1 year ago

3a1c1c4d

[MATH] Fix MATH normalization (#162)

lewtun committed 1 year ago

Verified fc428f60

add maj at 4 for math with preprocessing

clefourrier committed 1 year ago

24d46922

seems to be working!

clefourrier committed 1 year ago

8adfc077

now working, need to check why the metric is not displayed

clefourrier committed 1 year ago

1549fda2

testing how to pad and gather with an added dimension for the num_samples

clefourrier committed 1 year ago

98c1c12a

clefourrier committed 1 year ago

cfdc6ee5

clefourrier committed 1 year ago

b06e18e1

Nathan add logging to metrics (#157)

NathanHB committed 1 year ago

Verified 6a48e4e7

Fix nanotron models input size bug (#156)

clefourrier committed 1 year ago

Verified 6ea07cd7

CLI list tasks (#142)

DimbyTa committed 1 year ago

Verified f28a270a

Add llm as judge in metrics (#146)

NathanHB committed 1 year ago

Verified 7f6f8077

Added Namespace parameter for InferenceEndpoints, added option for passing model config directly (#147)

shaltielshmid committed 1 year ago

Verified 23987aa7

Add `Ger-RAG-eval`tasks. (#149)

PhilipMay committed 1 year ago

Verified 5a732e6c

Add fun widgets to the README (#145)

clefourrier committed 1 year ago

Verified 996b5042

Add config files for models (#131)

clefourrier committed 1 year ago

Verified efa17331

Fix max_length type in base_model.py (#138)

csarron committed 1 year ago

Verified ce087cda

NathanHB committed 1 year ago

Verified 017581f7

Older