Add maj@k metric #158
NathanHB
merged
16 commits into
main
from
add_maj_at_k
init
b06e18e1
wip
cfdc6ee5
testing how to pad and gather with an added dimension for the num_sam…
98c1c12a
lewtun
commented
on 2024-04-18
lewtun
commented
on 2024-04-18
now working, need to check why the metric is not displayed
1549fda2
seems to be working!
8adfc077
add maj at 4 for math with preprocessing
24d46922
NathanHB
commented
on 2024-04-20
NathanHB
dismissed these changes
on 2024-04-21
Uses a homogeneized system for all greedy evaluations - we can do eva…
3a1c1c4d
edit to prevent sampling for providing too many answers to some metrics
71aa2b88
added some doc
e36d6e09
Merge branch 'main' into add_maj_at_k
e1ea3e2a
clefourrier
commented
on 2024-04-22
neither nanotron nor endpoints models cover sampling atm
4ef4c997
add readme
26c7868d
NathanHB
commented
on 2024-04-22
NathanHB
commented
on 2024-04-22
NathanHB
commented
on 2024-04-22
NathanHB
approved these changes
on 2024-04-22
Merge branch 'main' into add_maj_at_k
e35a6f49
Update src/lighteval/metrics/metrics_sample.py
39cf6765
added review change
5a6988ef
Merge branch 'main' into add_maj_at_k
1431283a
Login to write a write a comment.