PR #169 Data split depending on eval params

Data split depending on eval params #169

NathanHB merged 21 commits into main from data_split_depending_on_eval_params

init

b06e18e1

wip

cfdc6ee5

testing how to pad and gather with an added dimension for the num_sam…

98c1c12a

now working, need to check why the metric is not displayed

1549fda2

seems to be working!

8adfc077

add maj at 4 for math with preprocessing

24d46922

Uses a homogeneized system for all greedy evaluations - we can do eva…

3a1c1c4d

edit to prevent sampling for providing too many answers to some metrics

71aa2b88

added some doc

e36d6e09

Merge branch 'main' into add_maj_at_k

e1ea3e2a

neither nanotron nor endpoints models cover sampling atm

4ef4c997

add readme

26c7868d

Now, tasks are sorted by conditions. Need to cut the batch size lower…

edcdc729

clefourrier changed the base branch from main to add_maj_at_k 1 year ago

add mini fix

17607303

fix! indices were incorrect

a480fc0b

clefourrier requested a review from

NathanHB 1 year ago

NathanHB commented on 2024-04-30

clefourrier changed the base branch from add_maj_at_k to main 1 year ago

clefourrier changed the base branch from main to add_maj_at_k 1 year ago

Update src/lighteval/data.py

9b35ff41

clefourrier changed the base branch from add_maj_at_k to main 1 year ago

Apply suggestions from code review

39954943

Merge branch 'main' into data_split_depending_on_eval_params

8a1814b2

clefourrier requested a review from

NathanHB 1 year ago

NathanHB commented on 2024-07-10

clefourrier commented on 2024-07-10

Apply suggestions from code review

5db69e8f

revert change

fadf2bb0

added doc

b6b85b5e

NathanHB approved these changes on 2024-07-11

NathanHB merged 66e6aaeb into main 1 year ago

Reviewers

NathanHB

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

lighteval Data split depending on eval params #169 Merged

Data split depending on eval params #169

lighteval
Data split depending on eval params
#169

Merged