Refacto and remove bloated code #709
NathanHB
merged
82 commits into
main
from
nathan-refactor-prompt-building
refacto prompt building
9684d160
commit
cab4027d
working state for generative metrics (mocked the model)
0b1e213b
working state, removed Metrictype and use_case
723daeb1
working state, all metrics should work, need to unmock the models now
65c2508b
remove unused functions from pipeline
4a16bec2
working for transformer's greedyuntil
29e66575
working on loglikelihood but getting random results
9624de6f
loglikelihood working
747ebf19
transformers model working
0358cb42
remove unused functions
471247ba
all unit tests pass
62658d4b
all unit tests pass
31c80cb8
loglikelihood vllm works
c30ff901
end to end works
6dfc502c
end to end works
d458183a
all tests pass
2913dfd3
all tests pass official
2c4cd69c
sglang works
9675dc35
fixing more models
52dcf117
Merge remote-tracking branch 'origin/main' into nathan-refactor-promp…
9691d335
all tests passing
8102e77e
all models files were reviewed except nanotron
81992a9a
working
c7502d39
load from details working
872a6bea
fix tests
e42d999e
documentation
c0a0b820
documentation
bbd0c115
documentation
55cdfb6c
documentation
3b75ac37
documentation
a8991f82
documentation
6eeee80d
fix linter
b13d8327
fix linter
0879ab43
NathanHB
commented
on 2025-06-16
NathanHB
commented
on 2025-06-16
NathanHB
commented
on 2025-06-16
NathanHB
commented
on 2025-06-16
NathanHB
commented
on 2025-06-16
NathanHB
commented
on 2025-06-16
NathanHB
commented
on 2025-06-16
fixes
3c49c33f
copilot-pull-request-reviewer
commented
on 2025-06-16
Update docs/source/quicktour.mdx
d493c39b
Update src/lighteval/metrics/sample_preparator.py
42c2cc3b
Update src/lighteval/tasks/registry.py
9c2fa74c
fixes from review
83703c9a
Merge branch 'nathan-refactor-prompt-building' of github.com:huggingf…
f7ad781c
fix tests
4c147d85
details
ff4c1b06
fix tests
0178e9a2
fix tests
6e9abcc7
clefourrier
approved these changes
on 2025-06-20
fails when using temp == 0 on sampling tasks
8da10dd2
revert metrics on aime24
bbd53ff6
data // for loglikelihood in transformers model
ac214628
system prompt part of model config
2e423be4
move use-chat-template and system promtp to model config
2125b24f
add docstring to model configs
d98b048c
Merge branch 'main' into nathan-refactor-prompt-building
4f284685
add doc for model responses
9c236376
Merge branch 'nathan-refactor-prompt-building' of github.com:huggingf…
d6069fc7
fix tests
456cd38c
add tests for prompt anager
1213c4ed
fix slow tests
68df230a
fix tests
eb652579
fix tests
79dc6415
fix tests
7b828d45
fix end to end tests to reflect changes in prompt manager
1d7a2bbb
fix tests
9fe5efba
fix tidi
566725d9
last details
288f9999
last details
093f465c
last details
a74740ed
Update tests/utils.py
75d00051
fixes from review
084d35cd
fixes from review
6b32de51
Merge branch 'nathan-refactor-prompt-building' of github.com:huggingf…
464dcf78
Merge branch 'main' into nathan-refactor-prompt-building
d88ec32d
fix tests
67cbbd82
Merge branch 'nathan-refactor-prompt-building' of github.com:huggingf…
3cd4b51e
gpqa extractive match bug
5c3be1cc
few shot management with instruction out of system prompt
3e743366
fix chat template, notably with few shots, for apis
bd5bab6c
fix gpqa metric
c688ebe8
Merge branch 'nathan-refactor-prompt-building' of github.com:huggingf…
2cf7219d
fix gpqa metric
655b9e3a
fix tests for prompt manager
a5400a3f
add gpqa to tests
e1584e59
remove gpqa from tests :)
a58ae419
fix main_tasks and ultilingual tasks
9a78a820
fxmarty-amd
commented
on 2025-07-10
Assignees
No one assigned
Labels
feature
Milestone
No milestone
Login to write a write a comment.
Login via GitHub