huggingface/lighteval

Pull Requests Commits

clefourrier committed 125 days ago

ce0015e2

we mock iep model creation

clefourrier committed 125 days ago

35b7e94e

added doc page to tree

clefourrier committed 125 days ago

730f226a

Merge branch 'main' into test_caching

clefourrier committed 125 days ago

Verified f4d5b145

we separate the interface function from the actual logic in models, and wrap up tests

clefourrier committed 125 days ago

d2ff97c0

Fix vLLM doc (#912)

lewtun committed 125 days ago

Verified bfa60767

clefourrier committed 125 days ago

348b7d23

cleanup, some things hanging when being removed from the refacto - no idea why the imports did not fail

clefourrier committed 125 days ago

19fa6d13

remove useless parameter

clefourrier committed 125 days ago

ba776ab8

removed loglikelihood single tokens since it was removed everywhere in theory - added docs on caching

clefourrier committed 125 days ago

3a9d1fbe

we now actively load cached samples after processing the needed items

clefourrier committed 128 days ago

94e31f18

cache management is working well with no DP for accelerate - need to 1)test with DP 2) add a system where we load cached samples in mem *after* processing the other items

clefourrier committed 128 days ago

03eb193c

Merge branch 'main' into test_caching

clefourrier committed 128 days ago

187aaa72

fixed caching system for predictions, and better logging

clefourrier committed 128 days ago

205ab1e5

added caching to transformers

clefourrier committed 128 days ago

39c03e7c

Merge branch 'main' into test_caching

clefourrier committed 128 days ago

76c8b6e8

Fix log system prompt (#907)

clefourrier committed 128 days ago

Verified 7693a0fd

Moved some files (#905)

clefourrier committed 129 days ago

Verified d0cd4c91

Debug log likelihood evals which are broken for accelerate (#901)

clefourrier committed 130 days ago

Verified ea1dd184

Number of fixes to run accelerate evaluations (#898)

clefourrier committed 131 days ago

Verified 64f93b0e

Update transformers and vllm dependencies (#899)

clefourrier committed 131 days ago

Verified 865335e4

Added post processing (for reasoning tokens) to pipeline (#882)

clefourrier committed 132 days ago

Verified d7beacbc

fix path concatenation (#891)

clefourrier committed 132 days ago

Verified 994e9e07

Fix DP>1 & TP>1 evals with vllm (#841)

NouamaneTazi committed 135 days ago

Verified 24895519

Adds continuous batching (#850)

NathanHB committed 135 days ago

Verified 99bfd9f2

Fix typo: CONCURENT_CALLS -> CONCURRENT_CALLS (#884)

muupan committed 135 days ago

Verified 1404ba1d

Fixed typo in Python API documentation (#862)

dtung8068 committed 135 days ago

Verified 3b1126ad

Automatically infer whether to use a chat template or not instead of using kwargs (#885)

clefourrier committed 135 days ago

Verified e7d70902

clefourrier committed 135 days ago

aaae0ea2

clefourrier committed 136 days ago

e0cd1330

Older