Add extended task for LiveCodeBench codegeneration #548
NathanHB
merged
23 commits into
huggingface:main
from
plaguss:lcb-codegeneration
Add draft for livecodebench code generation
9369a375
Add extra argument version_tag
2001e7b7
Fix import name
fece5528
Remove unused typed dict
e46fc2a5
Checkpoint, not ready yet, try simplifying code running and reuse pas…
6a3c0077
Add notes for expected values
987eb2a3
Pass version tag to downloader
42fb0f57
NathanHB
commented
on 2025-02-11
NathanHB
commented
on 2025-02-11
NathanHB
commented
on 2025-02-11
NathanHB
commented
on 2025-02-11
Modify helper module and remove dataset version tag
b700dc49
Remove version_tag
29b2bbe3
Initial version for lcb:codegeneration
a60e6620
Remove outdated argument docs
05a7f019
Remove hardcoded system prompt and pass it via arg
deea663a
Merge branch 'main' into lcb-codegeneration
a2863f93
Add kwargs to allow passing other arguments
44f45b5f
Make generic function to parse the metric name and obtain the number …
127b4cdb
Change metric name to make it more informative
a372e057
Add experimental way of passing the number of samples for a metric fr…
53ab4176
Add more processes to run the tests
f6a7c4f3
Allow reading the generation parameters from the CLI
d6abcd07
Update parsing arguments from CLI
158d660d
Remove dead code and fix test value
54fa0320
Fix num_samples update
4a0fe89b
Add docs for the new metric_options
f945fdf4
Login to write a write a comment.
Login via GitHub