llama-cookbook
Eval reproduce recipe using lm-evaluation-harness and our 3.1 evals datasets
#627

Merged

Eval reproduce recipe using lm-evaluation-harness and our 3.1 evals datasets #627

HamidShojanazeri merged 37 commits into main from eval_reproduce

first commit, local not working

9f0acebe

eval working

37be8e99

adding more on README

1f666708

adding some fix

ed201fc8

add result table to README

ff10442d

changed requirement.txt for eval

48ea26cb

add more on README

333aff70

wukaixingxp requested a review from

HamidShojanazeri 1 year ago

wukaixingxp assigned

wukaixingxp 1 year ago

facebook-github-bot added cla signed

fix typo

cf0e715b

refactor folder

307510b8

fix typo and broken links

320f800b

fixed dead link to pass lint check

c3f0dbfe

fix typo

fe2b9f03

HamidShojanazeri commented on 2024-08-12

HamidShojanazeri commented on 2024-08-13

now uses lm_eval cli instead

091f71e8

fix typo

3bee947e

fix dead links and typo

d095cd4c

minor typo fix

9dcbca56

wukaixingxp requested a review from

raghotham 1 year ago

wukaixingxp requested a review from

rohit-ptl 1 year ago

added the issue link back

da9ab4b9

fixing readme

9b8d6aa6

NathanHB commented on 2024-08-16

changed readme and added more comments

eef8b888

formating

38e6de84

added Acknowledgement

178d2d98

nkcheng255 requested a review from

nkcheng255 1 year ago

wukaixingxp requested a review from

init27 1 year ago

restructure readme

a42c0148

fix typo

b9cb3555

Merge branch 'main' into eval_reproduce

c1390e8c

update instruction

19dd9dcd

fixed commit and add nltk download

d691843a

use pip install for lm_eval

d74507ac

minor fix to readme

387fe503

add word to wordlist

e3abfe76

init27 approved these changes on 2024-08-20

wukaixingxp requested a review from

HamidShojanazeri 1 year ago

HamidShojanazeri commented on 2024-08-20

HamidShojanazeri commented on 2024-08-21