PR #742 Add eval loader to eval script

Add eval loader to eval script #742

aspfohl merged 14 commits into main from anna/evalloader

Add eval loader to eval script

774c62eb

small input tests

6c7af312

aspfohl commented on 2023-11-16

aspfohl requested a review from

bmosaicml 2 years ago

aspfohl requested a review from

dakinggg 2 years ago

aspfohl marked this pull request as ready for review 2 years ago

Merge branch 'main' into anna/evalloader

0797a6c4

dakinggg commented on 2023-11-16

updates

13cbc13e

fix typing and formatting

204d2f7c

fixes, add tests

5b852184

Merge branch 'main' into anna/evalloader

1a0a3da8

remove circular dependency

972ed965

tests pass

98405895

aspfohl requested a review from

dakinggg 2 years ago

aspfohl requested a review from

irenedea 2 years ago

irenedea commented on 2023-11-28

dakinggg commented on 2023-11-28

dakinggg commented on 2023-11-29

nits + small fixes

bc1c7c78

add metrics at the end, refactor to put icl/gauntlet as helpers

1050deff

aspfohl commented on 2023-11-30

Merge branch 'main' into anna/evalloader

beefdc34

NOT

549abb62

dakinggg commented on 2023-11-30

metrics instead of models, add unit tests

084d8b64

aspfohl requested a review from

irenedea 2 years ago

aspfohl requested a review from

dakinggg 2 years ago

dakinggg approved these changes on 2023-11-30

aspfohl merged 31008599 into main 2 years ago

aspfohl deleted the anna/evalloader branch 2 years ago

Reviewers

dakinggg

irenedea

bmosaicml

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

llm-foundry Add eval loader to eval script #742 Merged

Add eval loader to eval script #742

llm-foundry
Add eval loader to eval script
#742

Merged