PR #3545 batched : add bench tool

batched : add bench tool #3545

ggerganov merged 7 commits into master from batched-bench

batched : add bench tool

bf06d654

batched : minor fix table

7438728d

ggerganov added need feedback

batched-bench : add readme + n_kv_max is now configurable

026bb1b1

ggerganov force pushed to 026bb1b1 2 years ago

Merge branch 'master' into batched-bench

76e17f8d

batched-bench : init warm-up batch

c062ffd1

batched-bench : pass custom set of PP, TG and PL

daeb834d

batched-bench : add mmq CLI arg

2fcdf869

ggerganov merged 8c70a5ff into master 2 years ago

Reviewers

No reviews

Assignees

No one assigned

Labels

need feedback

Milestone

No milestone