Benchmark overhaul #41408
remi-or
merged
25 commits into
huggingface:main
from
remi-or:benchmark-various
Big refactor, still classes to move around and script to re-complexify
bf2165e2
Move to streamer, isolate benches, propagate num tokens
5004d3f3
Some refacto
0f2f6a53
Added compile mode to name
1f24b833
Re-order
3223c2e4
Move to dt_tokens
ec65db22
Better format
8c6c1ac9
Fix and disable use_cache by default
43297ae5
Fixed compile and SDPA backend default
7744c03b
Refactor results format
16ee6c09
Added default compile mode
631aad28
Always use cache
cde94a46
Fixed cache and added flex
03f79f6a
Plan for missing modules
5b80bc7c
Experiments: no cg and shuffle
f53a0d7d
Disable compile for FA
650656e7
Remove wall time, add sweep mode, get git commit
680d3da6
Review compliance, start
d2c89aa3
Apply suggestions from code review
fcc467bb
Update benchmark_v2/framework/benchmark_runner.py
0d51db4b
Disable workflow
e30e02a0
Pretty print
23a1a856
Added some pretty names to have pretty logs
bd599daa
Review n2 compliance (end?)
d701ca98
Style and end of PR
400a6165
Login to write a write a comment.
Login via GitHub