Benchmarking V2: framework impl (#40486)
* Start revamping benchmarking
* Start refactoring benchmarking
* Use Pandas for CSV
* import fix
* Remove benchmark files
* Remove sample data
* Address review comments
* Benchmarking v2
* Fix llama bench parameters
* Working checkpoint
* Readme touchups
* Remove unnecessary test
* Massage the framework a bit
* Small cleanup
* Remove unnecessary flushes
* Remove references to mock benchmark
* Take commit ID from CLI
* Address review comments
* Use Events for thread comms
* Tiny renaming
