Automatically collect best_config
Summary:
Previously, users would need to specify their own implementation of the `best_config` metric, and they would need to manually map the benchmark name to the respective config. We now monkeypatch Autotuner.run to check if a benchmark target has run an autotuned Triton kernel. This allows us to automatically log the `best_config` as a BenchmarkOperator-level generic metric.
I've also removed `dump_autotuner_best_config` in favor of using Triton's `Config.all_kwargs()` method. This should ensure that any new config parameters get automatically serialized (assuming upstream maintains this).
One minor downside is that we now have a `best_config` column regardless of whether we are benchmarking a Triton kernel. (Previously, some operator implementations would specify `skip_baseline=True` to skip this column just for the baseline, but in general we don't know if the baseline is a Triton kernel.) Possible follow-up would be to omit columns from the CSV if all the values in it are `None`.
Reviewed By: xuzhao9
Differential Revision: D59176133
fbshipit-source-id: d55be479671424a32879afba9dbf3c2808ec52e6
