Fix skipped compiled model is Dynamo benchmarks (#195703)
Summary:
FIXES #195696
Fixes a bug in the PT2 HF LLM benchmarks where the compiled model isn't used as intended. This effectively means we were ignoring our compile settings such as backend, and only using the HF default to compile for generation and not prefill.
The compiled model was being ignored in generate. The reason we see speedup is because of another bug where we forget to disable Transformers default compile settings in the generation config, so the decode is compiled, but the prefill is still fully eager. This can be seen in running with `--backend=eager`, which actually still resulted in a speedup. Aditionally to reuse the graphs compiled during warmup properly and make sure the compiled forward is used both prefill and decode, this PR directly uses `model.forward` in a couple of callsites. Without this I found that the compiled model wasn't being invoked.
Good new is the ground truth compile figures are better than were being reported (at least on qwen 0.6B)
| State | Backend | Speedup | Events | Report |
|---|---|---|---|---|
| Before | Inductor | 1.683× | 1,293 | [tlparse](https://manifold.edge.x2p.facebook.net/v0/read/tree/logs/arshzahed/4a2b107a-c6c6-478d-93e0-ddd4a40d49b0/custom/index.html?bucketName=tlparse_reports&apiKey=tlparse_reports-key&withPayload=1&timeoutMsec=10000) |
| Before | Eager | 1.770× | 1,293 | [tlparse](https://manifold.edge.x2p.facebook.net/v0/read/tree/logs/arshzahed/171a5642-1b08-4eb1-bcf4-db6260116132/custom/index.html?bucketName=tlparse_reports&apiKey=tlparse_reports-key&withPayload=1&timeoutMsec=10000) |
| After | Inductor | 3.171× | 3,698 | [tlparse](https://manifold.edge.x2p.facebook.net/v0/read/tree/logs/arshzahed/984810af-c93b-4121-9746-fe6a058c555d/custom/index.html?bucketName=tlparse_reports&apiKey=tlparse_reports-key&withPayload=1&timeoutMsec=10000) |
| After | Eager | 1.108× | 3,527 | [tlparse](https://manifold.edge.x2p.facebook.net/v0/read/tree/logs/arshzahed/e952ece6-3f8d-4406-b491-7b6442e99bf2/custom/index.html?bucketName=tlparse_reports&apiKey=tlparse_reports-key&withPayload=1&timeoutMsec=10000) |
We may want to backfill benchmark results across nightlies or releases so we have a better sense of trajectory.
X-link: https://github.com/pytorch/pytorch/pull/195703
Approved by: https://github.com/anijain2305
Reviewed By: huydhn
Differential Revision: D118560089
fbshipit-source-id: 1e150694c227a3cc9a757b8a97910c1c17305431