benchmark
fb146299 - Fix Dynamo Benchmarks: Stabilization iters counted in latency (#196060)

Commit
74 days ago
Fix Dynamo Benchmarks: Stabilization iters counted in latency (#196060) Summary: ## Summary Fix HF LLM compilation-latency accounting by measuring matched eager and compiled generations, while keeping additional compiled calls as untimed cache stabilization. Peak-memory and Dynamo-stat collection remain unchanged. There was a bug in the Dynamo benchmarks that included stabilization iterations in the compilation latency calculation. `run_performance_test()` sets `niters=1` for HuggingFace LLMs. Eager uses `niters` for warmup explicitly. However, compiled warmup skips `niters` and uses `warmups()`'s default of 5. `compilation_time = dynamo_latency - eager_latency`, so this compares five compiled geneartions againts one eager generation. ## Test Plan Adds a focused model-free regression test, and I manually ran a Qwen3-0.6B check with 1000 input and 2000 output tokens | Accounting | Compilation latency | |---|---:| | Before: 5 compiled calls - 1 eager call | 44.565 s | | After: 1 compiled call - 1 eager call | 44.448 s | | Overcount removed | 0.117 s (0.26%) | X-link: https://github.com/pytorch/pytorch/pull/196060 Approved by: https://github.com/anijain2305 Reviewed By: huydhn Differential Revision: D118929058 fbshipit-source-id: c0295d223500ae1b757a7ddda06f8ea01990cdaf
Author
Committer
Parents
Loading