benchmark
af93b8be - pack linear for FP32 dynamic mode (#157542)

Commit
152 days ago
pack linear for FP32 dynamic mode (#157542) Summary: Currently, Linear in FP32 dynamic mode(batch_size has free symbols) does not support weight prepacking since MKL Linear does not support dynamic mode. This PR uses oneDNN Linear to support Linear weight prepacking in FP32 dynamic mode. I tested the Inductor benchmark in FP32 dynamic mode on CPU using this PR, and saw ~8% improvement in timm_models geomean speedup, ~2% improvement in torchbench geomean speedup, and no change in huggingface. There are about 18 models with different degrees of performance improvement, among which BERT_pytorch, soft_actor_critic, BlenderbotForCausalLM, ElectraForCausalLM, crossvit_9_240, mobilevit_s, twins_pcpvt_base have more than 20% performance improvement. X-link: https://github.com/pytorch/pytorch/pull/157542 Approved by: https://github.com/CaoE, https://github.com/jansel Reviewed By: seemethere Differential Revision: D80465691 fbshipit-source-id: 1a3627884c3769f292eec4c3ad396e7c91162c46
Author
generatedunixname499836121
Parents
Loading