pytorch
92eb9d36 - Decoder native functions join the dead code society (#96025)

Commit
1 year ago
Decoder native functions join the dead code society (#96025) Summary: Decoder native joins the dead code society With the recent introduction of PT2, we no longer need native decoder operators: 1 - full-function SDPA kernels can be used to implement cross-attention efficiently without the (slower) decoder MHA blob. 2 - torch.compile() generates more efficient code across many platforms from the python implementation of decoders than the decoder layer blob by tailoring code to target Test Plan: github & sandcastle Differential Revision: D43811808 Pull Request resolved: https://github.com/pytorch/pytorch/pull/96025 Approved by: https://github.com/ezyang, https://github.com/albanD
Author
Committer
Parents
Loading