pytorch

9c875431 - Enforce check ad in test_jit (#18509)

Enforce check ad in test_jit (#18509) Summary: If a test triggers autodiff, it must have a `DifferentiableGraph` in its differentiated forward graph, and this subgraph must have either the original aten node, or the corresponding nodes used in AD formula. Typically a forward differentiable graph looks like this: ``` graph(%i0 : Float(), %i1 : Float()): %3 : Float() = prim::DifferentiableGraph_0(%i0, %i1) return (%3) with prim::DifferentiableGraph_0 = graph(%0 : Float(), %1 : Float()): %2 : Float() = aten::max(%0, %1) return (%2) ``` which tells us `aten::max(Tensor self, Tensor other) -> Tensor` is symbolically differentiable. Update: there're a lot of cases (fusions/ConstantChunk/python implementations) that breaks it so I decided to make the check optionally take node names if different from function name. ~~[OLD]Theoretically I could also check if `aten::max` is in the differentiable block or not to be more precise, but there're also cases like `chunk` where in a differentiable block it's replaced with a prim node (ConstantChunk) and we will have to special case them. Any suggestions here (to be more precise or no) is very welcome!~~ We used to have a list containing nn tests should be run against AD, I moved it to an field when constructing our test(either torch or nn). I think it's cleaner this way, and it matches the fact that for the same op we support one schema of it but not all, in this way we could just turn on the corresponding test which triggers that supported schema. cc: apaszke zdevito wanchaol ngimel for a review [Done] : - Going through a manual second pass of all tests to check if they should enable AD test or not.... - Add a readme about how to add AD for an op and how to add/enable its test in test_jit. Pull Request resolved: https://github.com/pytorch/pytorch/pull/18509 Differential Revision: D14696811 Pulled By: ailzhang fbshipit-source-id: c5e693277baac585cd3aed5ab2c0e7faa5e6f29f