pytorch

23de2e06 - [Dynamo] Fix staticmethods for FSDP (#100117)

[Dynamo] Fix staticmethods for FSDP (#100117) This PR fixes capturing static methods for FSDP-managed modules. Previously, if a static method was invoked using `self.<staticmethod>`, then Dynamo would pass `self` twice to the method, causing a graph break due to the method being "unsupported". This PR achieves this by checking for `staticmethod` and using `UserFunctionVariable` instead of `UserMethodVariable`, which handles the correct calling convention. This fixes FSDP + PT2 on HuggingFace's `T5ForConditionalGeneration`, which otherwise reports an error like the following based on the most recent trunk: ``` Output 0 of AsStridedBackward0 is a view of a view which was created in no_grad mode and is being modified inplace with grad mode enabled. ``` This is in reference to the `scores` tensor in `scores += position_bias_masked` ([code](https://github.com/huggingface/transformers/blob/a0ae2310ec46a2c592950babc85cf02e325bf6a7/src/transformers/models/t5/modeling_t5.py#L559)). I am not clear if this PR's fix is actually masking a different problem though. I wonder if there are edge cases with respect to Dynamo resuming execution and input mutations. Possibly, this PR only side steps the problem because there is no more recompilation at the static method `_relative_position_bucket()` ([code](https://github.com/huggingface/transformers/blob/a0ae2310ec46a2c592950babc85cf02e325bf6a7/src/transformers/models/t5/modeling_t5.py#L443)). In `UserDefinedObjectVariable.var_getattr()`, there is an existing branch: https://github.com/pytorch/pytorch/blob/e5291e633fca9639ace4fb4585e1bb0b9dca9e95/torch/_dynamo/variables/user_defined.py#L395-L398 I am not clear on when this branch can be triggered since if `subobj` is a static method, it still takes the `FunctionTypes` branch: https://github.com/pytorch/pytorch/blob/e5291e633fca9639ace4fb4585e1bb0b9dca9e95/torch/_dynamo/variables/user_defined.py#L403-L404 To preserve backward compatibility, the current version of this PR only modifies this `FunctionTypes` branch to differentiate between `staticmethod` and not `staticmethod`. The PR that added this `FunctionTypes` branch is https://github.com/pytorch/pytorch/pull/92050/, and I checked that the added test `test_torch_distributions_functions()` only exercises the non-`staticmethod` case (since `Independent.log_prob` is not a `staticmethod`). The last commit in `pytorch` that touched the `staticmethod` branch before https://github.com/pytorch/pytorch/pull/92050/ was the move from the `torchdynamo` repo into `pytorch`, so I cannot easily tell which test cases it corresponds to. Pull Request resolved: https://github.com/pytorch/pytorch/pull/100117 Approved by: https://github.com/anijain2305