pytorch

0ab2ec37 - [XPU][Profiler] Add Logic To The Profiler For Processing XPU-backend Data (#120185)

[XPU][Profiler] Add Logic To The Profiler For Processing XPU-backend Data (#120185) This pull request is writing to provide an update on the recent advancements made in the PyTorch profiler with regards to XPU backend support. Following the successful merge of a previous pull request #94502 that established a pathway for the XPU backend within PyTorch, we have now taken steps to enhance the profiler's capabilities for handling and displaying profile data directly related to the XPU backend. # Motivation The current pull request builds upon this foundation by refining the profiler's data processing scripts, particularly `profiler_util.py`, to accommodate XPU backend-specific profile data. The aim is to align the handling and presentation of this data with that of the CUDA backend, offering users a consistent experience across different device profiles. This includes generating outputs such as JSON files compatible with Chrome trace tooling, among other formats. # Principles 1. Minimal Impact: The modifications introduced should support XPU backend data with minimal disruption to the existing profiling scripts. 2. Consistency: Changes should maintain stylistic and functional consistency with existing `CUDA` and `privateuse1` pathways, ensuring no adverse effects on other logic paths. 3. Exclusivity: Ensure that the new XPU pathway does not interfere with or impede other pathways. # Solutions ### a. Pathway Identification: Introduction of a `use_xpu` flag within `torch.autograd.profiler.profile` interfaces to distinguish XPU-specific profiling. ### b. `use_device` Logic Revision: With the introduction of the XPU pathway, `use_device` no longer implies a binary relationship with `use_cuda`. Consequently, we have revised related logic to remove implicit assertions and establish independent device distinction. ### c. Kernel List Segregation: To accommodate the non-binary nature of device pathways, we have enabled kernel lists to identify specific device affiliations through separate list objects. ### d. Formatted Output: To ensure output consistency, we have employed code duplication and keyword substitution techniques to facilitate the formatting of XPU-related profile data. # Additional Enhancements ### a. Enumerations in `.pyi` Files: Added recognition items for `DeviceType` and `ProfilerActivity` specific to XPU. ### b. Correct DeviceType Returns: Revised `deviceTypeFromActivity` logic to accurately differentiate between device backends, even when they share common flags such as `libkineto::ActivityType::GPU_MEMCPY`. ### c. Bug Fixes in `cuda_corr_map`: Addressed a corner case where erroneous parent-child event relationships were formed due to shared function event identifiers. The solution involves refining `cuda_corr_map` processing to prevent a function event from being misidentified as both the linker and linkee. # Further Abstraction Looking forward, we acknowledge the potential for further abstraction in the codebase. The current changes necessitated by XPU support have highlighted opportunities for reducing redundancy by consolidating naming conventions and utilizing a singular `device` naming system that relies on `DeviceType` attributes or string flags for differentiation. This would involve significant refactoring to replace device-specific flags and variables. This topic needs further discussions about whether we could and when we should deprecate all those flags and variables named with `cuda`. # Next Pull Request The next pull request will be contingent on Kineto's adoption of Intel's forthcoming PTI-sdk library, which will enable direct usage of XPU-related tracers. Subsequent modifications to `libkineto_init()` will aim to endow PyTorch running on XPU backends with comprehensive profiling capabilities on XPU devices. We appreciate your attention to these enhancements and welcome any feedback or questions you may have regarding these developments. Pull Request resolved: https://github.com/pytorch/pytorch/pull/120185 Approved by: https://github.com/aaronenyeshi, https://github.com/gujinghui