mirror of
https://github.com/zebrajr/pytorch.git
synced 2025-12-06 12:20:52 +01:00
6aecfd1e80
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/33722 In order to improve CPU performance on floating-point models on mobile, this PR introduces a new CPU backend for mobile that implements the most common mobile operators with NHWC memory layout support through integration with XNNPACK. XNNPACK itself, and this codepath, are currently only included in the build, but the actual integration is gated with USE_XNNPACK preprocessor guards. This preprocessor symbol is intentionally not passed on to the compiler, so as to enable this rollout in multiple stages in follow up PRs. This changeset will build XNNPACK as part of the build if the identically named USE_XNNPACK CMAKE variable, defaulted to ON, is enabled, but will not actually expose or enable this code path in any other way. Furthermore, it is worth pointing out that in order to efficiently map models to these operators, some front-end method of exposing this backend to the user is needed. The less efficient implementation would be to hook these operators into their corresponding native implementations, granted that a series of XNNPACK-specific conditions are met, much like how NNPACK is integrated with PyTorch today for instance. Having said that, while the above implementation is still expected to outperform NNPACK based on the benchmarks I ran, the above integration would be leave a considerable gap between the performance achieved and the maximum performance potential XNNPACK enables, as it does not provide a way to compute and factor out one-time operations out of the inner most forward() loop. The more optimal solution, and one we will decide on soon, would involve either providing a JIT pass that maps nn operators onto these newly introduced operators, while allowing one-time calculations to be factored out, much like quantized mobile models. Alternatively, new eager-mode modules can also be introduced that would directly call into these implementations either through c10 or some other mechanism, also allowing for decoupling of op creation from op execution. This PR does not include any of the front end changes mentioned above. Neither does it include the mobile threadpool unification present in the original https://github.com/pytorch/pytorch/issues/30644. Furthermore, this codepath seems to be faster than NNPACK in a good number of use cases, which can potentially allow us to remove NNPACK from aten to make the codebase a little simpler, granted that there is widespread support for such a move. Regardless, these changes will be introduced gradually and in a more controlled way in subsequent PRs. Pull Request resolved: https://github.com/pytorch/pytorch/pull/32509 Test Plan: Build: CI Functionality: Not exposed Reviewed By: dreiss Differential Revision: D20069796 Pulled By: AshkanAliabadi fbshipit-source-id: d46c1c91d4bea91979ea5bd46971ced5417d309c |
||
|---|---|---|
| .. | ||
| appveyor | ||
| fbcode-dev-setup | ||
| model_zoo | ||
| onnx | ||
| add_apache_header.sh | ||
| apache_header.txt | ||
| apache_python.txt | ||
| build_android.sh | ||
| build_host_protoc.sh | ||
| build_ios.sh | ||
| build_local.sh | ||
| build_mobile.sh | ||
| build_pytorch_android.sh | ||
| build_raspbian.sh | ||
| build_tegra_x1.sh | ||
| build_tizen.sh | ||
| build_windows.bat | ||
| diagnose_protobuf.py | ||
| get_python_cmake_flags.py | ||
| proto.ps1 | ||
| read_conda_versions.sh | ||
| README.md | ||
| remove_apache_header.sh | ||
| run_mobilelab.py | ||
| temp.sh | ||
| xcode_build.rb | ||
This directory contains the useful tools.
build_android.sh
This script is to build PyTorch/Caffe2 library for Android. Take the following steps to start the build:
- set ANDROID_NDK to the location of ndk
export ANDROID_NDK=YOUR_NDK_PATH
- run build_android.sh
#in your PyTorch root directory
bash scripts/build_android.sh
If succeeded, the libraries and headers would be generated to build_android/install directory. You can then copy these files from build_android/install to your Android project for further usage.
You can also override the cmake flags via command line, e.g., following command will also compile the executable binary files:
bash scripts/build_android.sh -DBUILD_BINARY=ON
build_ios.sh
This script is to build PyTorch/Caffe2 library for iOS, and can only be performed on macOS. Take the following steps to start the build:
- Install Xcode from App Store, and configure "Command Line Tools" properly on Xcode.
- Install the dependencies:
brew install cmake automake libtool
- run build_ios.sh
#in your PyTorch root directory
bash scripts/build_ios.sh
If succeeded, the libraries and headers would be generated to build_ios/install directory. You can then copy these files to your Xcode project for further usage.