pytorch/scripts
David Berard b27ec57331 [JIT] script & logging for extracting IR from logs (#72889)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/72889

The script along with the GRAPH_EXPORT macro will allow for an easy way to extract IR from logs. One use case in this diff is to extract the fusion groups from nvfuser, so that the fusions can be tested individually.

Usage (e.g. for nvfuser test)

1. Write some test.py file that uses nvfuser
2. `PYTORCH_JIT_LOG_LEVEL=">>graph_fuser" python3 test.py 2>&1 | tee output.txt`
3. `python3 pytorch/scripts/jit/log_extract.py output.txt --nvfuser`

This will run with and without nvfuser to compare the output.

Alternatively, use `--output` to dump the IR so that it can be used in other applications.

Currently, only `--output` works (since generating input tensors is not supported)

Test Plan: Imported from OSS

Reviewed By: ngimel

Differential Revision: D34440189

Pulled By: davidberard98

fbshipit-source-id: fca0f619200ee37aba34bb39b69e6c640c263e26
(cherry picked from commit eb319166075db160f1628f0de545641fbecde8be)
2022-03-02 18:34:35 +00:00
..
appveyor
fbcode-dev-setup
jit [JIT] script & logging for extracting IR from logs (#72889) 2022-03-02 18:34:35 +00:00
model_zoo Disallow versionless Python shebangs (#58275) 2021-05-14 08:26:02 -07:00
onnx [ONNX] Support opset 15 (#67121) (#67805) 2021-11-08 14:37:00 -08:00
release Prune osx-arm64 binaries from nightly channel (#70132) 2021-12-20 11:28:43 -08:00
release_notes Fix CI quick-checks (#71773) 2022-01-25 18:32:13 +00:00
add_apache_header.sh
apache_header.txt
apache_python.txt
build_android.sh [PyTorch] Add codegen unboxing ability (#69881) 2022-03-01 23:28:13 +00:00
build_host_protoc.sh
build_ios.sh [PyTorch] Add codegen unboxing ability (#69881) 2022-03-01 23:28:13 +00:00
build_local.sh
build_mobile.sh Remove distutils (#57040) 2021-04-29 12:10:11 -07:00
build_pytorch_android.sh
build_raspbian.sh
build_tegra_x1.sh
build_tizen.sh
build_windows.bat
diagnose_protobuf.py
get_python_cmake_flags.py Remove distutils (#57040) 2021-04-29 12:10:11 -07:00
proto.ps1
read_conda_versions.sh
README.md
remove_apache_header.sh
temp.sh
xcode_build.rb [CoreML][OSS] Add Simulator tests (#65076) 2021-09-23 14:57:01 -07:00

This directory contains the useful tools.

build_android.sh

This script is to build PyTorch/Caffe2 library for Android. Take the following steps to start the build:

  • set ANDROID_NDK to the location of ndk
export ANDROID_NDK=YOUR_NDK_PATH
  • run build_android.sh
#in your PyTorch root directory
bash scripts/build_android.sh

If succeeded, the libraries and headers would be generated to build_android/install directory. You can then copy these files from build_android/install to your Android project for further usage.

You can also override the cmake flags via command line, e.g., following command will also compile the executable binary files:

bash scripts/build_android.sh -DBUILD_BINARY=ON

build_ios.sh

This script is to build PyTorch/Caffe2 library for iOS, and can only be performed on macOS. Take the following steps to start the build:

  • Install Xcode from App Store, and configure "Command Line Tools" properly on Xcode.
  • Install the dependencies:
brew install cmake automake libtool
  • run build_ios.sh
#in your PyTorch root directory
bash scripts/build_ios.sh

If succeeded, the libraries and headers would be generated to build_ios/install directory. You can then copy these files to your Xcode project for further usage.