pytorch/tools
chunhuanMeng 1f3d8896bc Fix mismatched tensor metadata between FakeTensor and Intel XPU concrete tensor when running F.logsigmoid (#141333)
Fixes https://github.com/pytorch/pytorch/issues/141332
`F.logsigmoid` will return two outputs: `output` and `buffer`.
For `F.logsigmoid` cpu path, it will use buffer to store some intermediate values and use them when computing gradients, so it returns a `buffer` tensor with nonzero size. For cuda and xpu paths, buffer is useless, so the `buffer ` tensor size of xpu `F.logsigmoid`  will be zero, just like cuda. The root cause of the issue is that the codes in `decompositions.py` (ref:https://github.com/pytorch/pytorch/blob/main/torch/_decomp/decompositions.py#L2803) only handle the cuda cases, when the a fake tensor with device is xpu run to here, it will use the cpu path and return a `buffer` with nonzero size, which is conflict to the  implementation of intel xpu concrete tensor. Therefore this pr add conditions to handle xpu cases. Make sure the two returned buffer sizes match each other.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/141333
Approved by: https://github.com/guangyey, https://github.com/EikanWang, https://github.com/ezyang
2024-12-02 22:09:20 +00:00
..
alerts [BE][Easy] enable postponed annotations in tools (#129375) 2024-06-29 09:23:35 +00:00
amd_build [reland][Inductor] Rename cpp_wrapper_cuda.py as cpp_wrapper_gpu.py (#136046) 2024-09-16 14:35:19 +00:00
autograd [Reland][7/N] Fix Wextra-semi warning (#140342) 2024-11-12 18:55:31 +00:00
bazel_tools
build/bazel Bump certifi from 2024.2.2 to 2024.7.4 in /tools/build/bazel (#130173) 2024-10-28 15:44:49 -07:00
build_defs [lint] Remove unnecessary BUCKRESTRICTEDSYNTAX suppressions 2024-07-19 07:19:11 -07:00
code_analyzer [BE][Easy] enable postponed annotations in tools (#129375) 2024-06-29 09:23:35 +00:00
code_coverage [BE][Easy] fix ruff rule needless-bool (SIM103) (#130206) 2024-07-14 08:17:52 +00:00
config
coverage_plugins_package [BE][Easy] enable postponed annotations in tools (#129375) 2024-06-29 09:23:35 +00:00
dynamo [BE][Easy] enable postponed annotations in tools (#129375) 2024-06-29 09:23:35 +00:00
flight_recorder Revert "[BE]: Update mypy to 1.13.0 (#140808)" 2024-12-02 20:47:43 +00:00
gdb Add gdb print methods support same as pytorch-lldb (#140935) 2024-11-19 01:28:30 +00:00
github [BE][Easy] enable postponed annotations in tools (#129375) 2024-06-29 09:23:35 +00:00
iwyu [BE][Easy] enable postponed annotations in tools (#129375) 2024-06-29 09:23:35 +00:00
jit [9/N] Replace c10::optional with std::optional (#130674) 2024-07-15 00:48:43 +00:00
linter Use pip corresponding to python executable (#141165) 2024-11-20 22:58:33 +00:00
lite_interpreter C10_UNUSED to [[maybe_unused]] (#6357) (#138364) 2024-10-19 13:17:43 +00:00
lldb [BE][Easy] enable postponed annotations in tools (#129375) 2024-06-29 09:23:35 +00:00
onnx [11/N] Fix extra warnings brought by clang-tidy-17 (#139599) 2024-11-04 23:57:41 +00:00
packaging tools: Add script to do split build in one command (#141359) 2024-11-26 20:51:05 +00:00
pyi Fix mismatched tensor metadata between FakeTensor and Intel XPU concrete tensor when running F.logsigmoid (#141333) 2024-12-02 22:09:20 +00:00
rules
rules_cc [BE] Fix incompatible-std-redefinition warning (#141630) 2024-11-27 05:06:36 +00:00
setup_helpers Allow users to overwrite ld with environment variable in linker optimization script (#137331) 2024-11-26 22:54:24 +00:00
shared
stats Add monitor script (#141438) 2024-11-29 04:14:31 +00:00
test Remove most rockset references (#139922) 2024-11-12 21:17:43 +00:00
testing [TD] Filepath heuristic also looks at file name (#140170) 2024-11-11 22:55:54 +00:00
__init__.py
bazel.bzl
BUCK.bzl [lint] Remove unnecessary BUCKRESTRICTEDSYNTAX suppressions 2024-07-19 07:19:11 -07:00
BUCK.oss
build_libtorch.py [BE] Rectify some references to caffe2 (#140204) 2024-11-09 14:14:20 +00:00
build_pytorch_libs.py Fix access to _msvccompiler from newer distutils (#141363) 2024-11-25 01:50:47 +00:00
build_with_debinfo.py Improve build_with_deb_info (#138290) 2024-10-18 18:50:12 +00:00
download_mnist.py [BE][Easy] enable postponed annotations in tools (#129375) 2024-06-29 09:23:35 +00:00
extract_scripts.py [BE][Easy] enable postponed annotations in tools (#129375) 2024-06-29 09:23:35 +00:00
gen_flatbuffers.sh
gen_vulkan_spv.py [BE][Easy] enable postponed annotations in tools (#129375) 2024-06-29 09:23:35 +00:00
generate_torch_version.py Add torch.version.xpu (#139466) 2024-11-09 13:31:21 +00:00
generated_dirs.txt
git_add_generated_dirs.sh
git_reset_generated_dirs.sh
nightly_hotpatch.py Add a nightly hotpatch utils for python only PR (#136535) 2024-09-27 15:58:48 +00:00
nightly.py Refactor nightly pull tool to use venv and pip (#141281) 2024-11-22 20:03:55 +00:00
nvcc_fix_deps.py [BE][Easy] enable postponed annotations in tools (#129375) 2024-06-29 09:23:35 +00:00
README.md
render_junit.py [BE][Easy] enable postponed annotations in tools (#129375) 2024-06-29 09:23:35 +00:00
substitute.py [Codemod][python/main_function] caffe2: (#113357) 2023-11-15 22:17:31 +00:00
update_masked_docs.py Use strict to toggle strict options in MYPYSTRICT (#118479) 2024-01-28 19:22:22 +00:00
vscode_settings.py [BE][Easy][5/19] enforce style for empty lines in import segments in tools/ and torchgen/ (#129756) 2024-07-17 06:44:35 +00:00

This folder contains a number of scripts which are used as part of the PyTorch build process. This directory also doubles as a Python module hierarchy (thus the __init__.py).

Overview

Modern infrastructure:

  • autograd - Code generation for autograd. This includes definitions of all our derivatives.
  • jit - Code generation for JIT
  • shared - Generic infrastructure that scripts in tools may find useful.
    • module_loader.py - Makes it easier to import arbitrary Python files in a script, without having to add them to the PYTHONPATH first.

Build system pieces:

  • setup_helpers - Helper code for searching for third-party dependencies on the user system.
  • build_pytorch_libs.py - cross-platform script that builds all of the constituent libraries of PyTorch, but not the PyTorch Python extension itself.
  • build_libtorch.py - Script for building libtorch, a standalone C++ library without Python support. This build script is tested in CI.

Developer tools which you might find useful:

Important if you want to run on AMD GPU:

  • amd_build - HIPify scripts, for transpiling CUDA into AMD HIP. Right now, PyTorch and Caffe2 share logic for how to do this transpilation, but have separate entry-points for transpiling either PyTorch or Caffe2 code.
    • build_amd.py - Top-level entry point for HIPifying our codebase.

Tools which are only situationally useful: