mirror of
https://github.com/zebrajr/pytorch.git
synced 2025-12-07 12:21:27 +01:00
6d24f8fe21
Summary: **Summary:** This PR contains the infrastructure of a new CUDA fuser. This CUDA fuser is based on many of the same principles of TensorExpressions and Halide, however the implementation is ground up. The fusion pass itself is similar to the default CUDA fuser, however, it has undergone some refactoring and is using the new code generation infrastructure. For those who are interested in how the code generation in this PR works, I would recommend reviewing _test/cpp/jit/test_gpu_fusion.cpp_ as well as the long comment section at the beginning of _torch/csrc/jit/codegen/cuda/transform_replay.h_ One of the largest differences between our approach and that of TVM/Halide, is the concept of "TensorView". TensorView from a high level should be thought of similarly to how we think of working with Tensors in PyTorch. It's an N-D object which can undergo transformations that change its dimensionality. Dimensionality changes are done through the operations split/merge/reorder/computeAt. These transformations are similar to split/fuse/reorder/compute_at of TVM, they modify how a tensor is iterated over to generate GPU code. Interestingly, in our scheme these transformations are applied to tensors and only impact how that tensor is generated. **Warning:** This PR is purposefully not feature complete with the current fuser. We wanted to separate out the infrastructure from the fusion capabilities. Once in, smaller incremental PRs will be submitted to expand capabilities of the fuser. **Short term goals:** Parity with current CUDA fuser (including performance): - Dynamic shapes (no recompilation) - Implicit handling of braodcast (broadcasted tensors are treated as tensors of the braodcasted size in the generated code) - Dropout **Mid-term goals:** - Transposes fused with pointwise operations where transpose involves only 2 axes (across the fused operation). - 1-D reductions fused with pointwise operations Pull Request resolved: https://github.com/pytorch/pytorch/pull/34785 Reviewed By: ZolotukhinM Differential Revision: D20650977 Pulled By: soumith fbshipit-source-id: ee39c95a880e1b9822e874ed4cc180971572bf63
38 lines
1.1 KiB
C++
38 lines
1.1 KiB
C++
#pragma once
|
|
|
|
#include <torch/csrc/WindowsTorchApiMacro.h>
|
|
#include <torch/csrc/jit/ir/ir.h>
|
|
|
|
#include <torch/csrc/jit/codegen/cuda/fusion.h>
|
|
|
|
/*
|
|
* This file handles Parsing PyTorch jit ir;
|
|
*
|
|
* It is used in two places:
|
|
* 1. When partitioning PyTorch jit ir to create prim::CudaFusionGroup, each
|
|
* node is queried by `isNodeParsible` to determine whether the node could
|
|
* be handled by the fuser (whether a given PyTorch jit operator should be
|
|
* merged);
|
|
* 2. lowering PyTorch jit ir to CUDA codegen ir.
|
|
* creates a `Fusion` by traversing a PyTorch jit graph.
|
|
*
|
|
* TODO: we could consider exposing API to allow custom registration of parsing
|
|
* rules for a given PyTorch jit operator.
|
|
*/
|
|
|
|
namespace torch {
|
|
namespace jit {
|
|
namespace fuser {
|
|
namespace cuda {
|
|
|
|
// returns whether or not a parsing function exists for the given node type.
|
|
TORCH_CUDA_API bool isNodeParsible(const Node* const node);
|
|
|
|
// lowers PyTorch jit graph to `Fusion`.
|
|
TORCH_CUDA_API void parseJitIR(std::shared_ptr<Graph>& graph, Fusion& fusion);
|
|
|
|
} // namespace cuda
|
|
} // namespace fuser
|
|
} // namespace jit
|
|
} // namespace torch
|