OSSForks/pytorch
1
0
Fork 0
mirror of https://github.com/zebrajr/pytorch.git synced 2025-12-06 12:20:52 +01:00
Code Issues Actions 38 Packages Projects Releases Wiki Activity
71cb13869b
pytorch/torch/csrc/jit
History
Behrang Javaherian 386776c49a [torch] Reduce the memory usage by adding flags to clearing intermediate graphs used for optimization during the ineference. (#115657) 
Summary: During the inference time the intermediate graphs for optimization are not used so the Executor's graph is the only graph we need to keep around these two flags

Test Plan:
the FLAGS are all off by default

baseline
```
buck run mode/opt-clang  sigrid/predictor/client/localnet:run_model -- --model_id_to_load=951679039 --model_snapshot_to_load=244 --torch_jit_do_not_store_optimized_graph=true
I1212 10:24:20.407408 401092 SigridPredictorLocalModelFactory.cpp:32] Memory usage for 951679039_244 is 182863 Kb
```
```
buck run mode/opt-clang  sigrid/predictor/client/localnet:run_model -- --model_id_to_load=951679039 --model_snapshot_to_load=244 --torch_jit_do_not_store_optimized_graph=true --torch_jit_release_profiling_graph_after_optimization=true
I1212 10:31:37.663487 464000 SigridPredictorLocalModelFactory.cpp:32] Memory usage for 951679039_244 is 186127 Kb
```
```
buck run mode/opt-clang  sigrid/predictor/client/localnet:run_model -- --model_id_to_load=951679039 --model_snapshot_to_load=244 --torch_jit_do_not_store_optimized_graph=true --torch_jit_release_profiling_graph_after_optimization=true --torch_jit_execution_plan_avoid_extra_graph_copy=true
I1212 10:29:42.848093 447218 SigridPredictorLocalModelFactory.cpp:32] Memory usage for 951679039_244 is 129451 Kb```

Differential Revision: D52081631

Pull Request resolved: https://github.com/pytorch/pytorch/pull/115657
Approved by: https://github.com/houseroad
2023-12-18 17:56:39 +00:00
..
api [BE][JIT] Do not wrap shared_ptr with optional (#115473) 2023-12-09 20:43:40 +00:00
backends [CI] Update clang-format (#116002) 2023-12-18 14:58:46 +00:00
codegen Update oneDNN submodule to v3.3.2 (#112700) 2023-12-05 17:51:55 +00:00
cuda
docs
frontend Add support for torch.Generator type in TorchScript (#110413) 2023-11-21 23:07:21 +00:00
ir [Reland] [1/N] Fixes clang-tidy warnings in header files (#114668) 2023-11-29 07:11:51 +00:00
mobile [Reland] [1/N] Fixes clang-tidy warnings in header files (#114668) 2023-11-29 07:11:51 +00:00
operator_upgraders [BE] [3/N] Use nested namespaces (#110314) 2023-09-30 02:23:48 +00:00
passes [PyTorch] Redirect c10::optional to std::optional (#101995) 2023-11-30 02:46:41 +00:00
python fix error message in pytorch (#115349) 2023-12-07 19:27:29 +00:00
runtime [torch] Reduce the memory usage by adding flags to clearing intermediate graphs used for optimization during the ineference. (#115657) 2023-12-18 17:56:39 +00:00
serialization [Reland] [1/N] Fixes clang-tidy warnings in header files (#114668) 2023-11-29 07:11:51 +00:00
tensorexpr [llvm][oncall] Fix build for llvm-18+ (#115652) 2023-12-13 20:11:31 +00:00
testing Add regex matching to Inductor all2all collective unit tests (#112077) 2023-10-26 08:29:30 +00:00
jit_log.cpp [RELAND] Remove some unnecessary <iostream> includes from headers (#108150) 2023-09-20 21:55:15 +00:00
jit_log.h
jit_opt_limit.cpp [Reland] Eliminate invocations of c10::stoi,c10::stod,c10::stoull,c10::stoll (#109566) 2023-09-19 07:15:25 +00:00
jit_opt_limit.h
JIT-AUTOCAST.md
OVERVIEW.md [docs][jit] Mention dynamic-shapes settings in jit/OVERVIEW.md (#113964) 2023-11-21 06:21:38 +00:00
README.md
resource_guard.h

README.md

PyTorch JIT

This folder contains (most of) the C++ code for the PyTorch JIT, a language and compiler stack for executing PyTorch models portably and efficiently. To learn more about the JIT from a user perspective, please consult our reference documentation and tutorials.

A brief summary of the source tree:

  • OVERVIEW.md: High-level technical overview of the JIT.
  • frontend/: Taking PyTorch modules in Python and translating them into the JIT IR.
  • ir/: Core IR abstractions.
  • runtime/: Interpreter, graph execution, and JIT operators.
  • codegen/: Generating efficient, hardware-specific code for JIT subgraphs.
  • serialization/: Saving and loading modules.
  • api/: Any user-facing C++ or Python interfaces.
  • python/: Binding stuff into Python or accessing information from the Python environment.
  • testing/: Utilities and helpers for testing.
  • mobile/: Mobile-specific implementations of runtime components.
  • passes/: IR-to-IR passes, generally for optimization and lowering.
  • generated/: This folder is generated by the PyTorch build, and contains bindings for native PyTorch operators into the JIT.

Refer to each folder for more in-depth documentation.

Other relevant parts of the codebase not contained here:

  • aten/src/ATen/core: contains JIT code re-used by other elements of the runtime system (eager, mobile, etc.)