Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/65861
First in a series. This PR changes the code in deploy.h/cpp and
interpreter_impl.h/cpp to be camel case instead of snake case. Starting
with this as it has the most impact on downstream users.
Test Plan: Imported from OSS
Reviewed By: shannonzhu
Differential Revision: D31291183
Pulled By: suo
fbshipit-source-id: ba6f74042947c9a08fb9cb3ad7276d8dbb5b2934
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/62669
Useful to avoid having to implement null checking on the application side.
Test Plan: Add unit tests
Reviewed By: suo, houseroad
Differential Revision: D30074406
fbshipit-source-id: 881aec735953b43cb24786c1a2d79e8e724928b8
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/58117
Previously it was not possible to load C extension modules with deploy because extension
modules need to link against the Python.h API functions. Since
each libtorchdeploy_interpreter.so had its own copy of these functions, it is not possible
to tell dlopen to resolve symbols in a loaded SO from one of these libraries without exposing
its symbols globally.
This patch adds a custom ELF loader which does the custom loading of attaching c extension libraries
to the Python API that loaded the shared library. Simple use of numpy and regex modules appears to work.
This diff has some limitations:
* 64-bit Linux only. OSX and windows use different formats for shared libraries. 32-bit ELF files are not supported.
* debug info is not immediately availiable to debuggers. A script for lldb is provided which can be loaded
so that lldb knows about the libraries as they are loaded.
* shared libraries can directly use the Python API, but libraries they depend on
(via DT_NEEDED entries in their dynamic segment) may not use Python. In the future, we can
try to detect whether a sub library uses the Python API and load it with our customer loader.
* TLS initialization and library initialization may occur in a different order than what would happen with dlopen,
potentially leading to some issues running destructors in TLS segments. Use of this C++ features is relatively rare.
Test Plan: Imported from OSS
Reviewed By: suo
Differential Revision: D28435305
Pulled By: zdevito
fbshipit-source-id: 10f046053dd1d250e3c73f2cce8eb945eeba31b6
Summary:
As GoogleTest `TEST` macro is non-compliant with it as well as `DEFINE_DISPATCH`
All changes but the ones to `.clang-tidy` are generated using following script:
```
for i in `find . -type f -iname "*.c*" -or -iname "*.h"|xargs grep cppcoreguidelines-avoid-non-const-global-variables|cut -f1 -d:|sort|uniq`; do sed -i "/\/\/ NOLINTNEXTLINE(cppcoreguidelines-avoid-non-const-global-variables)/d" $i; done
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/62008
Reviewed By: driazati, r-barnes
Differential Revision: D29838584
Pulled By: malfet
fbshipit-source-id: 1b2f8602c945bd4ce50a9bfdd204755556e31d13
Summary:
This PR suppresses clang-tidy warnings in the codebase (for now) so that we can re-enable clang-tidy checks on master.
I ran this script to add the `NOLINTNEXTLINE` comments (on a devserver):
```bash
python3 setup.py develop
# Uses same script that's run on CI and adds the -j (parallel), -s (add comments), -k (continue if diagnostic errors are found) options
python3 tools/clang_tidy.py \
-j \
-s \
-k \
-v \
--paths torch/csrc/ \
-g"-torch/csrc/jit/passes/onnx/helper.cpp" \
-g"-torch/csrc/jit/passes/onnx/shape_type_inference.cpp" \
-g"-torch/csrc/jit/serialization/onnx.cpp" \
-g"-torch/csrc/jit/serialization/export.cpp" \
-g"-torch/csrc/jit/serialization/import.cpp" \
-g"-torch/csrc/jit/serialization/import_legacy.cpp" \
-g"-torch/csrc/onnx/init.cpp" \
-g"-torch/csrc/cuda/nccl.*" \
-g"-torch/csrc/cuda/python_nccl.cpp" \
-g"-torch/csrc/autograd/FunctionsManual.cpp" \
-g"-torch/csrc/generic/*.cpp" \
-g"-torch/csrc/jit/codegen/cuda/runtime/*" \
-g"-torch/csrc/deploy/interpreter/interpreter.cpp" \
-g"-torch/csrc/deploy/interpreter/interpreter.h" \
-g"-torch/csrc/deploy/interpreter/interpreter_impl.h" \
-g"-torch/csrc/deploy/interpreter/test_main.cpp"
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/60649
Test Plan: Verified changes by re-running the script (without the `-s` option) and seeing no warnings/errors.
Reviewed By: walterddr, janeyx99
Differential Revision: D29504258
Pulled By: 1ntEgr8
fbshipit-source-id: 78310b30ee8213b73ddb4771ad874665323e7a4e
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/58290
this is a helper function to get some python source code loaded
on each interpreter without having to use the standard import system
or packages. Useful for debugging or for writing wrapper classes for
handling loaded modules.
Test Plan: Imported from OSS
Reviewed By: wconstab
Differential Revision: D28435306
Pulled By: zdevito
fbshipit-source-id: b85c16346b9001cd7350d65879cb990098060813
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/57748
To be used by PyTorchPredictor integration for deploy.
Original commit changeset: 4d41efc733b2
Test Plan: tested via new unit tests
Reviewed By: suo
Differential Revision: D28258525
fbshipit-source-id: 8b9436e47501d7c1c16e79909e668100f825711e
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/57484
To be used by PyTorchPredictor integration for deploy.
Test Plan: tested via new unit tests
Reviewed By: suo
Differential Revision: D28154522
fbshipit-source-id: 5ba57a8d7f01686180e6fd47663635ec3ab2120d
Summary:
1. Delete dead code relating to maskrcnn_benchmark extension module
2. Add some more commentary on why we define a meta path finder
isthisimpact
Test Plan: sandcastle
Reviewed By: wconstab
Differential Revision: D28078211
fbshipit-source-id: cfc6f47861c14ec7482b55ee585504271ae0f365
Summary:
In my last PR I've missed CUDA and distributed folders, fixing this now
This change is autogenerated by `python tool/clang_tidy.py -s`
Pull Request resolved: https://github.com/pytorch/pytorch/pull/57235
Reviewed By: janeyx99
Differential Revision: D28084444
Pulled By: malfet
fbshipit-source-id: bf222f69ee90c7872c3cb0931e8cdb84f0cb3cda
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/54120
Construct InterpreterManager inside PyTorchDeployModel
- add ReadAdapterInterface to deploy::Package
Implement PyTorchDeployModel::makePrediction for FeatureStore Examples
- Basic test of loading and executing 'simple' model
Test Plan: ran unit tests locally and CI
Differential Revision: D26961744
fbshipit-source-id: fce72bc83b9005500d9b7ce3fab2ed466f73d6ed
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/53670
This puts deploy into the torch::deploy namespace. It also renames some
objects to better match their behavior:
PythonObject -> Obj, in the future it will refer to either a python object or a handle to a script obj, so rename it torch::deploy::Obj to be generic
MovableObject -> ReplicatedObj, to prevent confusion with "std::move" which is unrelated, and to note that we are replicating this object across interpreters.
Test Plan: Imported from OSS
Reviewed By: wconstab
Differential Revision: D26932131
Pulled By: zdevito
fbshipit-source-id: 8041d6c5b2041a7c3192c1a17d2edb38112a89f3
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/51754
This API allows you to manage multiple python interpreters in a single
process to deploy PyTorch models packaged with torch.package.
torch/csrc/deploy/deploy.h contains the API definition
torch/csrc/deploy/test_deploy.cpp has some examples.
Notes:
* mutex is added to PyTorchStreamReader to make it safe to use from multiple threads at once.
* USE_DEPLOY is only true for the special libtorch_deployinterpreter.so library, when enabled
we use a hash table to maintain PyObject <> at::Tensor mappping rather than the internal pointer
in Tensor since >1 interpreter may have a reference to the tensor.
* serialization.py has some additional functions for creating pickle objects
but keeping storages in memory for use transfering tensors between interpreters
Test Plan: Imported from OSS
Reviewed By: wconstab
Differential Revision: D26329468
Pulled By: zdevito
fbshipit-source-id: d75f4ebb9a27f1d911179d9996041bcb3ca04a07
