pytorch/torch/csrc/jit
Horace He f81db8afb8 Initial torchbind prototype (#21098)
Summary:
I have some test code in there as well, along with a script "test_libtorch" to run it. You'll need to modify `test_libtorch` to point to where you have `pytorch` built. I currently require that `pybind11` is included as a subdirectory of the test, but added it to the `.gitignore` to make this reviewable.

Currently, something like this works:
```cpp
struct Foo {
  int x, y;
  Foo(): x(2), y(5){}
  Foo(int x_, int y_) : x(x_), y(y_) {}
  void display() {
    cout<<"x: "<<x<<' '<<"y: "<<y<<endl;
  }
  int64_t add(int64_t z) {
    return (x+y)*z;
  }
};
static auto test = torch::jit::class_<Foo>("Foo")
                    .def(torch::jit::init<int64_t, int64_t>())
                    .def("display", &Foo::display)
                    .def("add", &Foo::add)
                    .def("combine", &Foo::combine);

```
with
```py
torch.jit.script
def f(x):
    val = torch._C.Foo(5, 3)
    val.display()
    print(val.add(3))
```
results in
```
x: 5 y: 3
24
```

Current issues:
- [x] The python class created by torchscript doesn't interactly properly with the surrounding code.
```
torch.jit.script
def f(x):
    val = torch._C.Foo(5, 3)
    return val
```
- [x] Doesn't properly take in non-pointer classes. Can't define this function signature in cpp (We don't want to support this I believe).
```cpp
  void combine(Foo x) {
```

- [x] Has some issues with memory for blobs when constructing multiple objects (fix constant propagation pass to not treat capsules as the same object).
```py
torch.jit.script
def f(x):
    val = torch._C.Foo(5, 3)
    val2 = torch._C.Foo(100, 0)
    val.display()
    print(val.add(3))
```
- [ ] Can't define multiple constructors (need to define overload string. Currently not possible since we don't support overloaded methods).
- [x] `init` is a little bit different syntax than `pybind`. `.init<...>()` instead of `.def(py::init<>())`
- [x] I couldn't figure out how to add some files into the build so they'd be copied to the `include/` directories, so I symlinked them manually.
- [ ] Currently, the conversion from Python into Torchscript doesn't work.
- [ ] Torchbind also currently requires Python/Pybind dependency. Fixing this would probably involve some kind of macro to bind into Python when possible.
- [ ] We pass back into Python by value, currently. There's no way of passing by reference.
- [x] Currently can only register one method with the same type signature. This is because we create a `static auto opRegistry`, and the function is templated on the type signature.

Somewhat blocked on https://github.com/pytorch/pytorch/pull/21177. We currently use some structures that will be refactored by his PR (namely `return_type_to_ivalue` and `ivalue_to_arg_type`.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/21098

Differential Revision: D16634872

Pulled By: Chillee

fbshipit-source-id: 1408bb89ea649c27d560df59e2cf9920467fe1de
2019-08-02 18:45:15 -07:00
..
backends
docs Update relative links in OVERVIEW.md 2019-07-31 15:45:04 -07:00
fuser Remove more uses of DimensionedTensorType 2019-08-01 21:19:28 -07:00
passes Remove more uses of DimensionedTensorType 2019-08-01 21:19:28 -07:00
script Initial torchbind prototype (#21098) 2019-08-02 18:45:15 -07:00
testing
alias_info.h add #pragma once to jit headers 2019-07-04 11:10:59 -07:00
argument_spec.cpp Support for NamedTuple (#21428) 2019-06-14 16:45:56 -07:00
argument_spec.h Make ProfiledTensorType hashable 2019-07-30 13:11:06 -07:00
attributes.cpp
attributes.h Better error message for using Python builtin_function_or_method (#22935) 2019-07-16 16:49:04 -07:00
autodiff.cpp Cleanup interface of inlineCallTo. 2019-07-30 11:26:31 -07:00
autodiff.h
catch_utils.hpp
code_template.h
constants.cpp AliasAnalysisKind::CONSERVATIVE/FROM_SCHEMA (#22175) 2019-07-25 11:53:51 -07:00
constants.h
custom_operator.h Fix compiler warnings (#22162) 2019-07-02 14:12:55 -07:00
dynamic_dag.h
exception_message.h Interpreter support for CallFunction/CallMethod (#21562) 2019-06-09 15:28:26 -07:00
export.cpp Compress all non-Tensor components of a serialized TorchScript model. (#23723) 2019-08-02 12:39:20 -07:00
export.h
function.cpp Make CompilationUnit own Functions (#22202) 2019-07-04 17:12:00 -07:00
function.h Make optimize a thread_local flag 2019-07-24 23:09:21 -07:00
graph_executor_impl.h Cleanup interface of inlineCallTo. 2019-07-30 11:26:31 -07:00
graph_executor.cpp AliasAnalysisKind::CONSERVATIVE/FROM_SCHEMA (#22175) 2019-07-25 11:53:51 -07:00
graph_executor.h Make optimize a thread_local flag 2019-07-24 23:09:21 -07:00
graph_node_list.h
hooks_for_testing.cpp Make CompilationUnit own Functions (#22202) 2019-07-04 17:12:00 -07:00
hooks_for_testing.h Make CompilationUnit own Functions (#22202) 2019-07-04 17:12:00 -07:00
import_export_helpers.cpp
import_export_helpers.h
import_source.cpp add simple inheritance support to AST 2019-07-23 12:21:27 -07:00
import_source.h Make classtypes hold a weak_ptr to their CU 2019-07-16 12:04:20 -07:00
import.cpp prefix module qualified names with __module__ (#23630) 2019-07-31 18:30:13 -07:00
import.h kill module_lookup 2019-07-23 12:21:23 -07:00
init.cpp Make traced fns also go into the global python CU 2019-07-16 12:04:16 -07:00
init.h Back out "Revert D15435461: [pytorch][PR] PyTorch ThroughputBenchmark" (#22185) 2019-06-26 16:05:51 -07:00
interned_strings_class.h
interpreter.cpp Make optimize a thread_local flag 2019-07-24 23:09:21 -07:00
interpreter.h Hook up profiled execution in the interpreter (#21799) 2019-06-14 16:56:13 -07:00
ir_views.h add support for breaks and continues (#21692) 2019-07-12 15:02:44 -07:00
ir.cpp Cleanup interface of inlineCallTo. 2019-07-30 11:26:31 -07:00
ir.h Cleanup interface of inlineCallTo. 2019-07-30 11:26:31 -07:00
irparser.cpp
irparser.h add #pragma once to jit headers 2019-07-04 11:10:59 -07:00
jit_log.cpp add line numbers to jit_log.h 2019-07-10 15:28:29 -07:00
jit_log.h Fix a few clang warnings. 2019-07-29 22:08:50 -07:00
named_value.h
netdef_converter.cpp s/uniqueName/debugName/ (#22096) 2019-06-21 20:54:53 -07:00
netdef_converter.h
node_hashing.cpp Initial torchbind prototype (#21098) 2019-08-02 18:45:15 -07:00
node_hashing.h
operator_options.cpp
operator_options.h
operator.cpp AliasAnalysisKind::CONSERVATIVE/FROM_SCHEMA (#22175) 2019-07-25 11:53:51 -07:00
operator.h AliasAnalysisKind::CONSERVATIVE/FROM_SCHEMA (#22175) 2019-07-25 11:53:51 -07:00
pass_manager.cpp
pass_manager.h
pickler.cpp Switch keys to be sequential and stable in pickle serialization 2019-07-24 17:13:51 -07:00
pickler.h Switch keys to be sequential and stable in pickle serialization 2019-07-24 17:13:51 -07:00
print_handler.cpp Override print when python is present (#21625) 2019-06-11 22:58:22 -07:00
print_handler.h Override print when python is present (#21625) 2019-06-11 22:58:22 -07:00
profiling_graph_executor_impl.cpp Make optimize a thread_local flag 2019-07-24 23:09:21 -07:00
profiling_graph_executor_impl.h Make optimize a thread_local flag 2019-07-24 23:09:21 -07:00
profiling_record.cpp Enable more passes in ProfilingGraphExecutor 2019-07-10 10:44:18 -07:00
profiling_record.h Hook up profiled execution in the interpreter (#21799) 2019-06-14 16:56:13 -07:00
pybind_utils.h Initial torchbind prototype (#21098) 2019-08-02 18:45:15 -07:00
pybind.h
python_arg_flatten.cpp
python_arg_flatten.h Remove Type dispatch (#21964) 2019-06-30 04:11:35 -07:00
python_interpreter.cpp AliasAnalysisKind::CONSERVATIVE/FROM_SCHEMA (#22175) 2019-07-25 11:53:51 -07:00
python_ir.cpp Remove more uses of DimensionedTensorType 2019-08-01 21:19:28 -07:00
python_ir.h
python_tracer.cpp remove uses of std::shared_ptr<Module> (#21934) 2019-06-25 13:24:38 -07:00
python_tracer.h remove uses of std::shared_ptr<Module> (#21934) 2019-06-25 13:24:38 -07:00
register_c10_ops.cpp ListPtr->List DictPtr->Dict step 2 (#21937) 2019-06-19 18:02:05 -07:00
register_prim_ops.cpp dictKeys and dictItems ops on typed dicts return typed lists (#23270) 2019-07-29 20:00:34 -07:00
register_special_ops.cpp support torch._C._get_tracing_state in script 2019-07-29 15:05:50 -07:00
register_string_ops.cpp AliasAnalysisKind::CONSERVATIVE/FROM_SCHEMA (#22175) 2019-07-25 11:53:51 -07:00
resource_guard.h
scope.cpp
scope.h
source_range_serialization_impl.h Cleanup some logic in pickler 2019-07-17 11:00:34 -07:00
source_range_serialization.cpp Fix SourceRange comparison 2019-07-26 18:08:43 -07:00
source_range_serialization.h Load original SourceRanges on import (#22180) 2019-07-01 21:14:39 -07:00
source_range.cpp Fix SourceRange comparison 2019-07-26 18:08:43 -07:00
source_range.h Fix a few clang warnings. 2019-07-29 22:08:50 -07:00
subgraph_matcher.cpp
subgraph_matcher.h
symbolic_script.cpp Changed tensor comparison return type from uint8 to bool (#21113) 2019-08-01 07:54:53 -07:00
symbolic_script.h
symbolic_variable.h Rereapply optional ScalarType interface changes that were reverted in D16079809 (#22456) 2019-07-03 20:03:25 -07:00
tracer.cpp move casting ops from prim to aten 2019-07-03 22:22:28 -07:00
tracer.h Adding memory_format to empty and empty_like operators (#20558) 2019-06-26 11:48:27 -07:00
variable_tensor_list.h