pytorch/test
Yanli Zhao ab39a55331 python udf over rpc (#23569)
Summary:
This diff is to support python user defined function over rpc for https://github.com/pytorch/pytorch/issues/23110, work flow is like this:
1. pickle python udf
2. pass pickle to C++
3. C++ pass over rpc from client to server
4. server call runPythonUDF() python function to unpickle and run python udf and pickle the udf result using python embedder
6. pass back serialized result from server to client
7. client call loadPythonUDFResult() python function to unpickle result
7. return it to python

right now, put rpc_sync_builtin() and rpc_async_builtin() as temporary interfaces for builtin operator remote calls, they accept qualified name string, this interface can execute builtin operators in C++ land.

rpc_sync() and rpc_async() accept python callables only right now, it could be user define python functions or builtin operator python functions, the python functions will be executed in python land.

once we can resolve builtin operator python callables to qualified name string, we can merge rpc_sync_builtin() into rpc_sync() then
Pull Request resolved: https://github.com/pytorch/pytorch/pull/23569

Test Plan: unit tests

Differential Revision: D16390764

Pulled By: zhaojuanmao

fbshipit-source-id: 2cf2c22a979646830b5581bd75eabf8b3cca564c
2019-08-14 23:13:33 -07:00
..
bottleneck
cpp clean up import_source (#24282) 2019-08-14 11:26:26 -07:00
cpp_extensions Add overload names to native_functions.yaml (#23532) 2019-08-01 02:08:37 -07:00
custom_operator Initial torchbind prototype (#21098) 2019-08-02 18:45:15 -07:00
data Implement kthvalue in ATen (#17544) 2019-03-01 19:00:10 -08:00
error_messages Use latest stable flake8-bugbear in CI and fix B011 flake8 error. (#21944) 2019-06-24 20:48:15 -07:00
expect added mesh plugin (#24039) 2019-08-09 10:22:43 -07:00
jit use fully qualified name for ScriptClasses (#19239) 2019-04-26 19:17:21 -07:00
onnx Resolve unused variables in tests (#24075) 2019-08-14 21:02:52 -07:00
optim Implement AdamW optimizer (#21250) 2019-07-02 09:09:10 -07:00
test_module Turn on F401: Unused import warning. (#18598) 2019-03-30 09:01:17 -07:00
common_cuda.py Update ROCm CI to python3.6 (#23088) 2019-07-23 23:07:45 -07:00
common_distributed.py Extract common classes and functions from test_c10d to common_distributed (#23660) 2019-08-02 09:19:32 -07:00
common_methods_invocations.py Allowing batching for det/logdet/slogdet operations (#22909) 2019-07-31 10:01:32 -07:00
common_nn.py Added a flatten module (#22245) 2019-07-25 22:48:52 -07:00
common_quantization.py JIT trace testing 2019-08-14 22:11:32 -07:00
common_quantized.py Add dynamic quantized Linear op in PyTorch (#23464) 2019-08-13 19:59:35 -07:00
common_utils.py Rename torchtest.test_all_device_types to torchtest.for_all_device_types (#24337) 2019-08-14 12:09:51 -07:00
expecttest.py
HowToWriteTestsUsingFileCheck.md Add a document 'How to Write Tests Using FileCheck' (#19005) 2019-04-08 12:12:30 -07:00
hypothesis_utils.py Rewriting hypothesis_utils (#22830) 2019-07-17 10:16:13 -07:00
jit_utils.py serialize modules as classes 2019-08-11 15:50:29 -07:00
run_test.py Run quantization tests first 2019-08-14 18:09:32 -07:00
test_autograd.py Fix CTC loss for zero-length targets on GPU (#23298) 2019-07-31 12:03:45 -07:00
test_c10d_spawn.py Fix ProcessGroupGloo allgather for tensors with shared storage (#21490) 2019-06-12 11:59:17 -07:00
test_c10d.py Extract common classes and functions from test_c10d to common_distributed (#23660) 2019-08-02 09:19:32 -07:00
test_cpp_extensions.py Remove old Type based backend extensions (#22009) 2019-07-30 14:07:46 -07:00
test_cuda_primary_ctx.py pin_memory should not copy on already pinned tensors (#23484) 2019-07-30 21:16:23 -07:00
test_cuda.py Refactor randperm test (#23526) 2019-08-09 08:33:35 -07:00
test_dataloader.py make more iterator attributes private (#23744) 2019-08-09 11:43:00 -07:00
test_distributed.py Changed tensor comparison return type from uint8 to bool (#21113) 2019-08-01 07:54:53 -07:00
test_distributions.py clip sigmoid to prevent transforms return inf/nan values (#20288) 2019-06-10 11:16:31 -07:00
test_docs_coverage.py improve test_docs_coverage error messages (#21029) 2019-05-31 11:13:39 -07:00
test_expecttest.py
test_fake_quant.py Enable OSS quantization tests (#23858) 2019-08-06 11:20:30 -07:00
test_indexing_cuda.py
test_indexing.py Updated docs and added deprecation warnings to acknowledge a bool tensor (#22261) 2019-08-05 07:42:34 -07:00
test_jit_fuser.py Moves clamp from autodiff cpp to symbolic script (#23927) 2019-08-09 13:57:03 -07:00
test_jit_py3.py Fix type hints for None constants (#23029) 2019-07-22 11:55:05 -07:00
test_jit_string.py Fixes lint error in py3 (#21883) 2019-06-17 22:20:06 -07:00
test_jit.py Add the ability to compile exports on traced modules (#24298) 2019-08-14 13:51:22 -07:00
test_logging.py Don't leak threads on exit (#21438) 2019-06-10 09:14:13 -07:00
test_mkldnn.py remove RTTI check for TensorImpl shadow copy (#22773) 2019-07-15 23:21:57 -07:00
test_multiprocessing_spawn.py
test_multiprocessing.py Fix error message for a wrong fork CUDA (#23322) 2019-07-25 12:58:14 -07:00
test_namedtensor.py Name inference for softmax, log_softmax and Dimname overloads. (#24087) 2019-08-14 12:19:27 -07:00
test_namedtuple_return_api.py Add overload names to native_functions.yaml (#23532) 2019-08-01 02:08:37 -07:00
test_nccl.py
test_nn.py Resolve unused variables in tests (#24075) 2019-08-14 21:02:52 -07:00
test_numba_integration.py "CharTensor" numpy conversion is supported now (#21458) 2019-06-06 10:06:41 -07:00
test_optim.py Implement AdamW optimizer (#21250) 2019-07-02 09:09:10 -07:00
test_qat.py Enable OSS quantization tests (#23858) 2019-08-06 11:20:30 -07:00
test_quantization.py Fix Lint (#24381) 2019-08-14 19:22:09 -07:00
test_quantized_conv.py Enable FBGEMM tests under UBSAN as well (#23570) 2019-08-12 17:59:22 -07:00
test_quantized_nn_mods.py JIT trace testing 2019-08-14 22:11:32 -07:00
test_quantized_tensor.py support at::view and at::reshape for quantized tensor (#23046) 2019-07-19 23:34:04 -07:00
test_quantized.py fix lint 2019-08-14 17:37:39 -07:00
test_quantizer.py fix test (#21594) 2019-06-10 11:23:18 -07:00
test_rpc.py python udf over rpc (#23569) 2019-08-14 23:13:33 -07:00
test_sparse.py Improve error message for changing tensor metadata after .data or .detach() (#23504) 2019-07-29 22:25:14 -07:00
test_tensorboard.py added mesh plugin (#24039) 2019-08-09 10:22:43 -07:00
test_throughput_benchmark.py ThroughputBenchmark: improve formatting for ExecutionStats (#22293) 2019-07-01 14:24:34 -07:00
test_torch.py (#23574) 2019-08-14 21:12:07 -07:00
test_type_hints.py Turn on F401: Unused import warning. (#18598) 2019-03-30 09:01:17 -07:00
test_type_info.py
test_utils.py Reduce excessive CI printing in TestHub (#22043) 2019-06-21 20:08:44 -07:00