OSSForks/pytorch
1
0
Fork 0
mirror of https://github.com/zebrajr/pytorch.git synced 2025-12-07 12:21:27 +01:00
Code Issues Actions 37 Packages Projects Releases Wiki Activity
e392d428b1
pytorch/caffe2/python
History
Hassan Eslami e392d428b1 Allowing TaskGroups to carry remote nets (#14342) 
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/14342

Sometimes, when we are creating a TaskGroup, we are in fact creating a TaskGroup for a distributed job. In some cases, we may want to register a few nets as "remote" to a TaskGroup. The remote net should have sufficient attributes on where they should be executed later on.

This diff adds the remote net attribute to the TaskGroup class. It exposes two minimal functionalities: adding a remote net, and getting all remote nets added to a TaskGroup.

Reviewed By: d4l3k

Differential Revision: D13188320

fbshipit-source-id: efe947aec30817e9512a5e18be985713b9356bdc
2018-11-27 13:34:11 -08:00
..
docs adapting caffe2 operator docs generator to pytorch url 2018-10-11 12:55:06 -07:00
examples Add resnext model to OSS (#11468) 2018-09-12 15:59:20 -07:00
helpers [Caffe2] Add full impl of GroupNorm (#7058) 2018-04-29 11:26:40 -07:00
ideep Add "axis" and "axis_w" arguments in FC to support customized axix to reduce dim. (#12971) 2018-11-21 15:44:50 -08:00
layers Add Recency Weighted into SparseLookup (#14291) 2018-11-24 02:43:31 -08:00
mint move flags to c10 (#12144) 2018-10-04 02:09:56 -07:00
mkl seperate mkl, mklml, and mkldnn (#12170) 2018-10-29 10:52:55 -07:00
modeling diagnose option: get_entry to print a whole row (#11308) 2018-09-06 21:26:30 -07:00
models Add resnext model to OSS (#11468) 2018-09-12 15:59:20 -07:00
onnx Fix caffe2 => onnx exporter for ConvTranspose (#14143) 2018-11-26 15:51:42 -08:00
operator_test Added MIOpen conv transpose op (#13938) 2018-11-13 21:01:52 -08:00
predictor Caffe2: Fix for creating entries of external_input in predic_net (#12979) 2018-11-15 22:33:50 -08:00
rnn
serialized_test operator serialized test coverage summary document (#13703) 2018-11-09 15:04:08 -08:00
test Remove async_polling (#13825) 2018-11-15 16:23:15 -08:00
trt Clean up a couple of items in the C2 test scaffolding (WIP) (#7847) 2018-11-07 09:16:13 -08:00
__init__.py caffe2::DeviceType -> at::DeviceType (#11254) 2018-09-05 16:28:09 -07:00
_import_c_extension.py Completely remove build_aten and use_aten (#10469) 2018-08-20 20:26:42 -07:00
allcompare_test.py
attention.py [Caffe2] Update elementwise ops to support numpy style boradcast (#8070) 2018-06-05 15:49:16 -07:00
benchmark_generator.py
binarysize.py
brew_test.py Move tanh function to math (#9328) 2018-07-11 13:59:50 -07:00
brew.py [Caffe2] Changes done inside Facebook (#6378) 2018-04-10 21:11:43 -07:00
build.py
cached_reader.py Update from Facebook (#8887) 2018-06-26 14:55:48 -07:00
caffe_translator_test.py Fix skip logic in caffe_translator_test.py (#13627) 2018-11-15 16:45:49 -08:00
caffe_translator.py Fix bug in caffe_translator tool (#10056) 2018-10-11 13:13:12 -07:00
checkpoint_test.py Revert D9566744: [New Checkpoint] Kill the dummy TaskOutput when task.get_step() (#11164) 2018-08-31 22:25:57 -07:00
checkpoint.py Create class constant for string literal 'blob_names' 2018-08-24 22:11:43 -07:00
CMakeLists.txt seperate mkl, mklml, and mkldnn (#12170) 2018-10-29 10:52:55 -07:00
cnn.py Rename cuda_gpu_id to device_id in DeviceOption (#12456) 2018-10-09 15:54:04 -07:00
compatibility.py migrating deprecated calls without abc module for containers (#11515) 2018-09-13 15:09:22 -07:00
context_test.py
context.py Resolve name conflict of ContextManager (#7244) 2018-06-22 00:41:51 -04:00
control_ops_grad.py
control_ops_util.py
control_test.py
control.py
convert_test.py New serialization format (#12384) 2018-10-16 16:36:58 -07:00
convert.py New serialization format (#12384) 2018-10-16 16:36:58 -07:00
convnet_benchmarks_test.py
convnet_benchmarks.py
core_gradients_test.py [Caffe2] Update elementwise ops to support numpy style boradcast (#8070) 2018-06-05 15:49:16 -07:00
core_test.py device inference for Adam (#13990) 2018-11-14 17:21:08 -08:00
core.py Back out "Migrate DeviceOption.numa_node_id to DeviceOption.device_id" 2018-10-24 17:11:25 -07:00
crf_predict.py Move crf in caffe2 from fb to oss (#12200) 2018-10-01 18:31:41 -07:00
crf_viterbi_test.py Move crf in caffe2 from fb to oss (#12200) 2018-10-01 18:31:41 -07:00
crf.py Productionize CRF layer in PyText (#10362) 2018-08-22 00:25:26 -07:00
data_parallel_model_test.py Support GPU version of Spatial Batch Norm (#11711) 2018-10-18 11:22:13 -07:00
data_parallel_model.py video model test workflow on CPU (#13203) 2018-10-26 20:48:18 -07:00
data_workers_test.py
data_workers.py Fixed log message (#10874) 2018-09-05 09:55:52 -07:00
dataio_test.py Update from Facebook (#8887) 2018-06-26 14:55:48 -07:00
dataio.py Fixing stop condition on composite reader (#9888) 2018-08-20 03:02:20 -07:00
dataset.py Update from facebook (#7855) 2018-05-29 11:38:02 -07:00
db_file_reader.py Update from Facebook (#8887) 2018-06-26 14:55:48 -07:00
db_test.py
device_checker.py Update from facebook (#7451) 2018-05-10 23:14:27 -07:00
dlpack.h Upgrade DLPack 2018-11-12 15:59:46 -08:00
dyndep.py
embedding_generation_benchmark.py
experiment_util.py
extension_loader.py Completely remove build_aten and use_aten (#10469) 2018-08-20 20:26:42 -07:00
functional_test.py Add support for specifying device_option in Functional (#9619) 2018-07-24 14:41:59 -07:00
functional.py Caffe2 Functional enforcing inplace output (#10797) 2018-08-23 22:42:47 -07:00
fused_8bit_rowwise_conversion_ops_test.py
gradient_check_test.py Enable junk fill for the default CPU allocator (#13377) 2018-11-08 00:02:37 -08:00
gradient_checker.py make the variable declaration closer to usage 2018-10-12 12:07:08 -07:00
gru_cell.py
hip_test_util.py Make CUDNN an alias of MIOPEN for HIP ops (#12278) 2018-10-24 17:07:31 -07:00
hsm_util.py
hypothesis_test_util.py Clean up a couple of items in the C2 test scaffolding (WIP) (#7847) 2018-11-07 09:16:13 -08:00
hypothesis_test.py Remove unsafecoalesce op (#12897) 2018-10-22 09:42:26 -07:00
ideep_test_util.py [feature request] [Caffe2] Enable MKLDNN support for inference (#6699) 2018-04-22 21:58:14 -07:00
layer_model_helper.py parallize the dense part in event models 2018-08-22 22:40:07 -07:00
layer_model_instantiator.py [caffe2] Fbcode to GitHub sync (#6208) 2018-04-02 16:35:27 -07:00
layer_parameter_sharing_test.py Clean up a couple of items in the C2 test scaffolding (WIP) (#7847) 2018-11-07 09:16:13 -08:00
layer_test_util.py
layers_test.py Add Recency Weighted into SparseLookup (#14291) 2018-11-24 02:43:31 -08:00
lengths_reducer_fused_8bit_rowwise_ops_test.py
lengths_reducer_rowwise_8bit_ops_test.py
lstm_benchmark.py
memonger_test.py
memonger.py
mkl_test_util.py
model_device_test.py
model_helper_test.py keep net type info when generating model complete net (#11032) 2018-09-04 21:10:06 -07:00
model_helper.py Rename cuda_gpu_id to device_id in DeviceOption (#12456) 2018-10-09 15:54:04 -07:00
modifier_context.py
mpi_python.cc
muji_test.py [Caffe2] Support non peer access in muji and fix bug when reduced_affix is empty (#6896) 2018-06-05 03:14:43 +08:00
muji.py Rename cuda_gpu_id to device_id in DeviceOption (#12456) 2018-10-09 15:54:04 -07:00
net_builder_test.py
net_builder.py Update from Facebook (#6692) 2018-04-17 23:36:40 -07:00
net_drawer.py
net_printer_test.py
net_printer.py Rename cuda_gpu_id to device_id in DeviceOption (#12456) 2018-10-09 15:54:04 -07:00
nomnigraph_test.py nomnigraph - support subgraph visualization (#13795) 2018-11-16 08:19:20 -08:00
nomnigraph_transformations_test.py Add transpose network pass (#13437) 2018-11-01 14:27:07 -07:00
nomnigraph_transformations.py Add transpose network pass (#13437) 2018-11-01 14:27:07 -07:00
nomnigraph.py createUniqueDataNode 2018-10-31 11:16:38 -07:00
normalizer_context.py Update from Facebook (#8887) 2018-06-26 14:55:48 -07:00
normalizer_test.py Update from Facebook (#8887) 2018-06-26 14:55:48 -07:00
normalizer.py Enable alternative LayerNorm impl in FisherGan (#12178) 2018-10-11 17:36:11 -07:00
numa_benchmark.py Back out "Migrate DeviceOption.numa_node_id to DeviceOption.device_id" 2018-10-24 17:11:25 -07:00
numa_test.py Back out "Migrate DeviceOption.numa_node_id to DeviceOption.device_id" 2018-10-24 17:11:25 -07:00
observer_test.py
optimizer_context.py
optimizer_test_util.py Implementation of Wngrad optimizer caffe2 python wrapper and unit test on least square regression (#9001) 2018-07-13 18:54:52 -07:00
optimizer_test.py Add Adadelta optimizer to caffe2 (#9088) 2018-07-24 20:09:21 -07:00
optimizer.py Rename cuda_gpu_id to device_id in DeviceOption (#12456) 2018-10-09 15:54:04 -07:00
parallel_workers_test.py
parallel_workers.py Update from facebook (#7696) 2018-05-19 23:10:48 -07:00
parallelize_bmuf_distributed_test.py
pipeline_test.py
pipeline.py SNNTest with Data Preproc Service (#11707) 2018-09-17 21:25:49 -07:00
predictor_constants.py
pybind_state_dlpack.cc Upgrade DLPack 2018-11-12 15:59:46 -08:00
pybind_state_dlpack.h Upgrade DLPack 2018-11-12 15:59:46 -08:00
pybind_state_gpu.cc Renaming dims() to sizes() (caffe2/caffe2) - 4/4 2018-10-24 16:32:51 -07:00
pybind_state_hip.cc Change hip filename extension to .hip (#14036) 2018-11-16 11:55:59 -08:00
pybind_state_ideep.cc FeedTensor returns a Tensor (#14196) 2018-11-26 13:05:44 -08:00
pybind_state_int8.cc Renaming meta() to dtype() - 2/2 (#13334) 2018-10-30 18:24:30 -07:00
pybind_state_nomni.cc nomnigraph - support subgraph visualization (#13795) 2018-11-16 08:19:20 -08:00
pybind_state_registry.cc Move registry fully to c10 (#12077) 2018-09-27 03:09:54 -07:00
pybind_state_registry.h Move registry fully to c10 (#12077) 2018-09-27 03:09:54 -07:00
pybind_state.cc FeedTensor returns a Tensor (#14196) 2018-11-26 13:05:44 -08:00
pybind_state.h FeedTensor returns a Tensor (#14196) 2018-11-26 13:05:44 -08:00
python_op_test.py Clean up a couple of items in the C2 test scaffolding (WIP) (#7847) 2018-11-07 09:16:13 -08:00
queue_util.py
record_queue.py
recurrent.py
regularizer_context.py Update from Facebook (#8887) 2018-06-26 14:55:48 -07:00
regularizer_test.py Add GroupL1Norm regularizer (#9115) 2018-07-06 13:26:09 -07:00
regularizer.py Add GroupL1Norm regularizer (#9115) 2018-07-06 13:26:09 -07:00
rnn_cell.py Rewrite http://pytorch.org -> https://pytorch.org throughout project (#12636) 2018-10-15 13:03:27 -07:00
schema_test.py Add util function from core type to dtype (#10716) 2018-08-21 10:55:19 -07:00
schema.py Add util function from core type to dtype (#10716) 2018-08-21 10:55:19 -07:00
scope_test.py Update from Facebook (#8887) 2018-06-26 14:55:48 -07:00
scope.py Update from Facebook (#8887) 2018-06-26 14:55:48 -07:00
session_test.py
session.py
sparse_to_dense_mask_test.py
sparse_to_dense_test.py
task.py Allowing TaskGroups to carry remote nets (#14342) 2018-11-27 13:34:11 -08:00
test_util.py Enable junk fill for the default CPU allocator (#13377) 2018-11-08 00:02:37 -08:00
text_file_reader.py
timeout_guard.py
toy_regression_test.py Enable junk fill for the default CPU allocator (#13377) 2018-11-08 00:02:37 -08:00
transformations_test.py nomnigraph - easy - some code cleanup for transformations_test (#12101) 2018-10-01 11:31:08 -07:00
transformations.py Enable Conv fusion optimizations in optimizeForIdeep (#9255) 2018-07-16 21:28:50 -07:00
tt_core_test.py
tt_core.py
utils_test.py Convert Arguments to dictionary (#13436) 2018-11-01 14:27:05 -07:00
utils.py Convert Arguments to dictionary (#13436) 2018-11-01 14:27:05 -07:00
visualize.py
workspace_test.py Enable junk fill for the default CPU allocator (#13377) 2018-11-08 00:02:37 -08:00
workspace.py Adding Fetching Real number representation 2018-11-05 23:35:24 -08:00