Xu Zhao
7f66fa62ca
Fix typing errors in torch.distributed.nn.* directory. ( #47533 )
...
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/47533
Test Plan: Imported from OSS
Reviewed By: walterddr
Differential Revision: D24952500
Pulled By: xuzhao9
fbshipit-source-id: 8e66784fd8f9f111b6329e0bb48d6cd61c690a4a
2020-11-16 23:27:55 -08:00
Yi Wang
cab32d9cdf
[RPC Framework] Support remote device format "<workername>/<device>" ( #46773 )
...
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/46773
Changed the constructor of RemoteModule to accept a `remote_device` arg in the following format:
"<workername>/<device>" (e.g., "trainer0/cpu", "ps0/cuda:0")
This arg merges the original `on` and `device` arg.
Original PR issue: RemoteDevice Format #46554
ghstack-source-id: 115448051
Test Plan: buck test mode/dev-nosan caffe2/test/distributed/rpc:process_group_agent -- RemoteModule
Reviewed By: pritamdamania87
Differential Revision: D24482562
fbshipit-source-id: 5acfc73772576a4b674df27625bf560b8f8e67c1
2020-10-29 00:14:56 -07:00
Yi Wang
c68cc78299
Add a device parameter to RemoteModule ( #44254 )
...
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/44254
Add a device parameter to RemoteModule, so it can be placed on any device
and not just CPU.
Original PR issue: RemoteModule enhancements #40550
Test Plan: buck test test/distributed/rpc:process_group_agent -- RemoteModule
Reviewed By: pritamdamania87
Differential Revision: D23483803
fbshipit-source-id: 4918583c15c6a38a255ccbf12c9168660ab7f6db
2020-09-18 10:31:03 -07:00
Yi Wang
396469f18c
Explicitly forbidden the other inherited methods of RemoteModule. ( #43895 )
...
Summary:
Throw exceptions when the methods except for forwardXXX are used.
Original PR issue: RemoteModule enhancements #40550
Pull Request resolved: https://github.com/pytorch/pytorch/pull/43895
Test Plan: buck test test/distributed/rpc:process_group_agent -- RemoteModule
Reviewed By: rohan-varma
Differential Revision: D23392842
Pulled By: SciPioneer
fbshipit-source-id: 7c09a55a03f9f0b7e9f9264a42bfb907607f4651
2020-09-05 14:48:56 -07:00
Yi Wang
8b17fd2516
Add remote_parameters() into RemoteModule class. ( #43906 )
...
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/43906
This method returns a list of RRefs of remote parameters that can be fed into the DistributedOptimizer.
Original PR issue: RemoteModule enhancements #40550
Test Plan: buck test caffe2/test/distributed/rpc:process_group_agent -- RemoteModule
Reviewed By: rohan-varma
Differential Revision: D23399586
fbshipit-source-id: 4b0f1ccf2e47c8a9e4f79cb2c8668f3cdbdff820
2020-09-04 16:22:40 -07:00
wudenggang
9600ed9af3
typo fixes ( #41632 )
...
Summary:
typo fixes
Pull Request resolved: https://github.com/pytorch/pytorch/pull/41632
Reviewed By: ezyang
Differential Revision: D22617827
Pulled By: mrshenli
fbshipit-source-id: c2bfcb7cc36913a8dd32f13fc9adc3aa0a9b682f
2020-07-20 07:23:00 -07:00
Stanislau Hlebik
b774ce54f8
remediation of S205607
...
fbshipit-source-id: 798decc90db4f13770e97cdce3c0df7d5421b2a3
2020-07-17 17:19:47 -07:00
Stanislau Hlebik
8fdea489af
remediation of S205607
...
fbshipit-source-id: 5113fe0c527595e4227ff827253b7414abbdf7ac
2020-07-17 17:17:03 -07:00
Shihao Xu
00651b8c93
[distribtued.nn] Implement TorchScript-compatible RemoteModule API ( #37139 )
...
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/37139
See design doc in https://github.com/pytorch/pytorch/issues/37136
ghstack-source-id: 105926270
Test Plan:
TODO:
- Make the generated Interface usable. https://github.com/pytorch/pytorch/pull/37139#discussion_r434190978
-
- Avoid generating the same template instances for Module that is not scriptable.
- Remove "infer_module_interface_cls".
- Use Python format instead of a CodeTemplate
- Use Python tempfile to track and delete file. Does it work if there is crash.
```
buck test mode/dev-nosan //caffe2/test/distributed/nn/jit:test_instantiator
buck build mode/dev-nosan //caffe2/test/distributed/nn/jit:test_instantiator && \
buck-out/gen/caffe2/test/distributed/nn/jit/test_instantiator\#binary.par -r test_instantiate_scripted_remote_module_template
buck build mode/dev-nosan //caffe2/test/distributed/nn/jit:test_instantiator && \
buck-out/gen/caffe2/test/distributed/nn/jit/test_instantiator\#binary.par -r test_instantiate_non_scripted_remote_module_template
```
```
buck test mode/dev-nosan //caffe2/test/distributed/nn/api:remote_module_spawn
```
```
buck test mode/dev-nosan //caffe2/test/distributed/nn/api:remote_module_fork
buck build mode/dev-nosan //caffe2/test/distributed/nn/api:remote_module_fork && \
buck-out/gen/caffe2/test/distributed/nn/api/remote_module_fork\#binary.par -r test_user_provided_global_unique_name
buck build mode/dev-nosan //caffe2/test/distributed/nn/api:remote_module_fork && \
buck-out/gen/caffe2/test/distributed/nn/api/remote_module_fork\#binary.par -r test_forward_async_script
buck build mode/dev-nosan //caffe2/test/distributed/nn/api:remote_module_fork && \
buck-out/gen/caffe2/test/distributed/nn/api/remote_module_fork\#binary.par -r test_forward_sync_script
buck build mode/dev-nosan //caffe2/test/distributed/nn/api:remote_module_fork && \
buck-out/gen/caffe2/test/distributed/nn/api/remote_module_fork\#binary.par -r test_forward_with_kwargs
buck build mode/dev-nosan //caffe2/test/distributed/nn/api:remote_module_fork && \
buck-out/gen/caffe2/test/distributed/nn/api/remote_module_fork\#binary.par -r test_user_provided_global_unique_name
```
```
buck test mode/dev-nosan //caffe2/test/distributed/rpc:rpc_fork
```
buck test mode/opt-asan //caffe2/test:jit -- 'test_script_forward_method_replacement
buck build mode/dev-nosan //caffe2/test:jit && \
buck-out/gen/caffe2/test/jit\#binary.par -r 'test_script_forward_method_replacement'
buck build mode/dev-nosan //caffe2/test:jit && \
buck-out/gen/caffe2/test/jit\#binary.par -r 'test_imported_classes'
Differential Revision: D20499658
fbshipit-source-id: dd9383ae4eb2343366c11127664f845b91ca3b0a
2020-06-15 19:07:35 -07:00
