OSSForks/pytorch
1
0
Fork 0
mirror of https://github.com/zebrajr/pytorch.git synced 2025-12-07 00:21:07 +01:00
Code Issues Actions 38 Packages Projects Releases Wiki Activity

[DDP Note] Remove refs to RoundRobin PG until we officially support it (#40380)

Browse Source 
Summary:
Removes line mentioning `ProcessGroupRoundRobin` since we don't intend it to be used as a public API just yet. We can add this back when we officially support the API
Pull Request resolved: https://github.com/pytorch/pytorch/pull/40380

Differential Revision: D22165556

Pulled By: rohan-varma

fbshipit-source-id: 24d0477d881dc74f2ff579de61dfd1ced2b09e75
This commit is contained in:
Rohan Varma 2020-06-22 16:17:16 -07:00 committed by Facebook GitHub Bot
parent 016cf7d66e
commit ae2f1f0372
1 changed files with 5 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

13
docs/source/notes/ddp.rst
View File

@ -145,14 +145,11 @@ ProcessGroup
- `ProcessGroup.hpp <https://github.com/pytorch/pytorch/blob/v1.4.0/torch/lib/c10d/ProcessGroup.hpp>`__:
contains the abstract API of all process group implementations. The ``c10d``
library provides 4 implementations out of the box, namely,
`ProcessGroupGloo`, `ProcessGroupNCCL`, `ProcessGroupMPI`, and
`ProcessGroupRoundRobin`, where `ProcessGroupRoundRobin` is a composition of
multiple process group instances and launches collective communications in a
round-robin manner. ``DistributedDataParallel`` uses
``ProcessGroup::broadcast()`` to send model states from the process with rank
0 to others during initialization and ``ProcessGroup::allreduce()`` to sum
gradients.
library provides 3 implementations out of the box, namely,
`ProcessGroupGloo`, `ProcessGroupNCCL`, and `ProcessGroupMPI`.
``DistributedDataParallel`` uses ``ProcessGroup::broadcast()`` to send
model states from the process with rank 0 to others during initialization
and ``ProcessGroup::allreduce()`` to sum gradients.
- `Store.hpp <https://github.com/pytorch/pytorch/blob/v1.4.0/torch/lib/c10d/Store.hpp>`__: