pytorch/torch/distributed/algorithms
Yi Wang c1415a0a72 [Reland] [Model Averaging] Simplify PostLocalSGD Optimizer API (#65197)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/65197

1. The constructor accepts a local optimizer instance instead of the inputs of local optimizer constructor and the class type.
2. The parameters are read from local optimizer's param_groups instead of a separate input.

Proposal: https://github.com/pytorch/pytorch/issues/59699
ghstack-source-id: 138307226

Test Plan: buck test mode/dev-nosan //caffe2/test/distributed:distributed_nccl_spawn -- test_post_localSGD_optimizer_parity

Reviewed By: rohan-varma

Differential Revision: D31007439

fbshipit-source-id: bbb0526e6763ef76775b85088571506b3942c722
2021-09-17 10:31:58 -07:00
..
ddp_comm_hooks [DDP Comm Hook] Create a noop hook for performance debugging (#64344) 2021-09-01 17:36:22 -07:00
model_averaging [Reland] [Model Averaging] Simplify PostLocalSGD Optimizer API (#65197) 2021-09-17 10:31:58 -07:00
quantization Adding alltoall_single collective to collective quantization API (#63154) 2021-08-27 12:46:31 -07:00
__init__.py Make _Join, _Joinable, _JoinHook public (#62605) 2021-08-03 12:20:11 -07:00
join.py Make _Join, _Joinable, _JoinHook public (#62605) 2021-08-03 12:20:11 -07:00