pytorch/torch/distributed/algorithms
Yi Wang aeea5bf4a1 [Model Averaging] Provide a util function for model averaging (#60303)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/60303

The util function can be used for averaging parameters.

More optimizations can be done in the future.
ghstack-source-id: 132214212

Test Plan:
buck test mode/dev-nosan caffe2/test/distributed:distributed_nccl_fork -- test_average_parameters
buck test mode/dev-nosan caffe2/test/distributed:distributed_gloo_fork -- test_average_parameters

Reviewed By: rohan-varma

Differential Revision: D29242806

fbshipit-source-id: 76fb5a92adb4bdc6151a9f411e366a0ed2a31f47
2021-06-23 15:41:15 -07:00
..
ddp_comm_hooks [Reland][Gradient Compression] Apply division first to avoid overflow (#59576) 2021-06-08 10:03:21 -07:00
model_averaging [Model Averaging] Provide a util function for model averaging (#60303) 2021-06-23 15:41:15 -07:00
__init__.py [Gradient Compression] Add unit tests that test default Python comm hook implementations (#47158) 2020-11-06 00:28:09 -08:00