Isaac Seessel 3078233e9a [Gradient Compression] Make FP16 compression as a wrapper that can be combined with other communication hooks (#53808) ... Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/53808 Create a FP16 wrapper that can combine FP16 gradient compression with any gradient compression algorithm. Test Plan: Unit test: ``` buck test mode/dev-nosan caffe2/test/distributed:c10d -- test_fp16_compress_wrapper ``` Performance Test on DDP QPS Benchmark: Check if AllReduce + FP16 Wrapper = FP16 Compression 1) FP16 Compression: f256897690 2) FP16 Wrapper + AllReduce (after patching D26960986): f256897289 Reviewed By: SciPioneer Differential Revision: D26978832 fbshipit-source-id: 0dcd18b050c02f5e9f3cff56344d1f39a04e20c0		2021-03-12 17:31:07 -08:00
..
ddp_comm_hooks	[Gradient Compression] Make FP16 compression as a wrapper that can be combined with other communication hooks (#53808)	2021-03-12 17:31:07 -08:00
__init__.py	[Gradient Compression] Add unit tests that test default Python comm hook implementations (#47158)	2020-11-06 00:28:09 -08:00