Mark Astley 4bf90558e0 [Gradient Compression] Add logging for gradient compression stats. (#54647) ... Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/54647 Regularly log stats showing effect of gradient compression when using the PowerSGD DDP communication hook. Test Plan: buck run mode/dev-nosan scripts/wayi/torch:power_sgd Play with the layer sizes of the input model (you can just use linear layers for convenience), and check the log that shows compression stats. For convenience, you can change `logging.info` to `print` locally. You can create some test diffs on top of this diff, to show that the compression stats are correct in different cases. Run with power_sgd script: {F537381542} Diff with example using a simple linear model: D27299934 sample output: {F538486535} Reviewed By: SciPioneer Differential Revision: D27240254 fbshipit-source-id: 9e142b2f7957cc874804f799b7bb3bffdf824858		2021-03-25 07:44:17 -07:00
..
ddp_comm_hooks	[Gradient Compression] Add logging for gradient compression stats. (#54647)	2021-03-25 07:44:17 -07:00
__init__.py	[Gradient Compression] Add unit tests that test default Python comm hook implementations (#47158)	2020-11-06 00:28:09 -08:00