Rohan Varma 1dba329d20 Enable step_param for Adam functional optimizer (#62611) ... Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/62611 Enables optimizer overlap with backwards in DDP for Adam. Additional optimizers, especially Adagrad will be done in follow up diffs. 1. Implement `step_param` method based on `step` in _FunctionalAdam (perf permitting we can later dedupe `step` to call `step_param` 2. Modify tests to test all current functional optimizers. ghstack-source-id: 135207143 Test Plan: CI Reviewed By: SciPioneer Differential Revision: D29891783 fbshipit-source-id: 321915982afd5cb0a9c2e43d27550f433bff00d1		2021-08-06 10:53:55 -07:00
..
_sharded_tensor	Initialize RRefs only when explicitly asked for. (#62618)	2021-08-02 22:17:17 -07:00
_sharding_spec	[sharded_tensor] add new init_from_local_shards API (#60479)	2021-07-29 22:04:13 -07:00
algorithms	[DDP Communication Hook] Update get_tensor and set_tensor to be cleaner naming conventions (buffer() and set_buffer()) (#62662)	2021-08-04 09:27:31 -07:00
autograd	Add Python declaration of torch._C and torch._C._autograd modules. (#46622)	2020-11-06 01:25:47 -08:00
benchmarks	Add lint for unqualified type: ignore (#56290)	2021-04-21 08:07:23 -07:00
elastic	[torchelastic][multiprocessing] Print warning message only when child processes are stuck (#62823)	2021-08-05 19:57:31 -07:00
launcher	[torchelastic] Make sure rdzv_configs[timeout] is not getting overwritten (#61471)	2021-07-09 15:27:00 -07:00
nn	remove unused type: ignore directives (#60006)	2021-06-18 07:23:31 -07:00
optim	Enable step_param for Adam functional optimizer (#62611)	2021-08-06 10:53:55 -07:00
pipeline	Adds _LazyInstanceNorm and LazyInstanceNormXd (#60982)	2021-07-21 06:45:45 -07:00
rpc	[profiler][refactor] Refactor the usage of legacy profiler implementation (#61931)	2021-08-03 18:51:29 -07:00
__init__.py	[C10d] Support subgroups (#59111)	2021-06-09 22:35:11 -07:00
argparse_util.py	[19/n][torch/elastic][upstream] Replace pytorch.distributed.launch with torchelastic launcher (#56214)	2021-04-16 13:38:23 -07:00
constants.py	make ProcessGroupDefaultTimeout the same as python (#56549)	2021-04-21 17:56:05 -07:00
CONTRIBUTING.md	[c10d] Move pg wrapper tests to their own file. (#59840)	2021-06-14 15:05:55 -07:00
distributed_c10d.py	Adding warning on isend about modifying after send (#61875)	2021-07-29 07:37:18 -07:00
launch.py	[torch] Various improvements to torch.distributed.launch and torch.distributed.run (#61294)	2021-07-08 16:28:06 -07:00
rendezvous.py	[torch/elastic] Fix the agent store key prefix used by workers (#61590)	2021-07-13 14:57:27 -07:00
run.py	[torch] Set nproc_per_node to 1 (#61552)	2021-07-13 13:35:25 -07:00
utils.py	Introduce ChunkShardingSpec as a model sharding specification. (#55728)	2021-05-23 16:04:57 -07:00