Andrew Gu 15953fdf35 [FSDP][8/N] Replace _FSDPPolicy.policy with _Policy._run_policy (#104969) ... This does some code organization improvement. - It renames `_FSDPPolicy` to `_Policy` to show that it is not only for FSDP but for any module-level API. - It formalizes the contract that such a policy should return something like `target_module_to_kwargs: Dict[nn.Module, Dict[str, Any]]` that maps each module to wrap to its kwargs. It does so by requiring a `_run_policy` abstract method (this time private since users do not need to care about it). Then, our auto wrapping can just call `_run_policy()` to generate the dict and do any validation or post-processing. This PR is technically BC-breaking because it removes the public `ModuleWrapPolicy.policy`. However, I do not think anyone was using that anyway, so this is a pretty safe breakage. Pull Request resolved: https://github.com/pytorch/pytorch/pull/104969 Approved by: https://github.com/rohan-varma ghstack dependencies: #104427, #104967, #104999		2023-08-03 12:42:14 +00:00
..
_checkpoint	[FSDP][8/N] Replace _FSDPPolicy.policy with _Policy._run_policy (#104969)	2023-08-03 12:42:14 +00:00
_comm_hooks	[FSDP] Optimize away intermediate div_ for HSDP (#106034)	2023-07-28 18:36:26 +00:00
_optimizer_overlap
_quantization
ddp_comm_hooks	Enable fused optimizer for DP (#98270)	2023-04-13 20:16:32 +00:00
model_averaging	Convert logging f-strings to use % format, part four (#98705)	2023-04-11 13:17:59 +00:00
__init__.py
join.py	[BE] Enable ruff's UP rules and autoformat distributed/ (#105433)	2023-07-19 14:27:11 +00:00