pytorch/torch/distributed/algorithms
Rohan Varma c4281cc92d Prototype checkpoint_wrapper (#69955)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/69955

Implements a checkpoint_wrapper function, which wraps nn.Module with checkpointing so user won't have to call checkpoint() everytime they want to checkpoint the module.

Currently only support for reentrant-based checkpointing is added and only tested with FSDP to unblock a use case.

Future work is to add support for new checkpointing API, add more tests, upstream to torch.utils.checkpoint.
ghstack-source-id: 145811242

Test Plan: CI

Reviewed By: mrshenli

Differential Revision: D33107276

fbshipit-source-id: c4a1c68d71d65713a929994940a8750f73fbdbdb
2021-12-16 09:59:19 -08:00
..
_checkpoint Prototype checkpoint_wrapper (#69955) 2021-12-16 09:59:19 -08:00
ddp_comm_hooks [DDP Comm Hook] Create a noop hook for performance debugging (#64344) 2021-09-01 17:36:22 -07:00
model_averaging [Reland] [Model Averaging] Simplify PostLocalSGD Optimizer API (#65197) 2021-09-17 10:31:58 -07:00
quantization [BE] minor improvement to dist quantization (#67401) 2021-11-21 23:31:59 -08:00
__init__.py Make _Join, _Joinable, _JoinHook public (#62605) 2021-08-03 12:20:11 -07:00
join.py Make _Join, _Joinable, _JoinHook public (#62605) 2021-08-03 12:20:11 -07:00