mirror of
https://github.com/zebrajr/pytorch.git
synced 2025-12-07 00:21:07 +01:00
9b69da2b55
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/19821 It is possible that not a single parameter is used during an iteration. If this is the case, the `prepare_for_backward` function marks all parameters as unused, kicks off reduction of all buckets, *and* finalizes the reduction. This is different from the prior implementation where we assumed that autograd would produce a gradient for at least a single parameter. We then used the autograd callback mechanism to queue a finalizer callback. Now, this finalizer may be executed in line. Reviewed By: mrshenli Differential Revision: D15113272 fbshipit-source-id: dc91458b569cd8c106ddaeea558464b515683550 |
||
|---|---|---|
| .. | ||
| _thnn | ||
| autograd | ||
| backends | ||
| contrib | ||
| csrc | ||
| cuda | ||
| distributed | ||
| distributions | ||
| for_onnx | ||
| jit | ||
| legacy | ||
| lib | ||
| multiprocessing | ||
| nn | ||
| onnx | ||
| optim | ||
| sparse | ||
| testing | ||
| utils | ||
| __config__.py | ||
| __init__.py | ||
| __init__.pyi.in | ||
| _jit_internal.py | ||
| _ops.py | ||
| _six.py | ||
| _storage_docs.py | ||
| _tensor_docs.py | ||
| _tensor_str.py | ||
| _torch_docs.py | ||
| _utils_internal.py | ||
| _utils.py | ||
| abi-check.cpp | ||
| CMakeLists.txt | ||
| extension.h | ||
| functional.py | ||
| hub.py | ||
| py.typed | ||
| quasirandom.py | ||
| random.py | ||
| README.txt | ||
| script.h | ||
| serialization.py | ||
| storage.py | ||
| tensor.py | ||
Note [TH abstraction violation] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ TH/THC provide some hpp headers, which are proper C++ headers rather than C headers. These headers serve double duty as *internal implementation detail* headers, whose contents should largely not be used by external clients. Ideally, we would not install these headers at all; instead, you should use public functions (in headers like `THTensor.h`, NOT `THTensor.hpp`) to manipulate these structs. However, there are a few places in torch/csrc where we violate this abstraction. They are marked with a pointer to this note. Each of those sites will have to be refactored when we refactor the guts of THTensor and related structures.