Use `typing_extensions.deprecated` for deprecation annotation if possible. Otherwise, add `category=FutureWarning` to `warnings.warn("message")` if the category is missing.
Note that only warnings that their messages contain `[Dd]eprecat(ed|ion)` are updated in this PR.
Resolves#126888
- #126888
This PR is split from PR #126898.
- #126898
------
Pull Request resolved: https://github.com/pytorch/pytorch/pull/127689
Approved by: https://github.com/Skylion007
Use `typing_extensions.deprecated` for deprecation annotation if possible. Otherwise, add `category=FutureWarning` to `warnings.warn("message")` if the category is missing.
Note that only warnings that their messages contain `[Dd]eprecat(ed|ion)` are updated in this PR.
UPDATE: Use `FutureWarning` instead of `DeprecationWarning`.
Resolves#126888
- #126888
Pull Request resolved: https://github.com/pytorch/pytorch/pull/126898
Approved by: https://github.com/albanD
Summary:
Avoids the following deprecation warning:
```python
loss.backward(*args, **kwargs)
/usr/local/lib/python3.7/dist-packages/torch/tensor.py:245: in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
/usr/local/lib/python3.7/dist-packages/torch/autograd/__init__.py:147: in backward
allow_unreachable=True, accumulate_grad=True) # allow_unreachable flag
/usr/local/lib/python3.7/dist-packages/torch/autograd/function.py:89: in apply
return self._forward_cls.backward(self, *args) # type: ignore
/usr/local/lib/python3.7/dist-packages/torch/nn/parallel/_functions.py:34: in backward
return (None,) + ReduceAddCoalesced.apply(ctx.input_device, ctx.num_inputs, *grad_outputs)
/usr/local/lib/python3.7/dist-packages/torch/nn/parallel/_functions.py:45: in forward
return comm.reduce_add_coalesced(grads_, destination)
/usr/local/lib/python3.7/dist-packages/torch/nn/parallel/comm.py:143: in reduce_add_coalesced
flat_result = reduce_add(flat_tensors, destination)
/usr/local/lib/python3.7/dist-packages/torch/nn/parallel/comm.py:96: in reduce_add
nccl.reduce(inputs, output=result, root=root_index)
/usr/local/lib/python3.7/dist-packages/torch/cuda/nccl.py:69: in reduce
_check_sequence_type(inputs)
/usr/local/lib/python3.7/dist-packages/torch/cuda/nccl.py:48: in _check_sequence_type
if not isinstance(inputs, collections.Container) or isinstance(inputs, torch.Tensor):
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
name = 'Container'
def __getattr__(name):
# For backwards compatibility, continue to make the collections ABCs
# through Python 3.6 available through the collections module.
# Note, no new collections ABCs were added in Python 3.7
if name in _collections_abc.__all__:
obj = getattr(_collections_abc, name)
import warnings
warnings.warn("Using or importing the ABCs from 'collections' instead "
"of from 'collections.abc' is deprecated since Python 3.3,"
"and in 3.9 it will stop working",
> DeprecationWarning, stacklevel=2)
E DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working
/usr/lib/python3.7/collections/__init__.py:52: DeprecationWarning
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/72239
Reviewed By: ngimel
Differential Revision: D34387815
Pulled By: mruberry
fbshipit-source-id: 30c9b4fe518351bc9a6f211269e27ee3ab73a13c
(cherry picked from commit 1f68cdfac5)
Summary:
https://github.com/pytorch/pytorch/issues/62295
Previously the packing and unpacking of the NCCL version "integer" was done to have parity with the upstream NCCL version encoding. However, there doesn't seem to be any place where this integer is directly compared with a version integer sourced from upstream NCCL, and syncing the encoding seems to be error-prone (e.g., a recent change where a special case was added for minor versions >= 10 7e51592129/src/nccl.h.in (L22)).
This patch changes the reporting to return a tuple of version numbers instead (to preserve ease-of-use for comparisons) and tweaks the passing between C/Python to avoid the digit overflow problem.
CC ngimel mcarilli
Pull Request resolved: https://github.com/pytorch/pytorch/pull/62916
Reviewed By: anjali411
Differential Revision: D30201069
Pulled By: mrshenli
fbshipit-source-id: 2e4e7c69f001c3f22bd04aa6df6a992e538bea45
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/43247
`torch.cuda.nccl` APIs didn't throw appropriate errors when called
with inputs/outputs that were of the wrong type and it resulted in some cryptic
errors instead.
Adding some error checks with explicit error messages for these APIs.
ghstack-source-id: 110683546
Test Plan: waitforbuildbot
Reviewed By: rohan-varma
Differential Revision: D23206069
fbshipit-source-id: 8107b39d27f4b7c921aa238ef37c051a9ef4d65b
Summary:
While working on https://github.com/pytorch/pytorch/issues/38911, I realized that `nccl.reduce` only needs a single output tensor, while our current implementation requires a list of output tensors. This, along with a TODO I fixed in reduce_add, should have some speed up for data parallel.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/39688
Differential Revision: D22034547
Pulled By: mrshenli
fbshipit-source-id: e74d54d673ebbb062474b1bb5cc93a095a3a5f6c
Adds streams and comms as optional arguments to the NCCL calls in
torch.cuda.nccl. Also exposes ncclUniqueId and ncclCommInitRank for
multi-process mode.
Moves Py_RETURN_NONE statements after the GIL is re-acquired.
This ensures that we use the same library at the C++ level and with
Python ctypes. It moves the searching for the correct library from
run-time to compile-time.
Here's the command I used to invoke autopep8 (in parallel!):
git ls-files | grep '\.py$' | xargs -n1 -P`nproc` autopep8 -i
Several rules are ignored in setup.cfg. The goal is to let autopep8
handle everything which it can handle safely, and to disable any rules
which are tricky or controversial to address. We may want to come back
and re-enable some of these rules later, but I'm trying to make this
patch as safe as possible.
Also configures flake8 to match pep8's behavior.
Also configures TravisCI to check the whole project for lint.
