Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/51759
Some unit tests actually register a comm hook on other backends like GLOO. Example: `test_ddp_comm_hook_future_passing_cpu`
Therefore, only do the check on `register_builtin_comm_hook`.
Currently DDP communication hook can only be supported on NCCL. Add a check in the registration methods.
ghstack-source-id: 121115814
Test Plan: unit tests.
Reviewed By: pritamdamania87
Differential Revision: D26268581
fbshipit-source-id: c739fa4dca6d320202dc6689d790c2761c834c30
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/50590
Larger blocking across M dim such as 8 in previous PR is likely
introducing wasted compute on the shapes being benchmarked.
Here we introduced 4x8 blocking of mrxnr. This helps 1) in packing
smaller data for small values of M and 2) for compute kernel it writes
same number of bytes but more contiguously. It is not certain but it
likely helps.
Test Plan:
q8gemm-sparse-test
fully-connected-sparse-test
Imported from OSS
Reviewed By: AshkanAliabadi
Differential Revision: D25925499
fbshipit-source-id: 01c661ceea38bd6ee8321bb85cf1d5da5de4e984
Summary:
Usage explanation will be in the release note runbook.
This allows to generate diffs like:
```
Processing torch.nn
Things that were added:
{'quantizable', 'ChannelShuffle', 'LazyConvTranspose2d', 'LazyConv2d', 'LazyConvTranspose3d', 'LazyConv1d', 'GaussianNLLLoss', 'LazyConv3d', 'PixelUnshuffle', 'UninitializedParameter', 'LazyLinear', 'LazyConvTranspose1d'}
Things that were removed:
set()
```
This can then be shared with module owners along with the commits to help them validate that the namespace changes for their submodule is as expected.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/51685
Reviewed By: zhangguanheng66
Differential Revision: D26260258
Pulled By: albanD
fbshipit-source-id: 40e40f86314e17246899d01ffa4b2631e93b52f7
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/51683
**Summary**
This commit enables implicit boolean conversion of lists, strings, and
dictionaries in conditional expressions. Like Python, empty lists,
strings and dictionaries evaluate to `False` and their non-empty
counterparts evaluate to `True`. This allows users to write code like
```
torch.jit.script
def fn(l: List[int]):
if l:
...
else:
...
```
This has been requested by some users and would be a good usability
improvement.
**Test Plan**
This commit adds unit tests to `TestList`, `TestDict` and
`test_jit_string.py` to test this new feature.
Test Plan: Imported from OSS
Reviewed By: pbelevich
Differential Revision: D26264410
Pulled By: SplitInfinity
fbshipit-source-id: b764c18fd766cfc128ea98a02b7c6c3fa49f8632
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/50983
There is currently no way to handle/propagate errors with the python-based futures API (they are raised correctly if set with an error, but this is only possible from C++).
This diff allows the Future's `unwrap_func` to be set in python optionally, so users can set futures completed with an exception and the error will throw as expected. This is mostly to support the following use case in the next diff:
```
ret_fut = torch.futures.Future(unwrap_func = lambda python_result: {
# throw exception if needed
if isinstance(python_result, Exception):
throw python_result
})
rpc_fut = rpc.rpc_async(...) # RPC future that times out
# Goal is to propagate RPC error to this future
rpc_fut.add_done_callback(
res => {
# Note that ret_fut.set_result(res.wait()) won't propagate the error
try:
ret_fut.set_result(res.wait())
except Exception as e:
ret_fut.set_result(e)
}
)
```
ghstack-source-id: 121021434
Test Plan:
unittest
```
buck test mode/dev-nosan mode/no-gpu //caffe2/test:futures -- te
st_unwrap --print-passing-details
```
Reviewed By: mrshenli
Differential Revision: D25950304
fbshipit-source-id: 7ee61e98fcd783b3f515706fa141d538e6d2174d
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/50979
Noticed that the documentation is inconsisntent about the arg needed
in the callback. It appears to require the future, so fix this in the docs.
ghstack-source-id: 121021431
Test Plan: CI
Reviewed By: mrshenli
Differential Revision: D25944637
fbshipit-source-id: 0bfcd4040c4a1c245314186d29a0031e634b29c3
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/50978
Noticed that the documentation is not clear that the cbs are invoked
inline if the future is already completed. We should probably document this
behavior.
ghstack-source-id: 121021432
Test Plan: CI
Reviewed By: mrshenli
Differential Revision: D25944636
fbshipit-source-id: f4ac133d076ba9a5690fecfa56bde6d614a40191
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/50977
Adds a `blocking` flag that can be set to False to make this API return a `Future` to the type. This is to make this function non-blocking, mostly for a future change that will allow `rref.rpc_async()` to be completely non-blocking (it currently calls and waits for this function that issues an RPC in-line).
ghstack-source-id: 121021433
Test Plan: Modified UT
Reviewed By: mrshenli
Differential Revision: D25944582
fbshipit-source-id: e3b48a52af2d4578551a30ba6838927b489b1c03
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/50589
Adds 1. Input prepacking kernel 2. Compute kernels that processes
prepacked activation.
Hunch is that input prepacking will help with 1. Cache locality and 2.
Avoid a lot of address compute instructions.
Cache locality helps mainly comes from the fact that we are doing mr=8
and nr=4.
mr being 8 likely results in cache line evictions as likely cache
associativity is 4. Laying out transposed activations which are blocked
by mr=8 will lay all the transposed activation in one contiguous block.
Downside is that now we will tranpose all the blocks regardless of them
participating in compute. However it is likely that entire activation
matrix participates in compute for some output block.
Also add benchmark
Test Plan:
q8gemm-sparse-test
fully-connected-test-sparse
Imported from OSS
Reviewed By: AshkanAliabadi
Differential Revision: D25925502
fbshipit-source-id: b2c36419a2c5d23b4a49f25f9ee41cee8397c3be
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/51706
Pull Request resolved: https://github.com/pytorch/pytorch/pull/50280
As mentioned in gh-43874, this adds a `rounding_mode={'true', 'trunc', 'floor'}`
argument so `torch.div` can be used as a replacement for `floor_divide` during
the transitional period.
I've included dedicated kernels for truncated and floor division which
aren't strictly necessary for float, but do perform significantly better (~2x) than
doing true division followed by a separate rounding kernel.
Note: I introduce new overloads for `aten::div` instead of just adding a default
`rounding_mode` because various JIT passes rely on the exact operator schema.
Test Plan: Imported from OSS
Reviewed By: ngimel
Differential Revision: D26123271
Pulled By: mruberry
fbshipit-source-id: 51a83717602114597ec9c4d946e35a392eb01d46
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/51572
Modifications in remove_inplace_ops_for_onnx pass and remove_inplace_ops pass to better handle binary inplace ops
* Handles special case of binary inplace ops, where the first input node has a lower type precedence than the second input node.
* When the inplace node is converted to a regular op, this information is lost and the resulting type is based on type precedence, just like regular ops. To avoid this loss of information, we add a cast node before the input node with the higher data type precedence, so that both the input types are the same.
Test Plan: Imported from OSS
Reviewed By: pbelevich
Differential Revision: D26203117
Pulled By: SplitInfinity
fbshipit-source-id: f018b503701b9067dba053c2764c3b92ef1abc38
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/51524
* def unsafe_chunk() support and test in ops13.
* Use _unsqueeze_helper insteadof Unsqueeze operator
* Cast the splits into long.
* Change the test to a fixed dimension.
* Update test_pytorch_onnx_onnxruntime.py
* Disable test_loop_with_list for opset 13.
Test Plan: Imported from OSS
Reviewed By: pbelevich
Differential Revision: D26203123
Pulled By: SplitInfinity
fbshipit-source-id: b273aeff8339faa0e8e9f1fcfbf877d1b703209f
Co-authored-by: Negin Raoof <neginmr@utexas.edu>
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/51521
* Add loop & if node to the list of nodes that could produce sequence type output.
* Switch from `[]` to `at()` to avoid segfault of out of range access.
Test Plan: Imported from OSS
Reviewed By: pbelevich
Differential Revision: D26203112
Pulled By: SplitInfinity
fbshipit-source-id: e990eeed933124b195be0be159271e33fb485063
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/51520
Previous insertBefore approach might end-up inserting clone node in inner sub-blocks, while then the node being used later at other outside call sites.
Test Plan: Imported from OSS
Reviewed By: pbelevich
Differential Revision: D26203124
Pulled By: SplitInfinity
fbshipit-source-id: 999511e901ad1087f360bb689fcdfc3743c78aa4
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/51518
* enable remaining test in opset13
* add comments for error version test info
* fix comments:opset12 unbind problem
* add ignore[no-redef]
* fix format
Test Plan: Imported from OSS
Reviewed By: pbelevich
Differential Revision: D26203122
Pulled By: SplitInfinity
fbshipit-source-id: e7d95bd2ce13f79f11965be82f640379cd55ff0f
Co-authored-by: hwangdeyu <deyhuang@qq.com>
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/51517
Fix get/set attributes when getting/setting a model parameter.
This PR also fixes inplace ops in If blocks.
Test Plan: Imported from OSS
Reviewed By: pbelevich
Differential Revision: D26203116
Pulled By: SplitInfinity
fbshipit-source-id: bed6ee6dd92b5b43febc8c584a6872290f8fe33f
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/51514
Update constant-folding of Gather operator so it also includes cases where rank of indices input is 0.
Currently it only support cases where rank of indices is 1.
Test Plan: Imported from OSS
Reviewed By: pbelevich
Differential Revision: D26191323
Pulled By: SplitInfinity
fbshipit-source-id: 7edcbd8835b0248fefb908aca394f5cca5eae29e
