There are a few ONNX operators do not support non-float (e.g., integer) inputs at early versions. For example, Clip supports non-float types until [opset 12](https://github.com/onnx/onnx/blob/main/docs/Changelog.md#type-constraints-280), that said older versions like [opset 6](https://github.com/onnx/onnx/blob/main/docs/Changelog.md#type-constraints-107) cannot deal with integer types.
I initially find such a bug in Clip (https://github.com/pytorch/pytorch/pull/70584), but later found more:
1. Clip < 12;
2. Min/Max < 12;
3. ReLU < 14;
4. Pad < 11;
In PyTorch, if we export Max-11 with integer inputs, actually the exportation will succeed; however, fail when imported by other frameworks like ONNXRuntime.
```python
import torch
class Net(torch.nn.Module):
def __init__(self) -> None:
super().__init__()
def forward(self, x: torch.Tensor):
return torch.max(x, x + 1)
net = Net()
onnx_model = 'test.onnx'
torch.onnx.export(net, (torch.zeros((3, 3), dtype=torch.int32),),
onnx_model, verbose=True, opset_version=11)
```
This is an unexpected behavior as we want to ensure that every model exported by PyTorch is valid (https://github.com/pytorch/pytorch/pull/70584#issuecomment-1020636579). Theoretically, we can simply forbid such cases (e.g., `Clip<int>` < 12, `ReLU<int>` < 14). But actually we can enhance the compatibility and flexibility of PyTorch by simply casting inputs of those operators into float tensors, which allows the float operator functions, and then casting it back to original types.
This PR implements the second approach to achieve better compatibility in PyTorch.
@garymm @thiagocrepaldi
Pull Request resolved: https://github.com/pytorch/pytorch/pull/72401
Approved by: https://github.com/garymm, https://github.com/thiagocrepaldi
Summary:
And add a new tool to update it in the future, which follows the policy
of using "latest as of 18 months ago". This policy is meant to balance:
* recent enough to increase the odds of being able to successfully
export
* old enough to increase the odds of exported model being runnable by
different ONNX implementations
Related changes:
* test_models.py: explicitly fix opset_version to 9 rather than relying on default. Caffe2 doesn't support newer versions.
* symbolic_helper.py:
* Remove a misleading comment
* Remove unnecessary check in `_set_opset_version`
* Use a range to define `_onnx_stable_opsets`
* test_pytorch_common.py:
* Rename a variable from min -> max. I think it was a copy-paste error.
* Make skip test messages more informative.
* Remove unused `skipIfONNXShapeInference`. More on that below.
* test_pytorch_onnx_onnxruntime.py:
* Make all the `TestCase` classes explicitly specify opset version.
* Make `test_unsupported_pad` respect `opset_version` by using `run_test`
* Unrelated simplification: make it obvious that all tests run with `onnx_shape_inference=True`. AFAICT this was already the case.
* There was one test that was entirely disabled (test_tolist) because it was asking to be skipped whenever `onnx_shape_inference=True`, but it was always True. I changed the model being tested so as to preserve the intended test coverage but still have the test actually pass.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/73898
Reviewed By: msaroufim
Differential Revision: D35264615
Pulled By: malfet
fbshipit-source-id: cda8fbdffe4cc8210d8d96e659e3a9adf1b5f1d2
(cherry picked from commit b5e639e88828d34442282d0b50c977e610a2ba3a)
Fixes#74142
Previous check `dim is not None and end_dim == dim - 2` didn't consider `end_dim` being negative. However the error only occurs when input tensor has rank 1, and the rank is known to symbolic function. So a better fix is to return `input` directly when rank is 1.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/74595
Approved by: https://github.com/garymm
Avoid creating unsqueeze nodes for ListConstruct Int[] ouput case with tensor inputs
For example, if the listConstruct is (tensor[2], 1, 5), avoid adding unsqueeze nodes for tensor[2]
Pull Request resolved: https://github.com/pytorch/pytorch/pull/73927
Approved by: https://github.com/garymm
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/74372
In preparation to the multi-weight support porting, we pass explicitly the pretrained_blackbone value. We use the default value `True` for most cases, except for when the use-case is clearly a test and thus should avoid downloading the weights of the backbone.
Test Plan: running project unit-tests
Reviewed By: jdsgomes
Differential Revision: D34961147
fbshipit-source-id: cf29e42545302716a7cd3f3eb0d69e44d5fb6c73
(cherry picked from commit c4613b7abacd106d097de1b73b13af92132e1739)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/73282
Enable tests that are fixed by ORT 1.10
Test Plan: Imported from OSS
Reviewed By: jbschlosser
Differential Revision: D34625649
Pulled By: malfet
fbshipit-source-id: 103ae5275f3a7eb891ad4dd2b606ebf198a800fe
(cherry picked from commit e0ddeb4205678e993d744efa06939b0977463a76)
Original `bias` is float in PyTorch. Quantization is applied in kernel.
To mimic behavior in ONNX, export the `bias` quantization step,
then append the dequantization step to ready `bias` for unquantized operators.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/73336
Current we are unable to utilize ONNX's SpaceToDepth operator due to the lack of the mode_s attribute, hence we add an alternative symbolic in opset 9 to support pixel_unshuffle
- Adds support for pixel_unshuffle in opset9
- Adds support for dynamic input shapes for pixel_shuffle and pixel_unshuffle
Pull Request resolved: https://github.com/pytorch/pytorch/pull/72449
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/69548
* Add Concat to Scalar type analysis pass
By using scalar type analysis for Concat, the exported model can do
automatic type promotion for Concat nodes, including mixed fp16 and fp32
inputs, for example.
Unit tests based on the original PR https://github.com/pytorch/pytorch/pull/24378/
* Fix UTs
Test Plan: Imported from OSS
Reviewed By: msaroufim
Differential Revision: D32994268
Pulled By: malfet
fbshipit-source-id: 0deab88b0bb1e396770690af27730accb64fcf63
(cherry picked from commit a99322cadf)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/68491
* Allows implementing symbolic functions for domains other than `aten`, for example `prim`, in symbolic_opset#.py.
* Allows symbolic function to access extra context if needed, through `SymbolicFunctionState`.
* Particularly, the `prim::PythonOp` special case can access node without the need of passing node through inputs. Updates will be made downstreams, and in a follow-up PR we will remove the previous workaround in exporter.
* `prim::Loop`, `prim::If`, etc are now moved outside of `_run_symbolic_function` from utils.py, and to symbolic_opset9.py.
Motivation for this change:
- Better maintainability and reducing complexity. Easier to add symbolic for operators, both simple and complex ones (that need additional context), without the former needing to know the existence of the latter.
- The design idea was long outdated. prim ops are no longer rare special cases, and they shouldn't all be handled inside `_run_symbolic_function`. As a result this function becomes too clumsy. There were also prim ops symbolic added in symbolic_opset#.py with signature `prim_[opname]`, creating separation and confusion.
Test Plan: Imported from OSS
Reviewed By: jansel
Differential Revision: D32483782
Pulled By: malfet
fbshipit-source-id: f9affc31b1570af30ffa6668da9375da111fd54a
Co-authored-by: BowenBao <bowbao@microsoft.com>
(cherry picked from commit 1e04ffd2fd)
Summary:
Convert type comments in caffe2/test/onnx/
Produced by running:
```
python -m libcst.tool codemod convert_type_comments.ConvertTypeComment caffe2/test/onnx/
```
from the parent directory.
One question is whether we actually want to scrap type comment here. There are some jit tests where we're explicitly aiming to validate py2-style type comments; I don't think this test is one of those cases but if I'm misreading it I can close the PR.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/72632
Reviewed By: msaroufim
Differential Revision: D34112196
Pulled By: stroxler
fbshipit-source-id: a3d18cb36e7eeb4af9be781e98776bf24b96b854
(cherry picked from commit 9301019e51)
Summary:
- PyTorch and ONNX has supported BFloat16, add this to unblock some mixed-precision training model.
- Support PyTorch TNLG model to use BFloat16 tensors for the inputs/outputs of the layers that run on the NPU.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/66788
Reviewed By: jansel
Differential Revision: D32283510
Pulled By: malfet
fbshipit-source-id: 150d69b1465b2b917dd6554505eca58042c1262a
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/67808
torch.reciprocal implicitly casts the inputs to float, and ONNX
Reciprocal requires floating point inputs.
Also separate the reciprocal test from other tests, and test different
input types.
Test Plan: Imported from OSS
Reviewed By: msaroufim
Differential Revision: D32181307
Pulled By: malfet
fbshipit-source-id: 3e1109b3c85a49c51dc713656a900b4ee78c8340
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/67806
Previously new_full would fail with errors like:
`TypeError: only integer tensors of a single element can be converted to an index`
And full_like would trigger warnings like:
`DeprecationWarning: an integer is required (got type float). Implicit conversion to integers using __int__ is deprecated, and may be removed in a future version of Python.`
Test Plan: Imported from OSS
Reviewed By: msaroufim
Differential Revision: D32181301
Pulled By: malfet
fbshipit-source-id: 2cf262cfef36c18e7b2423efe1e1d4fa3438f0ba
Co-authored-by: Bowen Bao <bowbao@microsoft.com>
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/67805
Also fix Reduce ops on binary_cross_entropy_with_logits
The graph says the output is a scalar but with `keepdims=1`
(the default), the output should be a tensor of rank 1. We set keep
`keepdims=0` to make it clear that we want a scalar output.
This previously went unnoticed because ONNX Runtime does not strictly
enforce shape inference mismatches if the model is not using the latest
opset version.
Test Plan: Imported from OSS
Reviewed By: msaroufim
Differential Revision: D32181304
Pulled By: malfet
fbshipit-source-id: 1462d8a313daae782013097ebf6341a4d1632e2c
Co-authored-by: Bowen Bao <bowbao@microsoft.com>
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/67803
* Addresses comments from #63589
[ONNX] remove torch::onnx::PRODUCER_VERSION (#67107)
Use constants from version.h instead.
This simplifies things since we no longer have to update
PRODUCER_VERSION for each release.
Also add TORCH_VERSION to version.h so that a string is available for
this purpose.
[ONNX] Set `ir_version` based on opset_version. (#67128)
This increases the odds that the exported ONNX model will be usable.
Before this change, we were setting the IR version to a value which may
be higher than what the model consumer supports.
Also some minor clean-up in the test code:
* Fix string replacement.
* Use a temporary file so as to not leave files around in the test
current working directory.
Test Plan: Imported from OSS
Reviewed By: msaroufim
Differential Revision: D32181306
Pulled By: malfet
fbshipit-source-id: 02f136d34ef8f664ade0bc1985a584f0e8c2b663
Co-authored-by: BowenBao <bowbao@microsoft.com>
Co-authored-by: Gary Miguel <garymiguel@microsoft.com>
Co-authored-by: Nikita Shulga <nshulga@fb.com>
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/67270
* Add dim argument to all symbolic
* All symbolic depends on any symbolic
Test Plan: Imported from OSS
Reviewed By: msaroufim
Differential Revision: D31962518
Pulled By: malfet
fbshipit-source-id: f7ee05cf4eff5880fc508154267e060952b5b42d
