Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/73284
Some important ops won't support optional type until opset 16,
so we can't fully test things end-to-end, but I believe this should
be all that's needed. Once ONNX Runtime supports opset 16,
we can do more testing and fix any remaining bugs.
Test Plan: Imported from OSS
Reviewed By: albanD
Differential Revision: D34625646
Pulled By: malfet
fbshipit-source-id: 537fcbc1e9d87686cc61f5bd66a997e99cec287b
Co-authored-by: BowenBao <bowbao@microsoft.com>
Co-authored-by: neginraoof <neginmr@utexas.edu>
Co-authored-by: Nikita Shulga <nshulga@fb.com>
(cherry picked from commit 822e79f31ae54d73407f34f166b654f4ba115ea5)
PyTorch restricts activations to be in the range (0, 127).
In ONNX, the supported ranges are (0, 255) and (-128, 127),
respectfully, uint8 and int8. This PR extends support for range
(0, 127), by adding additional clipping when detected.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/76055
Approved by: https://github.com/garymm
Extending the support for quantization with per channel quantization.
An extra attribute `axis` can be found for per channel quantized tensors,
most commonly in quantized weight of Convolution or Linear module.
The PR adds support to correctly parse the `axis` attribute, and map to
ONNX representation in `QuantizeLinear` and `DequantizeLinear`.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/76002
Approved by: https://github.com/garymm
Previously pre-tracing model is required for exporting quantized model.
e.g. calling `traced_m = torch.jit.trace(model, inputs)` and export `traced_m`.
The reason was quantized weights are stored in a unique `PackedParam` structure,
and they need to be handled by tracing to be exportable.
This PR enables export api to call tracing underneath if it detects quantization
in the model.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/75921
Approved by: https://github.com/garymm
One of the origins for `onnx::ReduceProd` is `aten::numel`.
Adding constant fold support for `onnx::ReduceProd` closes the gap
and enables symbolic shape inference for `aten::numel` nodes that
has static shape input.
One example is `torch.nn.EmbeddingBag` when input is 2d. An `Offset`
tensor will be created by `tensor.numel()`. This `Offset` can be
properly exported as constant now, if the input has static shape.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/74082
Approved by: https://github.com/garymm
There are a few ONNX operators do not support non-float (e.g., integer) inputs at early versions. For example, Clip supports non-float types until [opset 12](https://github.com/onnx/onnx/blob/main/docs/Changelog.md#type-constraints-280), that said older versions like [opset 6](https://github.com/onnx/onnx/blob/main/docs/Changelog.md#type-constraints-107) cannot deal with integer types.
I initially find such a bug in Clip (https://github.com/pytorch/pytorch/pull/70584), but later found more:
1. Clip < 12;
2. Min/Max < 12;
3. ReLU < 14;
4. Pad < 11;
In PyTorch, if we export Max-11 with integer inputs, actually the exportation will succeed; however, fail when imported by other frameworks like ONNXRuntime.
```python
import torch
class Net(torch.nn.Module):
def __init__(self) -> None:
super().__init__()
def forward(self, x: torch.Tensor):
return torch.max(x, x + 1)
net = Net()
onnx_model = 'test.onnx'
torch.onnx.export(net, (torch.zeros((3, 3), dtype=torch.int32),),
onnx_model, verbose=True, opset_version=11)
```
This is an unexpected behavior as we want to ensure that every model exported by PyTorch is valid (https://github.com/pytorch/pytorch/pull/70584#issuecomment-1020636579). Theoretically, we can simply forbid such cases (e.g., `Clip<int>` < 12, `ReLU<int>` < 14). But actually we can enhance the compatibility and flexibility of PyTorch by simply casting inputs of those operators into float tensors, which allows the float operator functions, and then casting it back to original types.
This PR implements the second approach to achieve better compatibility in PyTorch.
@garymm @thiagocrepaldi
Pull Request resolved: https://github.com/pytorch/pytorch/pull/72401
Approved by: https://github.com/garymm, https://github.com/thiagocrepaldi
Summary:
And add a new tool to update it in the future, which follows the policy
of using "latest as of 18 months ago". This policy is meant to balance:
* recent enough to increase the odds of being able to successfully
export
* old enough to increase the odds of exported model being runnable by
different ONNX implementations
Related changes:
* test_models.py: explicitly fix opset_version to 9 rather than relying on default. Caffe2 doesn't support newer versions.
* symbolic_helper.py:
* Remove a misleading comment
* Remove unnecessary check in `_set_opset_version`
* Use a range to define `_onnx_stable_opsets`
* test_pytorch_common.py:
* Rename a variable from min -> max. I think it was a copy-paste error.
* Make skip test messages more informative.
* Remove unused `skipIfONNXShapeInference`. More on that below.
* test_pytorch_onnx_onnxruntime.py:
* Make all the `TestCase` classes explicitly specify opset version.
* Make `test_unsupported_pad` respect `opset_version` by using `run_test`
* Unrelated simplification: make it obvious that all tests run with `onnx_shape_inference=True`. AFAICT this was already the case.
* There was one test that was entirely disabled (test_tolist) because it was asking to be skipped whenever `onnx_shape_inference=True`, but it was always True. I changed the model being tested so as to preserve the intended test coverage but still have the test actually pass.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/73898
Reviewed By: msaroufim
Differential Revision: D35264615
Pulled By: malfet
fbshipit-source-id: cda8fbdffe4cc8210d8d96e659e3a9adf1b5f1d2
(cherry picked from commit b5e639e88828d34442282d0b50c977e610a2ba3a)
Fixes#74142
Previous check `dim is not None and end_dim == dim - 2` didn't consider `end_dim` being negative. However the error only occurs when input tensor has rank 1, and the rank is known to symbolic function. So a better fix is to return `input` directly when rank is 1.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/74595
Approved by: https://github.com/garymm
Avoid creating unsqueeze nodes for ListConstruct Int[] ouput case with tensor inputs
For example, if the listConstruct is (tensor[2], 1, 5), avoid adding unsqueeze nodes for tensor[2]
Pull Request resolved: https://github.com/pytorch/pytorch/pull/73927
Approved by: https://github.com/garymm
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/74372
In preparation to the multi-weight support porting, we pass explicitly the pretrained_blackbone value. We use the default value `True` for most cases, except for when the use-case is clearly a test and thus should avoid downloading the weights of the backbone.
Test Plan: running project unit-tests
Reviewed By: jdsgomes
Differential Revision: D34961147
fbshipit-source-id: cf29e42545302716a7cd3f3eb0d69e44d5fb6c73
(cherry picked from commit c4613b7abacd106d097de1b73b13af92132e1739)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/73282
Enable tests that are fixed by ORT 1.10
Test Plan: Imported from OSS
Reviewed By: jbschlosser
Differential Revision: D34625649
Pulled By: malfet
fbshipit-source-id: 103ae5275f3a7eb891ad4dd2b606ebf198a800fe
(cherry picked from commit e0ddeb4205678e993d744efa06939b0977463a76)
Original `bias` is float in PyTorch. Quantization is applied in kernel.
To mimic behavior in ONNX, export the `bias` quantization step,
then append the dequantization step to ready `bias` for unquantized operators.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/73336
Current we are unable to utilize ONNX's SpaceToDepth operator due to the lack of the mode_s attribute, hence we add an alternative symbolic in opset 9 to support pixel_unshuffle
- Adds support for pixel_unshuffle in opset9
- Adds support for dynamic input shapes for pixel_shuffle and pixel_unshuffle
Pull Request resolved: https://github.com/pytorch/pytorch/pull/72449
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/69548
* Add Concat to Scalar type analysis pass
By using scalar type analysis for Concat, the exported model can do
automatic type promotion for Concat nodes, including mixed fp16 and fp32
inputs, for example.
Unit tests based on the original PR https://github.com/pytorch/pytorch/pull/24378/
* Fix UTs
Test Plan: Imported from OSS
Reviewed By: msaroufim
Differential Revision: D32994268
Pulled By: malfet
fbshipit-source-id: 0deab88b0bb1e396770690af27730accb64fcf63
(cherry picked from commit a99322cadf)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/68491
* Allows implementing symbolic functions for domains other than `aten`, for example `prim`, in symbolic_opset#.py.
* Allows symbolic function to access extra context if needed, through `SymbolicFunctionState`.
* Particularly, the `prim::PythonOp` special case can access node without the need of passing node through inputs. Updates will be made downstreams, and in a follow-up PR we will remove the previous workaround in exporter.
* `prim::Loop`, `prim::If`, etc are now moved outside of `_run_symbolic_function` from utils.py, and to symbolic_opset9.py.
Motivation for this change:
- Better maintainability and reducing complexity. Easier to add symbolic for operators, both simple and complex ones (that need additional context), without the former needing to know the existence of the latter.
- The design idea was long outdated. prim ops are no longer rare special cases, and they shouldn't all be handled inside `_run_symbolic_function`. As a result this function becomes too clumsy. There were also prim ops symbolic added in symbolic_opset#.py with signature `prim_[opname]`, creating separation and confusion.
Test Plan: Imported from OSS
Reviewed By: jansel
Differential Revision: D32483782
Pulled By: malfet
fbshipit-source-id: f9affc31b1570af30ffa6668da9375da111fd54a
Co-authored-by: BowenBao <bowbao@microsoft.com>
(cherry picked from commit 1e04ffd2fd)
Summary:
Convert type comments in caffe2/test/onnx/
Produced by running:
```
python -m libcst.tool codemod convert_type_comments.ConvertTypeComment caffe2/test/onnx/
```
from the parent directory.
One question is whether we actually want to scrap type comment here. There are some jit tests where we're explicitly aiming to validate py2-style type comments; I don't think this test is one of those cases but if I'm misreading it I can close the PR.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/72632
Reviewed By: msaroufim
Differential Revision: D34112196
Pulled By: stroxler
fbshipit-source-id: a3d18cb36e7eeb4af9be781e98776bf24b96b854
(cherry picked from commit 9301019e51)
