- Rename `test_pytorch_common` -> `pytorch_test_common`, `test_onnx_common` -> `onnx_test_common`, removing the test_ prefix to show that the files are not test cases
- Remove import * in `test_pytorch_common` and adjust to import from `testing._internal.common_utils` (where functions are actually defined) instead
- Import modules only in `test_pytorch_onnx_onnxruntime` (too many to handle in a single PR in other tests) (The skips are exceptions)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/81141
Approved by: https://github.com/BowenBao
If installed with pep517 support, `torchvision` will be build againstreleased version of PyTorch rather than against the one currently installed on the system
Also update `torchvision` hash to 8a45147f9d and:
- Added `maskrcnn_resnet50_fpn_v2`, `maskrcnn_resnet50_fpn_v2`, `retinanet_resnet50_fpn_v2`, `ssd300_vgg16`, `fcos_resnet50_fpn` and `ssdlite320_mobilenet_v3_large` to the list of untraceable models
- Set default input size to (1, 3, 16, 224, 224) for `mvit_v1_b` model
- Skipped `test_roi_aligned`,`test_batched_nms`, `test_roi_pooled` and `test_roi_align_aligned` ONNX test (tracked in https://github.com/pytorch/pytorch/issues/81121 )
- Skipped TorchVision integration tests in `test_package` (tracked in https://github.com/pytorch/pytorch/issues/81115 )
Pull Request resolved: https://github.com/pytorch/pytorch/pull/81074
Approved by: https://github.com/kit1980
- moved test_uninitialized_optional into runtime test, because **OptionalHasElement** node now is supported on onnxruntime. test_optional_input will be moved later when tracing/None output paradox is fixed.
- changed _TestONNXRuntime to inherited by common_utils.Testcase as it's more rich on error message
- modified the left itertools implemented test cases to using parametrized decorator
Pull Request resolved: https://github.com/pytorch/pytorch/pull/80145
Approved by: https://github.com/justinchuby, https://github.com/BowenBao
* Move memory heavy tests from `test_pytorch_onnx_onnxruntime.py` to
`test_models_onnxruntime.py`. The former is run in parallel in CI,
while the latter is not. A change is that the moved tests are now
only covered in default opset export.
* Refactor and create base class for tests that export model to ONNX
and verify with ONNX Runtime. The new base class are parameterized
with `opset_version` and `is_script`. Further work can be done to
refactor existing test classes in `test_pytorch_onnx_onnxruntime.py`.
See #75630
* Reduce unnecessarily large tensor size in
`test_pytorch_onnx_onnxruntime.py` to further reduce memory usage
and test time.
After this PR, the running time for `test_pytorch_onnx_onnxruntime.py`
is reduced from `1338.82s (0:22:18)` to `225.07s (0:03:45)`,
benchmarked on 10900x with `-n 10`.
Fixes#79179
Pull Request resolved: https://github.com/pytorch/pytorch/pull/79640
Approved by: https://github.com/justinchuby, https://github.com/garymm
When `TrainingMode.PRESERVE` is set for export, the exporter used to change the model's training mode based on some logic. Now we respect the option and not touch the model's training state.
- Previously `_set_training_mode`'s behavior doesn't match what the global variable expects. This PR removes the deprecated `_set_training_mode` and makes the type correct.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/78583
Approved by: https://github.com/BowenBao
- Add quantization support for `interpolate`, `avgpool`, `sigmoid` and `add_relu`
- Return the inputs to ListUnpack if the previous node is ListConstruct so that `ListConstruct` and `ListUnpack` are canceled and removed in the jit passes. ONNX doesn't support them.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/78103
Approved by: https://github.com/garymm
This is a simple fix addressing the exportation when the input to `torch.log2` is scalar. `log2(x)` will be exported as `log(x) / log(2)`, which creates a `log` node followed by a `div` node that divides it by a constant. The constant is constructed not as a scalar but as a tensor of shape `[1]`, so a scalar input here will get broadcasted creating the output tensor with shape `[1]`, while originally the torch model's output is a scalar.
```python
import torch
import onnx
import numpy as np
class Model(torch.nn.Module):
def forward(self, x):
return torch.log2(x)
x = torch.tensor(1.) # scalar
model = Model()
torch.onnx.export(model, (x, ), "output.onnx", opset_version=14,
output_names=['o0'], input_names=['i0'])
y_trh = model(x).numpy()
model = onnx.load("output.onnx")
print(model.graph.output[0])
import onnxruntime as ort
sess = ort.InferenceSession(
"output.onnx", providers=['CPUExecutionProvider'])
y_ort = sess.run(['o0'], {'i0': x.numpy()})[0]
assert y_ort.shape == y_trh.shape, 'shape mismatch, ORT is `{}` but PyTorch is `{}`'.format(
y_ort.shape, y_trh.shape)
```
The resulting ONNX model has an output of shape `[1]` and causes shape mismatch between ORT and PyTorch. The output:
```
name: "o0"
type {
tensor_type {
elem_type: 1
shape {
dim {
dim_value: 1
}
}
}
}
Traceback (most recent call last):
File "test.py", line 501, in <module>
y_ort.shape, y_trh.shape)
AssertionError: shape mismatch, ORT is `(1,)` but PyTorch is `()`
```
After the fix, the output becomes:
```
name: "o0"
type {
tensor_type {
elem_type: 1
shape {
}
}
}
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/78701
Approved by: https://github.com/justinchuby, https://github.com/BowenBao
The default for `torch.onnx.export` is `TrainingMode.EVAL`:
0d76299ff7/torch/onnx/__init__.py (L63)
That means that this warning is only printed when the caller overrides
that and explicitly specifies that they want training ops like Dropout.
We should assume the user knows what they're doing and not warn.
Also set `do_constant_folding=False` in the dropout related training tests. Without this, warnings are printed like:
```
UserWarning: It is recommended that constant folding be turned off ('do_constant_folding=False') when exporting the model in training-amenable mode
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/78309
Approved by: https://github.com/justinchuby, https://github.com/BowenBao
Use pyupgrade(https://github.com/asottile/pyupgrade) and flynt to modernize python syntax
```sh
pyupgrade --py36-plus --keep-runtime-typing torch/onnx/**/*.py
pyupgrade --py36-plus --keep-runtime-typing test/onnx/**/*.py
flynt torch/onnx/ --line-length 120
```
- Use f-strings for string formatting
- Use the new `super()` syntax for class initialization
- Use dictionary / set comprehension
Pull Request resolved: https://github.com/pytorch/pytorch/pull/77935
Approved by: https://github.com/BowenBao
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/73284
Some important ops won't support optional type until opset 16,
so we can't fully test things end-to-end, but I believe this should
be all that's needed. Once ONNX Runtime supports opset 16,
we can do more testing and fix any remaining bugs.
Test Plan: Imported from OSS
Reviewed By: albanD
Differential Revision: D34625646
Pulled By: malfet
fbshipit-source-id: 537fcbc1e9d87686cc61f5bd66a997e99cec287b
Co-authored-by: BowenBao <bowbao@microsoft.com>
Co-authored-by: neginraoof <neginmr@utexas.edu>
Co-authored-by: Nikita Shulga <nshulga@fb.com>
(cherry picked from commit 822e79f31ae54d73407f34f166b654f4ba115ea5)
PyTorch restricts activations to be in the range (0, 127).
In ONNX, the supported ranges are (0, 255) and (-128, 127),
respectfully, uint8 and int8. This PR extends support for range
(0, 127), by adding additional clipping when detected.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/76055
Approved by: https://github.com/garymm
Extending the support for quantization with per channel quantization.
An extra attribute `axis` can be found for per channel quantized tensors,
most commonly in quantized weight of Convolution or Linear module.
The PR adds support to correctly parse the `axis` attribute, and map to
ONNX representation in `QuantizeLinear` and `DequantizeLinear`.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/76002
Approved by: https://github.com/garymm
Previously pre-tracing model is required for exporting quantized model.
e.g. calling `traced_m = torch.jit.trace(model, inputs)` and export `traced_m`.
The reason was quantized weights are stored in a unique `PackedParam` structure,
and they need to be handled by tracing to be exportable.
This PR enables export api to call tracing underneath if it detects quantization
in the model.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/75921
Approved by: https://github.com/garymm
One of the origins for `onnx::ReduceProd` is `aten::numel`.
Adding constant fold support for `onnx::ReduceProd` closes the gap
and enables symbolic shape inference for `aten::numel` nodes that
has static shape input.
One example is `torch.nn.EmbeddingBag` when input is 2d. An `Offset`
tensor will be created by `tensor.numel()`. This `Offset` can be
properly exported as constant now, if the input has static shape.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/74082
Approved by: https://github.com/garymm
There are a few ONNX operators do not support non-float (e.g., integer) inputs at early versions. For example, Clip supports non-float types until [opset 12](https://github.com/onnx/onnx/blob/main/docs/Changelog.md#type-constraints-280), that said older versions like [opset 6](https://github.com/onnx/onnx/blob/main/docs/Changelog.md#type-constraints-107) cannot deal with integer types.
I initially find such a bug in Clip (https://github.com/pytorch/pytorch/pull/70584), but later found more:
1. Clip < 12;
2. Min/Max < 12;
3. ReLU < 14;
4. Pad < 11;
In PyTorch, if we export Max-11 with integer inputs, actually the exportation will succeed; however, fail when imported by other frameworks like ONNXRuntime.
```python
import torch
class Net(torch.nn.Module):
def __init__(self) -> None:
super().__init__()
def forward(self, x: torch.Tensor):
return torch.max(x, x + 1)
net = Net()
onnx_model = 'test.onnx'
torch.onnx.export(net, (torch.zeros((3, 3), dtype=torch.int32),),
onnx_model, verbose=True, opset_version=11)
```
This is an unexpected behavior as we want to ensure that every model exported by PyTorch is valid (https://github.com/pytorch/pytorch/pull/70584#issuecomment-1020636579). Theoretically, we can simply forbid such cases (e.g., `Clip<int>` < 12, `ReLU<int>` < 14). But actually we can enhance the compatibility and flexibility of PyTorch by simply casting inputs of those operators into float tensors, which allows the float operator functions, and then casting it back to original types.
This PR implements the second approach to achieve better compatibility in PyTorch.
@garymm @thiagocrepaldi
Pull Request resolved: https://github.com/pytorch/pytorch/pull/72401
Approved by: https://github.com/garymm, https://github.com/thiagocrepaldi
