gen_static_runtime_ops hasn't been updated in a while. In preparation for https://github.com/pytorch/pytorch/pull/127675 in which I need to re-run the codegen step for cumprod, I want to land these changes beforehand in case there are any other issues that arise.
I added a number of ops to the blocklist:
```
+ "_nested_tensor_storage_offsets",
+ "_nested_get_values", # no CPU backend
+ "_nested_get_values_copy", # no CPU backend
+ "_nested_view_from_jagged", # testing needs to be patched
+ "_nested_view_from_jagged_copy", # testing needs to be patched
+ "_nested_view_from_buffer", # testing needs to be patched
+ "_nested_view_from_buffer_copy", # testing needs to be patched
+ "_int_mm", # testing needs to be patched
+ "_to_sparse_csc", # testing needs to be patched
+ "_to_sparse_csr", # testing needs to be patched
+ "segment_reduce", # testing needs to be patched
```
Most of these are added just because testing doesn't work right now.
Additionally, a few `fft` ops seem to have been removed from native_functions.yaml; I'm guessing it's unlikely FFT would have been used in many real models though.
Differential Revision: [D58329403](https://our.internmc.facebook.com/intern/diff/D58329403/)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/128299
Approved by: https://github.com/YuqingJ
This build uses the wrong BUILD_ENVIRONMENT `pytorch-linux-focal-py3`, thus it hasn't been run for a long time (forgotten). The name was probably the old name of the build environment we used in the past. The convention today doesn't have the `pytorch-` prefix. There is a TODO for this:
> TODO: this condition is never (BUILD_ENVIRONMENT doesn't start with pytorch-), need to fix this.
This is done as part of [T131829540](https://www.internalfb.com/intern/tasks/?t=131829540), where we want
`static_runtime_benchmark` build and test jobs to run in OSS CI to avoid breaking internal
* I also fix some compiler warning errors `-Werror=sign-compare`, `-Werror,-Wunused-const-variable`, and gcc7 compatibility issue along the way because this hasn't been run for a long time.
* Reviving this test also reveals a small bug in `PrepackWeights` test in `test_static_runtime.cc` added recently in https://github.com/pytorch/pytorch/pull/85289. The test refers to an internal ops and should only be run internally. This has been fixed by https://github.com/pytorch/pytorch/pull/87799 (To be merged)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/87660
Approved by: https://github.com/malfet
Summary:
The test is causing issues:
```
terminate called after throwing an instance of 'std::runtime_error'
what(): The following operation failed in the TorchScript interpreter.
Traceback of TorchScript (most recent call last):
graph(%A: Tensor, %driver: str?):
%bias: None = prim::Constant()
%ret = aten::linalg_svdvals(%A, %driver)
~~~~ <--- HERE
%cloned = aten::clone(%ret, %bias)
return (%cloned)
RuntimeError: torch.linalg.svd: keyword argument `driver=` is only supported on CUDA inputs with cuSOLVER backend.
```
Just block the op and re-run the codegen script to remove everything and update the generated ops.
Test Plan: Existing tests
Differential Revision: D39973860
Pull Request resolved: https://github.com/pytorch/pytorch/pull/85983
Approved by: https://github.com/xuzhao9, https://github.com/tenpercent
Summary: Added the third bool argument in inalg_solve test case to remove the runtime error.
Test Plan: buck run mode/opt caffe2/benchmarks/static_runtime:static_runtime_cpptest
Reviewed By: mikeiovine
Differential Revision: D37324419
Pull Request resolved: https://github.com/pytorch/pytorch/pull/79971
Approved by: https://github.com/tenpercent
Summary: The test is throwing a jit alias analysis not supporting error. Disabling it for now.
Test Plan: buck run mode/opt caffe2/benchmarks/static_runtime:static_runtime_cpptest
Reviewed By: mikeiovine
Differential Revision: D37056032
Pull Request resolved: https://github.com/pytorch/pytorch/pull/79574
Approved by: https://github.com/mikeiovine
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/76398
This diff adds the large files that include the newly generated ops from D34913736. Refer to the base diff for more details.
Test Plan: buck run mode/opt //caffe2/benchmarks/static_runtime:static_runtime_cpptest
Reviewed By: mikeiovine, tenpercent
Differential Revision: D35945633
fbshipit-source-id: 53497bd5c490a57ea1521837762f740deb42bfd8
(cherry picked from commit e0fbdcb0bf09f5c192430f95f450c0a946c80074)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/73574
T113070663 identified a test breakage in `StaticRuntime.autogen__convert_indices_from_csr_to_coo` using `mode/opt`. This change fixes it by using a correct test value.
By generating the out variants for Static Runtime, this change also includes an out variant wrapper for `mse_loss`.
**Generating out variants for Static Runtime**
```
[djang@devbig024.ftw2 ~/fbsource/fbcode/caffe2] buck run //caffe2/torch/fb/jit:gen_static_runtime_ops
Invalidating internal cached state: Buck configuration options changed between invocations. This may cause slower builds.
Changed value //fbcode.sanitizer='address-undefined-dev' (was 'thread')
... and 13 more. See logs for all changes
Parsing buck files: finished in 0.8 sec
Downloaded 14/25 artifacts, 159.45 Kbytes, 30.0% cache miss (for updated rules)
Building: finished in 1.6 sec (100%) 52/52 jobs, 35/52 updated
Total time: 2.5 sec
BUILD SUCCEEDED
total grouped native ops: 1501
structured grouped native ops: 540
generated grouped native ops: 137
```
Test Plan:
Ran the broken test in `mode/opt` and confirmed that it passes now.
```
[djang@devbig024.ftw2 ~/fbsource/fbcode/caffe2] buck test mode/opt //caffe2/benchmarks/static_runtime:static_runtime_cpptest -- --exact 'caffe2/benchmarks/static_runtime:static_runtime_cpptest - StaticRuntime.autogen__convert_indices_from_csr_to_coo' --run-disabled
Invalidating internal cached state: Buck configuration options changed between invocations. This may cause slower builds.
Changed value //project.buck_out='buck-out/opt' (was 'buck-out/dev')
... and 307 more. See logs for all changes
DEBUG: /data/users/djang/fbsource/tools/build_defs/fbcode_macros/build_defs/lib/cpp_common.bzl:287:14: Using disallowed linker flag 'arvr/third-party/toolchains/platform009/build/mesa/lib/libGL.so' in library rule 'fbsource//third-party/toolchains:opengl'
DEBUG: /data/users/djang/fbsource/tools/build_defs/fbcode_macros/build_defs/lib/cpp_common.bzl:287:14: Using disallowed linker flag 'arvr/third-party/freeglut/3.0.0/libs/x64-linux/libglut.a' in library rule 'fbsource//third-party/toolchains:GLUT'
I0301 08:28:08.884272 2239319 configeratorc.cpp:70] Attempting to get config buck/detectors/bypass_dirty_builds, timeout=10000
I0301 08:30:14.751745 2261718 configeratorc.cpp:70] Attempting to get config buck/detectors/bypass_dirty_builds, timeout=10000
Parsing buck files: finished in 10.1 sec
Creating action graph: finished in 6.1 sec
[RE] Metadata: Session ID=[https://fburl.com/b/reSessionID-fa0ba93b-33a1-4e6f-88f8-9f508d2c27c3]
[RE] Waiting on 0 remote actions. Completed 247 actions remotely, action cache hit rate: 0.00%.
Downloaded 13000/17457 artifacts, 463.99 Mbytes, 2.6% cache miss (for updated rules)
Building: finished in 04:16.6 min (100%) 28628/28628 jobs, 28628/28628 updated
Total time: 04:32.9 min
More details at https://www.internalfb.com/intern/buck/build/c774ff43-5311-49ce-a677-30e3f6afdad1
BUILD SUCCEEDED
Tpx test run coordinator for Facebook. See https://fburl.com/tpx for details.
Running with tpx session id: 16d9b24c-4a63-4671-84b5-690fac0ee086
Trace available for this run at /tmp/tpx-20220301-083049.472831-16d9b24c-4a63-4671-84b5-690fac0ee086/trace.log
RemoteExecution session id: reSessionID-16d9b24c-4a63-4671-84b5-690fac0ee086-tpx
Started reporting to test run: https://www.internalfb.com/intern/testinfra/testrun/4503599719295685
✓ ListingSuccess: caffe2/benchmarks/static_runtime:static_runtime_cpptest : 285 tests discovered (0.425)
✓ Pass: caffe2/benchmarks/static_runtime:static_runtime_cpptest - StaticRuntime.autogen__convert_indices_from_csr_to_coo (0.105)
Summary
Pass: 1
ListingSuccess: 1
If you need help understanding your runs, please follow the wiki: https://fburl.com/posting_in_tpx_users
Finished test run: https://www.internalfb.com/intern/testinfra/testrun/4503599719295685
```
Reviewed By: mikeiovine
Differential Revision: D34552645
fbshipit-source-id: 36f15b0f29edcb7deb71ba8a6f66ce2532bf7c82
(cherry picked from commit 2329afd8bfc89671cfbd864414e528241e7045fc)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/72603
This change adds out variant dispatchers generated by the previous diff.
The number of the out variant dispatchers generated by this diff is 133, which increases the out variant coverage by 309% (current: 43, this diff: 133 + 43 = 176). This number is expected to increase a lot as we develop this script further to cover more ops.
Test Plan:
**Unittest**
Confirmed
```
buck run //caffe2/benchmarks/static_runtime:static_runtime_cpptest
```
is passing.
Reviewed By: swolchok
Differential Revision: D33373928
fbshipit-source-id: 4d94d788282f3f313bb36f2f9452edecd9862246
(cherry picked from commit e4ce8b386d1fcc47b86cb9c9016a70e7a31b452c)
