to make them agnostic of ubuntu version, ROCm version and python minor version.
This should help avoid frequent updates to the docker image tags when upgrading ROCm version in PyTorch CI, which has creation of new ECR tags as a blocking step.
Reference: https://github.com/pytorch/pytorch/pull/88297#issuecomment-1307873280
The BUILD_ENVIRONMENT flag will continue to specify the exact versions for the above, in case it is needed for debug. @malfet @seemethere Hope that's not going away, otherwise we might have a harder time debugging issues where we need to figure out these environment details.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/90694
Approved by: https://github.com/malfet
Add bionic-py3.11-clang9, and move vulkan testing to it. Test only fx and jit for the time being (will add more in followup PRs)
Do not install numba, is it's not yet available for python-3.11
Change installed mkl version as the one installed before was incompatible with numpy
TODO: Remove `-c malfet` when required packages become available on default conda channel, namely `numpy`, `setuptools`, `coverage`, `mypy-exensions`, `typing-extensions`, `psutils` and `pyyaml`
Pull Request resolved: https://github.com/pytorch/pytorch/pull/92787
Approved by: https://github.com/albanD
Saw some places we missed some old requirements that are no longer necessary (dataclasses and future). Testing to see if all the CIs still work. We don't need dataclasses anymore now that we are on Python >= 3.7
Pull Request resolved: https://github.com/pytorch/pytorch/pull/92763
Approved by: https://github.com/ezyang
Set `cmake.dir` to `/usr/local` in `.circleci/scripts/build_android_gradle.sh `
Prep change for raising compiler standard to C++17: cmake-3.18 is the first one to support CUDA17 language
Pull Request resolved: https://github.com/pytorch/pytorch/pull/89570
Approved by: https://github.com/atalman
This readme was deleted here: https://github.com/pytorch/pytorch/pull/73224 I chatted with the author, who doesn't remember exactly why it was deleted but suspects it was due either to out of date contents or because of the upcoming migration to github actions.
With that said, we have references to this readme through our circleci directory, and since we do still have a lot of circleci workflows I feel this readme still adds a lot of value. (I recently did some CI tasks that required me to dig this readme up in order to solve a problem).
I recommend we restore this file with a warning that its contents may be out of date, until our CircleCI workflows are entirely migrated to Github Actions
Pull Request resolved: https://github.com/pytorch/pytorch/pull/85598
Approved by: https://github.com/clee2000, https://github.com/malfet
I always need to install these 2 tools whenever I use Docker manually to debug build and test issues:
* unzip is to extracted the zipped artifacts from PyTorch CI
* gdb is to do you know what :)
IMO, it makes sense to have them as part of the container image
Pull Request resolved: https://github.com/pytorch/pytorch/pull/86993
Approved by: https://github.com/ZainRizvi
currently failing with
```
To https://github.com/pytorch/cppdocs
+ 2825b2745bb...80ec4daa657 HEAD -> pytorchbot/temp-branch-cpp (forced update)
Branch 'master' set up to track remote branch 'pytorchbot/temp-branch-cpp' from 'origin'.
++ sleep 30
++ git push -u origin
fatal: The upstream branch of your current branch does not match
the name of your current branch. To push to the upstream branch
on the remote, use
git push origin HEAD:pytorchbot/temp-branch-cpp
To push to the branch of the same name on the remote, use
git push origin HEAD
```
just checked the settings, master of pytorch/cppdocs does not have easy cla as a required check, so we don't need the temp branch
Pull Request resolved: https://github.com/pytorch/pytorch/pull/87614
Approved by: https://github.com/huydhn
This is parts of the effort to consolidate pip and conda installation in the CI to improve our CI reliability. This moves conda cmake installation to Docker in those use cases that require it:
* Ubuntu bionic and focal
On the other hand:
* XLA doesn't seem to need conda cmake anymore (Build and test successfully)
* Centos is not in used anywhere in the CI
Pull Request resolved: https://github.com/pytorch/pytorch/pull/87309
Approved by: https://github.com/ZainRizvi, https://github.com/malfet
Mitigate https://github.com/pytorch/pytorch/issues/87148
### Testing
On AWS (m1, linux)
* Run `conda install blas:openblas`, it should failed with `ChecksumMismatchError`:
```
ChecksumMismatchError: Conda detected a mismatch between the expected content and downloaded content
for url 'https://repo.anaconda.com/pkgs/main/linux-64/blas-1.0-openblas.conda'.
download saved to: /tmp/debug/pkgs/blas-1.0-openblas.conda
expected sha256: c85b5d0a336b5be0f415c71fd7fe2eca59e09f42221bfa684aafef5510ba5487
actual sha256: 5dc5483db0d9785b19e021cee418a8ee03e0ff0e5ebd0b75af4927746604e187
```
* Run ` conda install -c conda-forge blas:openblas` works
Pull Request resolved: https://github.com/pytorch/pytorch/pull/87150
Approved by: https://github.com/kit1980
We deleted it when merging functorch into pytorch. This PR makes a new
functorch docs build.
The docs are relatively simple:
- cd into `functorch/docs` and run `make html` to build the docs.
- docs should get pushed to the pytorch/functorch repo's gh-pages
branch.
The long term plan is:
- one day, the functorch APIs will just be torch.* APIs, at which point
we can fold all of the functorch docs into the regular PyTorch docs
- When that happens, the functorch examples and tutorials (that are on
the functorch docs site) can be moved to the pytorch examples and
pytorch tutorials.
Test Plan:
- check docs preview
- watch this PR after it goes in
Differential Revision: [D40026222](https://our.internmc.facebook.com/intern/diff/D40026222)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/86125
Approved by: https://github.com/atalman, https://github.com/malfet
We deleted it when merging functorch into pytorch. This PR makes a new
functorch docs build.
The docs are relatively simple:
- cd into `functorch/docs` and run `make html` to build the docs.
- docs should get pushed to the pytorch/functorch repo's gh-pages
branch.
The long term plan is:
- one day, the functorch APIs will just be torch.* APIs, at which point
we can fold all of the functorch docs into the regular PyTorch docs
- When that happens, the functorch examples and tutorials (that are on
the functorch docs site) can be moved to the pytorch examples and
pytorch tutorials.
Test Plan:
- check docs preview
- watch this PR after it goes in
Pull Request resolved: https://github.com/pytorch/pytorch/pull/85838
Approved by: https://github.com/malfet
Depends on https://github.com/pytorch/pytorch/pull/84890.
This PR adds opt_einsum to CI, enabling path optimization for the multi-input case. It also updates the installation sites to install torch with einsum, but those are mostly to make sure it would work on the user's end (as opt-einsum would have already been installed in the docker or in prior set up steps).
This PR also updates the windows build_pytorch.bat script to use the same bdist_wheel and install commands as on Linux, replacing the `setup.py install` that'll become deprecated.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/85574
Approved by: https://github.com/huydhn, https://github.com/soulitzer
Bumps [protobuf](https://github.com/protocolbuffers/protobuf) from 3.20.1 to 3.20.2.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/protocolbuffers/protobuf/releases">protobuf's releases</a>.</em></p>
<blockquote>
<h2>Protocol Buffers v3.20.2</h2>
<h1>C++</h1>
<ul>
<li>Reduce memory consumption of MessageSet parsing</li>
<li>This release addresses a <a href="https://github.com/protocolbuffers/protobuf/security/advisories/GHSA-8gq9-2x98-w8hf">Security Advisory for C++ and Python users</a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="a20c65f2cd"><code>a20c65f</code></a> Updating changelog</li>
<li><a href="c49fe79af9"><code>c49fe79</code></a> Updating version.json and repo version numbers to: 20.2</li>
<li><a href="806d7e4ce6"><code>806d7e4</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/10544">#10544</a> from deannagarcia/3.20.x</li>
<li><a href="ae718b3902"><code>ae718b3</code></a> Add missing includes</li>
<li><a href="b4c395aaed"><code>b4c395a</code></a> Apply patch</li>
<li><a href="6439c5c013"><code>6439c5c</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/10531">#10531</a> from protocolbuffers/deannagarcia-patch-7</li>
<li><a href="22c79e6e4c"><code>22c79e6</code></a> Update version.json</li>
<li><a href="c1a2d2ec29"><code>c1a2d2e</code></a> Fix python release on macos (<a href="https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/10512">#10512</a>)</li>
<li><a href="a826282e15"><code>a826282</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/10505">#10505</a> from deannagarcia/3.20.x</li>
<li><a href="7639a710e1"><code>7639a71</code></a> Add version file</li>
<li>Additional commits viewable in <a href="https://github.com/protocolbuffers/protobuf/compare/v3.20.1...v3.20.2">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language
You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/pytorch/pytorch/network/alerts).
</details>
Pull Request resolved: https://github.com/pytorch/pytorch/pull/85572
Approved by: https://github.com/malfet
Adjusting retry times for conda upload.
Refer to this failure: https://github.com/pytorch/pytorch/actions/runs/3110932965/jobs/5043384691
```
Error: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))
+ sleep 1
......
Error: ('file osx-arm64/pytorch-1.13.0.dev20220923-py3.9_0.tar.bz2 already exists or being uploaded for package pytorch version 1.13.0.dev20220923. if your previous upload failed, please wait 2 minutes before trying again', 409)
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/85545
Approved by: https://github.com/datumbox
run tests in parallel at the test file granularity
runs 3 files in parallel using multiprocessing pool, output goes to a file, which is then printed when the test finishes. Some tests cannot be run in parallel (usually due to lacking memory), so we run those after. Sharding is changed to attempt to mask large files with other large files/run them on the same shard.
test_ops* gets a custom handler to run it because it is simply too big (2hrs on windows) and linalg_cholesky fails (I would really like a solution to this if possible, but until then we use the custom handler).
reduces cuda tests by a lot, reduces total windows test time by ~1hr
Ref. https://github.com/pytorch/pytorch/issues/82894
Pull Request resolved: https://github.com/pytorch/pytorch/pull/84961
Approved by: https://github.com/huydhn
- [x] Direct dependency on UCX is completely removed, UCC active set API always enabled
- [x] Remove `TORCH_UCC_PROFILING_ENABLE`, always enable profiling
- [x] Fixes profiling of `recv` and `all_gather`
- [x] Use the NCCL TL of UCC on CUDA, as the UCP TL is not well supported on CUDA
Most tests are passing, but there are a few skipped tests:
- `scatter` and `gather` are not supported by the UCP TL of UCC on CPU tensors
- A few flaky tests in PyTorch's CI environment
- Profiler-related failures, some of them will be fixed by @Fuzzkatt in https://github.com/pytorch/pytorch/pull/84368
After this PR is merged, I will continue to work on these skipped failures.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/83285
Approved by: https://github.com/vtlam, https://github.com/malfet, https://github.com/kwen2501
Recent update to conda 4.14.0 caused breakages in our docker builds:
754d7f05b6
This pins to prevent the errors:
```
Traceback (most recent call last):
2022-08-24T16:20:49.2412247Z File "/opt/conda/lib/python3.9/site-packages/conda/exceptions.py", line 1125, in __call__
2022-08-24T16:20:49.2413036Z File "/opt/conda/lib/python3.9/site-packages/conda/cli/main.py", line 86, in main_subshell
2022-08-24T16:20:49.2413615Z File "/opt/conda/lib/python3.9/site-packages/conda/cli/conda_argparse.py", line 93, in do_call
2022-08-24T16:20:49.2414282Z File "/opt/conda/lib/python3.9/site-packages/conda/notices/core.py", line 75, in wrapper
2022-08-24T16:20:49.2415036Z File "/opt/conda/lib/python3.9/site-packages/conda/notices/core.py", line 39, in display_notices
2022-08-24T16:20:49.2415853Z File "/opt/conda/lib/python3.9/site-packages/conda/notices/http.py", line 36, in get_notice_responses
2022-08-24T16:20:49.2416661Z File "/opt/conda/lib/python3.9/site-packages/conda/notices/http.py", line 39, in <genexpr>
2022-08-24T16:20:49.2417399Z File "/opt/conda/lib/python3.9/concurrent/futures/_base.py", line 609, in result_iterator
2022-08-24T16:20:49.2418145Z File "/opt/conda/lib/python3.9/concurrent/futures/_base.py", line 446, in result
2022-08-24T16:20:49.2418831Z File "/opt/conda/lib/python3.9/concurrent/futures/_base.py", line 391, in __get_result
2022-08-24T16:20:49.2419543Z File "/opt/conda/lib/python3.9/concurrent/futures/thread.py", line 58, in run
2022-08-24T16:20:49.2420292Z File "/opt/conda/lib/python3.9/site-packages/conda/notices/http.py", line 42, in <lambda>
2022-08-24T16:20:49.2421070Z File "/opt/conda/lib/python3.9/site-packages/conda/notices/cache.py", line 37, in wrapper
2022-08-24T16:20:49.2421712Z File "/opt/conda/lib/python3.9/site-packages/conda/notices/http.py", line 58, in get_channel_notice_response
2022-08-24T16:20:49.2422258Z File "/opt/conda/lib/python3.9/site-packages/requests/sessions.py", line 600, in get
2022-08-24T16:20:49.2422801Z File "/opt/conda/lib/python3.9/site-packages/requests/sessions.py", line 587, in request
2022-08-24T16:20:49.2423226Z File "/opt/conda/lib/python3.9/site-packages/requests/sessions.py", line 701, in send
2022-08-24T16:20:49.2423634Z File "/opt/conda/lib/python3.9/site-packages/requests/adapters.py", line 460, in send
2022-08-24T16:20:49.2424239Z File "/opt/conda/lib/python3.9/site-packages/requests/adapters.py", line 263, in cert_verify
2022-08-24T16:20:49.2424731Z OSError: Could not find a suitable TLS CA certificate bundle, invalid path: /opt/conda/lib/python3.9/site-packages/certifi/cacert.pem
2022-08-24T16:20:49.2424967Z
2022-08-24T16:20:49.2425110Z During handling of the above exception, another exception occurred:
2022-08-24T16:20:49.2425279Z
2022-08-24T16:20:49.2425377Z Traceback (most recent call last):
2022-08-24T16:20:49.2425610Z File "/opt/conda/bin/conda", line 13, in <module>
2022-08-24T16:20:49.2425845Z sys.exit(main())
2022-08-24T16:20:49.2426176Z File "/opt/conda/lib/python3.9/site-packages/conda/cli/main.py", line 129, in main
2022-08-24T16:20:49.2426614Z File "/opt/conda/lib/python3.9/site-packages/conda/exceptions.py", line 1413, in conda_exception_handler
2022-08-24T16:20:49.2427054Z File "/opt/conda/lib/python3.9/site-packages/conda/exceptions.py", line 1128, in __call__
2022-08-24T16:20:49.2427555Z File "/opt/conda/lib/python3.9/site-packages/conda/exceptions.py", line 1170, in handle_exception
2022-08-24T16:20:49.2427995Z File "/opt/conda/lib/python3.9/site-packages/conda/exceptions.py", line 1181, in handle_unexpected_exception
2022-08-24T16:20:49.2428471Z File "/opt/conda/lib/python3.9/site-packages/conda/exceptions.py", line 1251, in print_unexpected_error_report
2022-08-24T16:20:49.2428873Z ModuleNotFoundError: No module named 'conda.cli.main_info'
2022-08-24T16:20:55.5428691Z The command '/bin/sh -c bash ./install_conda.sh && rm install_conda.sh' returned a non-zero code: 1
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/83991
Approved by: https://github.com/malfet
This is a new version of #15648 based on the latest master branch.
Unlike the previous PR where I fixed a lot of the doctests in addition to integrating xdoctest, I'm going to reduce the scope here. I'm simply going to integrate xdoctest, and then I'm going to mark all of the failing tests as "SKIP". This will let xdoctest run on the dashboards, provide some value, and still let the dashboards pass. I'll leave fixing the doctests themselves to another PR.
In my initial commit, I do the bare minimum to get something running with failing dashboards. The few tests that I marked as skip are causing segfaults. Running xdoctest results in 293 failed, 201 passed tests. The next commits will be to disable those tests. (unfortunately I don't have a tool that will insert the `#xdoctest: +SKIP` directive over every failing test, so I'm going to do this mostly manually.)
Fixes https://github.com/pytorch/pytorch/issues/71105
@ezyang
Pull Request resolved: https://github.com/pytorch/pytorch/pull/82797
Approved by: https://github.com/ezyang
- Modifies the current cmake build definitions to use `find_package` to find UCX and UCC installed in the system
- Install UCX and UCC in CUDA dockers
- Build PyTorch with `USE_UCC=1` in pipelines
- Currently, we are not running unit tests with the UCC PG. Those tests will be added in future PRs.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/81583
Approved by: https://github.com/vtlam, https://github.com/malfet
This PR uses pytest to run test_ops, test_ops_gradients, and test_ops_jit in parallel in non linux cuda environments to decrease TTS. I am excluding linux cuda because running in parallel results in errors due to running out of memory
Notes:
* update hypothesis version for compatability with pytest
* use rerun-failures to rerun tests (similar to flaky tests, although these test files generally don't have flaky tests)
* reruns are denoted by a rerun tag in the xml. Failed reruns also have the failure tag. Successes (meaning that the test is flaky) do not have the failure tag.
* see https://docs.google.com/spreadsheets/d/1aO0Rbg3y3ch7ghipt63PG2KNEUppl9a5b18Hmv2CZ4E/edit#gid=602543594 for info on speedup (or slowdown in the case of slow tests)
* expecting windows tests to decrease by 60 minutes total
* slow test infra is expected to stay the same - verified by running pytest and unittest on the same job and check the number of skipped/run tests
* test reports to s3 changed - add entirely new table to keep track of invoking_file times
Pull Request resolved: https://github.com/pytorch/pytorch/pull/79898
Approved by: https://github.com/malfet, https://github.com/janeyx99
This test configuration runs PyTorch's test suite under torchdynamo.
Once stabilized, we will make this default and remove this particular
CI job.
Signed-off-by: Edward Z. Yang <ezyang@fb.com>
Pull Request resolved: https://github.com/pytorch/pytorch/pull/80342
Approved by: https://github.com/anijain2305
### Motivation
In order to match the internal platform010 builds, we are creating a new config to run on PRs that uses compiler and package versions >= used in platform010. Here are the versions used in the new build
- Ubuntu 22.04 (Jammy-Jellyfish)
- Clang-12
- Python 3.8
- CUDA 11.6
### Summary of Changes
- As `nvidia/docker` images only support Cuda 11.7 with Ubuntu 22.04, we are starting with base Ubuntu 22.04 docker images, and then installing Cuda 11.6
- Fetching `install_cuda.sh` from `pytorch/builder` repo in order to install Cuda using `wget`
- `libssl-dev` has been upgraded to libssl3 in Ubuntu 22.04. Instead, we are using `include` and `lib` folders downloaded with `Openssl1.1.1`
- `Clang-12` requires `libomp-12-dev` to work with `OpenMP` which is added to the `install_base.sh` file.
- Minor fixes to handle compilation errors generated when using `clang-12`.
- In `pow_test.cpp` adding a `static_cast` to input of `sqrt` method
- In `vec512_qint.h`, explicitly defining `copy-assignment` operator as its implicit definition is deprecated due to
user-declared `copy-constructor` in C++11
Pull Request resolved: https://github.com/pytorch/pytorch/pull/79945
Approved by: https://github.com/seemethere, https://github.com/kit1980
### Motivation
In order to match the internal platform010 builds, we are creating a new config to run on PRs that uses compiler and package versions >= used in platform010. Here are the versions used in the new build
- Ubuntu 22.04 (Jammy-Jellyfish)
- Clang-12
- Python 3.8
- CUDA 11.6
### Summary of Changes
- As `nvidia/docker` images only support Cuda 11.7 with Ubuntu 22.04, we are starting with base Ubuntu 22.04 docker images, and then installing Cuda 11.6
- Fetching `install_cuda.sh` from `pytorch/builder` repo in order to install Cuda using `wget`
- `libssl-dev` has been upgraded to libssl3 in Ubuntu 22.04. Instead, we are using `include` and `lib` folders downloaded with `Openssl1.1.1`
- `Clang-12` requires `libomp-12-dev` to work with `OpenMP` which is added to the `install_base.sh` file.
- Minor fixes to handle compilation errors generated when using `clang-12`.
- In `pow_test.cpp` adding a `static_cast` to input of `sqrt` method
- In `vec512_qint.h`, explicitly defining `copy-assignment` operator as its implicit definition is deprecated due to
user-declared `copy-constructor` in C++11
Pull Request resolved: https://github.com/pytorch/pytorch/pull/77591
Approved by: https://github.com/seemethere, https://github.com/atalman
Adds a failsafe for when docker images aren't available at your base at
the expense of TTS. Also adds the ability to set force_push and note
about what to do when TTS gets out of hand because docker images are
always re-building.
Signed-off-by: Eli Uriegas <eliuriegasfb.com>
Pull Request resolved: https://github.com/pytorch/pytorch/pull/78186
Signed-off-by: Eli Uriegas <eliuriegas@fb.com>
Approved by: https://github.com/janeyx99
crossref is a new strategy for performing tests when you want
to run a normal PyTorch API call, separately run some variation of
the API call (e.g., same thing but all the arguments are meta tensors)
and then cross-reference the results to see that they are consistent.
Any logic you add to CrossRefMode will get run on *every* PyTorch API
call that is called in the course of PyTorch's test suite. This can
be a good choice for correctness testing if OpInfo testing is not
exhaustive enough.
For now, the crossref test doesn't do anything except verify that
we can validly push a mode onto the torch function mode stack for all
functions.
Signed-off-by: Edward Z. Yang <ezyangfb.com>
Pull Request resolved: https://github.com/pytorch/pytorch/pull/75988
Approved by: https://github.com/seemethere
Summary:
Adding CUDA 11.6 workflows .
Please note we still depend on conda-forge for cuda 11.6.
Issue created to remove conda-forge dependency: [75532](https://github.com/pytorch/pytorch/issues/75532)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/75518
Reviewed By: janeyx99
Differential Revision: D35516057
Pulled By: atalman
fbshipit-source-id: 44a3a0f8954d98adca2280b2e9f203267ebe98cd
(cherry picked from commit 97a4e52ecee8540453e2871714275796dc1c4abb)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/74563
This is used inconsistently in all the generate_code program
invocations. Nevertheless, nothing consumes this flag, so we can
safely remove it.
This was removed in #25353.
ghstack-source-id: 152249818
Test Plan: Should be a no-op, rely on CI.
Reviewed By: malfet
Differential Revision: D35053096
fbshipit-source-id: 3ad19e83ca14649b514dc163c3caff6cbd118e14
(cherry picked from commit a43f05bb43553249caac3c3479986cbc45d286ae)
Use nvidia cuda image without cudnn for cudnn 8 and up.
We want to decouple the CUDA and cudnn versions so that we can evolve these version separately.
We want to use cudnn 8.3.2 for following CUDA versions 11.3, 11.5 and 11.6.
We are using Official Nvidia Cuda ubuntu image. And installing cudnn 8.3.2 on top of it.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/74545
Approved by: https://github.com/malfet
Summary:
This PR moves the Dockerfile conda dependencies into a requirements-ci.txt (and begins the requirements file for other parts of CI as well). Packages are listed alphabetically in the requirements-ci.txt. Uncommented packages before the mkl package have been tested and confirmed to work on all platforms. Commented out packages before mkl have broken at least one platform and so have been comment out. There appears to be some randomness with certain platforms not passing tests so it might be good to run a number of tests for the same configuration to confirm if it is indeed these commented out packages that cause the errors.
Remaining is to test all commented out packages to ensure they work on all platforms. This will likely involve repeat runs of the same configurations to ensure it is indeed the packages that break the platforms and not random errors.
This PR makes progress on task https://github.com/pytorch/pytorch/issues/72556
Pull Request resolved: https://github.com/pytorch/pytorch/pull/73404
Reviewed By: janeyx99
Differential Revision: D34730797
Pulled By: kstant0725
fbshipit-source-id: 3e4b171720fa33b604cebb9c6101d38ba11f2f8b
(cherry picked from commit 99cc445aadb95f92f6ef040f2d4b7c6c6d5b7f8b)
Summary:
Fixes : https://github.com/pytorch/pytorch/issues/73377
We've migrated to CUDA-11.3 as default toolkit in 1.9, it's time to stop builds (especially considering forward-compatibility guarantee across CUDA-11.x drivers)
Hence we are removing CUDA 11.1 support. We should also cleanup old cuda related code from our builder and pytorch repo making scripts a little more clean.
We have code that references cuda 9.2 , 10.1 , 11.0, 11.1, 11.2 and none of these are currently use
Pull Request resolved: https://github.com/pytorch/pytorch/pull/73514
Reviewed By: janeyx99
Differential Revision: D34551989
Pulled By: atalman
fbshipit-source-id: 9ceaaa9b25ad49689986f4b29a26d20370d9d011
(cherry picked from commit fe109c62daf429e9053c03f6e374568ba23cd041)
Summary:
- Target Sha1: ae108ef49aa5623b896fc93d4298c49d1750d9ba
- Make USE_XNNPACK a dependent option on cmake minimum version 3.12
- Print USE_XNNPACK under cmake options summary, and print the
availability from collet_env.py
- Skip XNNPACK based tests when XNNPACK is not available
- Add SkipIfNoXNNPACK wrapper to skip tests
- Update cmake version for xenial-py3.7-gcc5.4 image to 3.12.4
- This is required for the backwards compatibility test.
The PyTorch op schema is XNNPACK dependent. See,
aten/src/ATen/native/xnnpack/RegisterOpContextClass.cpp for
example. The nightly version is assumed to have USE_XNNPACK=ON,
so with this change we ensure that the test build can also
have XNNPACK.
- HACK: skipping test_xnnpack_integration tests on ROCM
Pull Request resolved: https://github.com/pytorch/pytorch/pull/72642
Reviewed By: kimishpatel
Differential Revision: D34456794
Pulled By: digantdesai
fbshipit-source-id: 85dbfe0211de7846d8a84321b14fdb061cd6c037
(cherry picked from commit 6cf48e7b64d6979962d701b5d493998262cc8bfa)
These builds weren't advertised on our websites and we've confirmed that
CUDA 11.3 binaries are forwards compatible with CUDA 11.1 drivers.
Signed-off-by: Eli Uriegas <eliuriegasfb.com>
Pull Request resolved: https://github.com/pytorch/pytorch/pull/73376
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/72834
This removes the upper bound to librosa's pin and updates the scipy
pin since librosa 0.9 requires SciPy 1.2 or newer.
Test Plan: Imported from OSS
Reviewed By: ngimel
Differential Revision: D34386898
Pulled By: mruberry
fbshipit-source-id: db654bd337b474cd5a2ff8dbb9a659ed272728cf
(cherry picked from commit 4790e8180c)
Summary:
Remove fx2trt test from oss CI
Pull Request resolved: https://github.com/pytorch/pytorch/pull/72595
Test Plan: CI
Reviewed By: houseroad
Differential Revision: D34112595
Pulled By: wushirong
fbshipit-source-id: 02376ef0f25381eff31b72dcbf964c1966af9793
(cherry picked from commit e3d698a942)
Summary:
Should fix the following [error](https://github.com/pytorch/pytorch/runs/5058514346#step:13:88):
```
+ git --git-dir /pytorch/pytorch/.git describe --tags --match 'v[0-9]*.[0-9]*.[0-9]*' --exact
fatal: not a git repository: '/pytorch/pytorch/.git'
```
By setting `workdir` correctly for GHA linux and Windows builds
Also, abort `tagged_version` if GIT_DIR does not exist (as this script should only be executed in context of git folder.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/72628
Reviewed By: atalman
Differential Revision: D34120721
Pulled By: malfet
fbshipit-source-id: 035e93e243e601f9c24659cd247f9c029210fba5
(cherry picked from commit 3a6c97b6dd)
Summary:
brianjo, malfet
The documentation team would prefer the [documentation versions] to only have a major.minor version, not major.minor.patch. See also pytorch/pytorch.github.io#921
The regex can be tested by this bash 1-liner (where $tag is something like `v10.1225.0rc1`)
```
echo $tag | sed -e 's/v*\([0-9]*\.[0-9]*\).*/\1/'
```
I have lost track a bit, is the CI run for a tag actually building and pushing documentation?
Pull Request resolved: https://github.com/pytorch/pytorch/pull/71985
Reviewed By: mrshenli
Differential Revision: D33845882
Pulled By: malfet
fbshipit-source-id: 3cb644d8b01f5ddf87c0ac7c43e23e9fd292d660
(cherry picked from commit f884bd8674)
Summary:
Small clean-up, realized this file isn't necessary after migrating to GHA, so removing this file
Pull Request resolved: https://github.com/pytorch/pytorch/pull/71943
Test Plan: running .circleci/regenerate.sh yields no config changes
Reviewed By: malfet
Differential Revision: D33901182
Pulled By: janeyx99
fbshipit-source-id: e8ff16395c81be25dae5b84619c6b4bfe749ada2
(cherry picked from commit e564c1ed5e)
