OSSForks/pytorch
1
0
Fork 0
mirror of https://github.com/zebrajr/pytorch.git synced 2025-12-07 12:21:27 +01:00
Code Issues Actions 38 Packages Projects Releases Wiki Activity
95,310 Commits 8,304 Branches 150 Tags 1.5 GiB
master
Commit Graph

23 Commits

Author SHA1 Message Date
PyTorch MergeBot
639b8cc51d Revert "cd: Add no-cache for test binaries (#149218)" 
This reverts commit 523bffd388.

Reverted https://github.com/pytorch/pytorch/pull/149218 on behalf of https://github.com/atalman due to Lets not use no-cache flags on test binaries ([comment](https://github.com/pytorch/pytorch/pull/149218#issuecomment-3214338844))
 2025-08-22 13:14:23 +00:00
Nikita Shulga
d0226719a9 [BE][EZ] Delete remains of split-build logic (#159990) 
Hopefully last piece of https://github.com/pytorch/pytorch/issues/138750

Pull Request resolved: https://github.com/pytorch/pytorch/pull/159990
Approved by: https://github.com/atalman
ghstack dependencies: #159986
 2025-08-07 01:59:30 +00:00
Eli Uriegas
523bffd388 cd: Add no-cache for test binaries (#149218) 
This is to make it so that we don't experience issues like https://github.com/pytorch/vision/actions/runs/13861462856/job/38795684317#step:13:212

```
ERROR: THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered with them.
    unknown package:
        Expected sha256 8e34a6f02ac5a63763251953063a19ba9df855ac2c8a13ef409dfef708e2ba26
             Got        341156cc5067488565c1e103be6e95105b0fc0d87d8ac24ff8891f63fd33216f
```

Pull Request resolved: https://github.com/pytorch/pytorch/pull/149218
Approved by: https://github.com/ZainRizvi, https://github.com/atalman, https://github.com/malfet
 2025-03-17 23:26:20 +00:00
Catherine Lee
67ed47d886 Binary upload checksum (#144887) 
Equivalent to https://github.com/pytorch/test-infra/pull/6172 but for pytorch
Pull Request resolved: https://github.com/pytorch/pytorch/pull/144887
Approved by: https://github.com/atalman
 2025-01-31 17:51:27 +00:00
PyTorch MergeBot
768ad0886f Revert "Binary upload checksum (#144887)" 
This reverts commit 2efa98d69d.

Reverted https://github.com/pytorch/pytorch/pull/144887 on behalf of https://github.com/atalman due to Broke nightly index ([comment](https://github.com/pytorch/pytorch/pull/144887#issuecomment-2610066277))
 2025-01-23 15:10:42 +00:00
Catherine Lee
2efa98d69d Binary upload checksum (#144887) 
Equivalent to https://github.com/pytorch/test-infra/pull/6172 but for pytorch
Pull Request resolved: https://github.com/pytorch/pytorch/pull/144887
Approved by: https://github.com/atalman
 2025-01-22 20:46:04 +00:00
atalman
519269a415 [BE] - Remove conda test and upload scripts and env variables from Workflows Part 1 (#144870) 
Remove conda test and upload scripts and env variables from Workflows

Related to: https://github.com/pytorch/pytorch/issues/138506
Pull Request resolved: https://github.com/pytorch/pytorch/pull/144870
Approved by: https://github.com/malfet
 2025-01-16 17:20:14 +00:00
Catherine Lee
0dca756832 Revert "Upload METADATA file with whl binaries (#143677)" (#144706) 
This reverts commit 3eb3f4ed55.

Also reverts https://github.com/pytorch/pytorch/pull/144164

Manual revert because the above causes merge conflicts

Reverting in favor of https://github.com/pytorch/test-infra/pull/6159
Pull Request resolved: https://github.com/pytorch/pytorch/pull/144706
Approved by: https://github.com/janeyx99, https://github.com/atalman, https://github.com/malfet
 2025-01-15 17:20:21 +00:00
Catherine Lee
3eb3f4ed55 Upload METADATA file with whl binaries (#143677) 
Upload the metadata file for wheels for pep658 https://peps.python.org/pep-0658/
Using a python script but using bash might be easier...

--

Testing

Example run https://github.com/pytorch/pytorch/actions/runs/12550595201/job/34994883276 without actual upload, just dry run

Lightly tested the script to make sure it uploads to s3, but integration with the bash script + workflow is untested

Pull Request resolved: https://github.com/pytorch/pytorch/pull/143677
Approved by: https://github.com/seemethere
 2025-01-03 00:32:05 +00:00
atalman
a1590e16df Add single Python 3.10, single Cuda 12.1 build with dependencies included (#130349) 
Build large wheel for Python 3.10, CUDA 12.1 that will be used in Colab. Build name: ``manywheel-py3_11-cuda12_1-full-build``

We still have all code to support the full build in builder repo, here:
https://github.com/pytorch/builder/blob/main/manywheel/build_cuda.sh#L151

Test:
```
import sys
import torch
sys.version_info
print(torch.__version__)
sys.version_info

2.3.0+cu121
sys.version_info(major=3, minor=10, micro=12, releaselevel='final', serial=0)
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/130349
Approved by: https://github.com/malfet
 2024-07-10 12:57:39 +00:00
Sahan Paliskara
f06e3a1569 [Split Build] Make script not crash if split build is not set (#129774) 
Fixes issue causing https://github.com/pytorch/pytorch/actions/runs/9704484834/job/26801889463 to crash
Pull Request resolved: https://github.com/pytorch/pytorch/pull/129774
Approved by: https://github.com/atalman
 2024-06-28 23:50:18 +00:00
PaliC
b0044e2e18 [Split Build] Support nightly release (#129011) 
This PR adds the split build to our binaries workflow. Validation for the workflow is done using the PR above in conjunction with https://github.com/pytorch/builder/pull/1876.

Test Workflow: Check CI in the workflow above
Pull Request resolved: https://github.com/pytorch/pytorch/pull/129011
Approved by: https://github.com/atalman
 2024-06-22 05:45:14 +00:00
atalman
995fae6060 Move small pypi build as default for linux cuda 12.1 (#114281) 
This is first PR to resolve: https://github.com/pytorch/pytorch/issues/113972
Move our small wheel build as default
Test:
```
pip3 install --no-cache-dir --pre torch-2.2.0.dev20231121%2Bcu121-cp310-cp310-linux_x86_64.whl  --index-url https://download.pytorch.org/whl/nightly/cu121
Looking in indexes: https://download.pytorch.org/whl/nightly/cu121
Processing ./torch-2.2.0.dev20231121%2Bcu121-cp310-cp310-linux_x86_64.whl
Collecting filelock (from torch==2.2.0.dev20231121+cu121)
  Downloading https://download.pytorch.org/whl/nightly/filelock-3.9.0-py3-none-any.whl (9.7 kB)
Collecting typing-extensions>=4.8.0 (from torch==2.2.0.dev20231121+cu121)
  Downloading https://download.pytorch.org/whl/nightly/typing_extensions-4.8.0-py3-none-any.whl (31 kB)
Collecting sympy (from torch==2.2.0.dev20231121+cu121)
  Downloading https://download.pytorch.org/whl/nightly/sympy-1.11.1-py3-none-any.whl (6.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.5/6.5 MB 253.4 MB/s eta 0:00:00
Collecting networkx (from torch==2.2.0.dev20231121+cu121)
  Downloading https://download.pytorch.org/whl/nightly/networkx-3.0rc1-py3-none-any.whl (2.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.0/2.0 MB 387.1 MB/s eta 0:00:00
Collecting jinja2 (from torch==2.2.0.dev20231121+cu121)
  Downloading https://download.pytorch.org/whl/nightly/Jinja2-3.1.2-py3-none-any.whl (133 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 133.1/133.1 kB 365.3 MB/s eta 0:00:00
Collecting fsspec (from torch==2.2.0.dev20231121+cu121)
  Downloading https://download.pytorch.org/whl/nightly/fsspec-2023.4.0-py3-none-any.whl (153 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 154.0/154.0 kB 370.6 MB/s eta 0:00:00
Collecting pytorch-triton==2.1.0+6e4932cda8 (from torch==2.2.0.dev20231121+cu121)
  Downloading https://download.pytorch.org/whl/nightly/pytorch_triton-2.1.0%2B6e4932cda8-cp310-cp310-linux_x86_64.whl (125.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 125.4/125.4 MB 384.1 MB/s eta 0:00:00
Collecting nvidia-cuda-nvrtc-cu12==12.1.105 (from torch==2.2.0.dev20231121+cu121)
  Downloading https://download.pytorch.org/whl/nightly/cu121/nvidia_cuda_nvrtc_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (23.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 23.7/23.7 MB 404.9 MB/s eta 0:00:00
Collecting nvidia-cuda-runtime-cu12==12.1.105 (from torch==2.2.0.dev20231121+cu121)
  Downloading https://download.pytorch.org/whl/nightly/cu121/nvidia_cuda_runtime_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (823 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 823.6/823.6 kB 402.5 MB/s eta 0:00:00
Collecting nvidia-cuda-cupti-cu12==12.1.105 (from torch==2.2.0.dev20231121+cu121)
  Downloading https://download.pytorch.org/whl/nightly/cu121/nvidia_cuda_cupti_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (14.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 14.1/14.1 MB 383.9 MB/s eta 0:00:00
Collecting nvidia-cudnn-cu12==8.9.2.26 (from torch==2.2.0.dev20231121+cu121)
  Downloading https://download.pytorch.org/whl/nightly/cu121/nvidia_cudnn_cu12-8.9.2.26-py3-none-manylinux1_x86_64.whl (731.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 731.7/731.7 MB 406.9 MB/s eta 0:00:00
Collecting nvidia-cublas-cu12==12.1.3.1 (from torch==2.2.0.dev20231121+cu121)
  Downloading https://download.pytorch.org/whl/nightly/cu121/nvidia_cublas_cu12-12.1.3.1-py3-none-manylinux1_x86_64.whl (410.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 410.6/410.6 MB 388.2 MB/s eta 0:00:00
Collecting nvidia-cufft-cu12==11.0.2.54 (from torch==2.2.0.dev20231121+cu121)
  Downloading https://download.pytorch.org/whl/nightly/cu121/nvidia_cufft_cu12-11.0.2.54-py3-none-manylinux1_x86_64.whl (121.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 121.6/121.6 MB 410.5 MB/s eta 0:00:00
Collecting nvidia-curand-cu12==10.3.2.106 (from torch==2.2.0.dev20231121+cu121)
  Downloading https://download.pytorch.org/whl/nightly/cu121/nvidia_curand_cu12-10.3.2.106-py3-none-manylinux1_x86_64.whl (56.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 56.5/56.5 MB 272.9 MB/s eta 0:00:00
Collecting nvidia-cusolver-cu12==11.4.5.107 (from torch==2.2.0.dev20231121+cu121)
  Downloading https://download.pytorch.org/whl/nightly/cu121/nvidia_cusolver_cu12-11.4.5.107-py3-none-manylinux1_x86_64.whl (124.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 124.2/124.2 MB 381.5 MB/s eta 0:00:00
Collecting nvidia-cusparse-cu12==12.1.0.106 (from torch==2.2.0.dev20231121+cu121)
  Downloading https://download.pytorch.org/whl/nightly/cu121/nvidia_cusparse_cu12-12.1.0.106-py3-none-manylinux1_x86_64.whl (196.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 196.0/196.0 MB 394.6 MB/s eta 0:00:00
Collecting nvidia-nccl-cu12==2.19.3 (from torch==2.2.0.dev20231121+cu121)
  Downloading https://download.pytorch.org/whl/nightly/cu121/nvidia_nccl_cu12-2.19.3-py3-none-manylinux1_x86_64.whl (166.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 166.0/166.0 MB 384.7 MB/s eta 0:00:00
Collecting nvidia-nvtx-cu12==12.1.105 (from torch==2.2.0.dev20231121+cu121)
  Downloading https://download.pytorch.org/whl/nightly/cu121/nvidia_nvtx_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (99 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 99.1/99.1 kB 281.8 MB/s eta 0:00:00
Collecting nvidia-nvjitlink-cu12 (from nvidia-cusolver-cu12==11.4.5.107->torch==2.2.0.dev20231121+cu121)
  Downloading https://download.pytorch.org/whl/nightly/cu121/nvidia_nvjitlink_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (19.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 19.8/19.8 MB 367.3 MB/s eta 0:00:00
Collecting MarkupSafe>=2.0 (from jinja2->torch==2.2.0.dev20231121+cu121)
  Downloading https://download.pytorch.org/whl/nightly/MarkupSafe-2.1.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
Collecting mpmath>=0.19 (from sympy->torch==2.2.0.dev20231121+cu121)
  Downloading https://download.pytorch.org/whl/nightly/mpmath-1.2.1-py3-none-any.whl (532 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 532.6/532.6 kB 391.3 MB/s eta 0:00:00
Installing collected packages: mpmath, typing-extensions, sympy, nvidia-nvtx-cu12, nvidia-nvjitlink-cu12, nvidia-nccl-cu12, nvidia-curand-cu12, nvidia-cufft-cu12, nvidia-cuda-runtime-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-cupti-cu12, nvidia-cublas-cu12, networkx, MarkupSafe, fsspec, filelock, pytorch-triton, nvidia-cusparse-cu12, nvidia-cudnn-cu12, jinja2, nvidia-cusolver-cu12, torch
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/114281
Approved by: https://github.com/malfet, https://github.com/huydhn
 2023-11-22 00:10:03 +00:00
Huy Do
7be233f3a5 Remove commit hash when building triton wheel and conda in release mode (#108203) 
This is the follow-up of https://github.com/pytorch/pytorch/pull/108187 to set the correct release version without commit hash for triton wheel and conda binaries when building them in release mode.

### Testing

* With commit hash (nightly): https://github.com/pytorch/pytorch/actions/runs/6019021716
* Without commit hash https://github.com/pytorch/pytorch/actions/runs/6019378616 (by adding `--release` into the PR)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/108203
Approved by: https://github.com/atalman
 2023-08-30 16:49:21 +00:00
Huy Do
cd07214a41 Fix various issues on build-triton-wheel workflow (#108187) 
There are more issues that I expect at the beginning:

* Triton was uploaded on `main` instead of `nightly` and release branch
* The environment `conda-aws-upload` wasn't used correctly in both wheel and conda upload
* Conda update wasn't run in a separate ephemeral runner
* Duplicated upload logic, should have just use `bash .circleci/scripts/binary_upload.sh` instead
* Handle `CONDA_PYTORCHBOT_TOKEN` and `CONDA_PYTORCHBOT_TOKEN_TEST` tokens in a similar way as https://github.com/pytorch/test-infra/pull/4530

Part of https://github.com/pytorch/pytorch/issues/108154
Pull Request resolved: https://github.com/pytorch/pytorch/pull/108187
Approved by: https://github.com/atalman
 2023-08-30 00:02:32 +00:00
atalman
1418a663b1 Fix upload condition pypi-cudnn build (#85799) 
Fix upload condition pypi-cudnn build
We excute this in sh and looks like the condition with "==" is not getting triggered.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/85799
Approved by: https://github.com/DanilBaibak, https://github.com/jeanschmidt, https://github.com/seemethere
 2022-09-28 22:27:52 +00:00
atalman
879ae45230 Increase timeout and retry count conda upload (#85802) 
Increase timeout and retry count conda upload. We are keep seeing conda upload failures even with 2 min timeout.
Hence increasing timeout to 5min and retry to 5 times

Pull Request resolved: https://github.com/pytorch/pytorch/pull/85802
Approved by: https://github.com/datumbox
 2022-09-28 20:34:13 +00:00
atalman
52a8be523c Adjust retry time for conda upload (#85545) 
Adjusting retry times for conda upload.
Refer to this failure: https://github.com/pytorch/pytorch/actions/runs/3110932965/jobs/5043384691

```
Error:  ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))
+ sleep 1
......
Error:  ('file osx-arm64/pytorch-1.13.0.dev20220923-py3.9_0.tar.bz2 already exists or being uploaded for package pytorch version 1.13.0.dev20220923. if your previous upload failed, please wait 2 minutes before trying again', 409)
```

Pull Request resolved: https://github.com/pytorch/pytorch/pull/85545
Approved by: https://github.com/datumbox
 2022-09-23 20:28:36 +00:00
atalman
e342976907 Adding conda retry upload to mitigate connection reset errors (#85407) 
Adding conda retry upload to mitigate connection reset errors

Mitigate errors like this:
https://github.com/pytorch/pytorch/actions/runs/3095808905/jobs/5012840560

```
Uploading file "pytorch-nightly/pytorch/1.13.0.dev20220921/linux-64/pytorch-1.13.0.dev20220921-py3.9_cuda11.6_cudnn8.3.2_0.tar.bz2"

  0%|          | 0.00/1.24G [00:00<?, ?B/s]
100%|██████████| 1.24G/1.24G [00:00<00:00, 2.08GB/s]
100%|██████████| 1.24G/1.24G [00:04<00:00, 271MB/s]
Error:  ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))
Error: Process completed with exit code 1.
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/85407
Approved by: https://github.com/weiwangmeta, https://github.com/malfet, https://github.com/seemethere
 2022-09-22 17:33:59 +00:00
Eli Uriegas
87484d67e3 .github: Enable linux binary builds (#68388) 
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/68388

Updates the gpu architectures as well as adding a trigger for
on_pull_request for the binary build workflows so that we can iterate on
this later

TODO:
* Create follow up PR to enable nightly linux GHA builds / disable CircleCI nighlty linux builds

Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

Test Plan: Imported from OSS

Reviewed By: janeyx99

Differential Revision: D33462294

Pulled By: seemethere

fbshipit-source-id: 5fa30517550d36f504b491cf6c1e5c9da56d8191
 2022-01-10 11:30:45 -08:00
Eli Uriegas
b23e9cdd64 .circleci: Add slash to end of s3 cp (#43792) 
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/43792

This fixes the issue we had with the nightlies not being uploaded
properly, basically what was happening was that `aws s3 cp` doesn't
automatically distinguish between prefixes that are already
"directories" vs a single file with the same name.

This means that if you'd like to upload a file to a "directory" in S3
you need to suffix your destination with a slash.

Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

Test Plan: Imported from OSS

Reviewed By: malfet

Differential Revision: D23402074

Pulled By: seemethere

fbshipit-source-id: 6085595283fcbbbab0836ccdfe0f8aa2a6abd7c8
 2020-08-28 13:37:25 -07:00
Eli Uriegas
66a79bf114 .circleci: Don't quote glob for conda upload (#43297) 
Summary:
Globs don't get expanded if you quote them in a bash script...
apparently.

Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

Pull Request resolved: https://github.com/pytorch/pytorch/pull/43297

Reviewed By: malfet

Differential Revision: D23227626

Pulled By: seemethere

fbshipit-source-id: d124025cfcaacbfb68167a062ca487c08f7f6bc9
 2020-08-20 10:24:27 -07:00
Eli Uriegas
eb7fc2e98f .circleci: Simplify binary upload process (#43159) 
Summary:
Binary uploads were gated into 3 separate scripts making it difficult to
actually contribute changes, this simplifies that by consolidating all 3
scripts into one single script and then further consolidates it by
putting them all into the same job.

This also further simplifies things by separating upload jobs into their
own function under binary_build_definitions.py, since following the
conditional logic tree under the generic function was too difficult.

Testing this change here: https://github.com/pytorch/pytorch/pull/43161

Proof of success:
* [libtorch](https://app.circleci.com/pipelines/github/pytorch/pytorch/201868/workflows/54ce962f-f35b-4d97-93a7-bee186b14ead/jobs/6791347)
* [conda](https://app.circleci.com/pipelines/github/pytorch/pytorch/201868/workflows/54ce962f-f35b-4d97-93a7-bee186b14ead/jobs/6794359)
* [manywheel](https://app.circleci.com/pipelines/github/pytorch/pytorch/201868/workflows/54ce962f-f35b-4d97-93a7-bee186b14ead/jobs/6794253)

Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

Pull Request resolved: https://github.com/pytorch/pytorch/pull/43159

Reviewed By: malfet

Differential Revision: D23175174

Pulled By: seemethere

fbshipit-source-id: a2de64c033df99b03a124d3a0a2c92560af62c37
 2020-08-19 11:34:14 -07:00