Commit Graph

3862 Commits

Author SHA1 Message Date
Anthony Shoumikhin
7d39e73c57 Fix more URLs (#153277)
Or ignore them.
Found by running the lint_urls.sh script locally with https://github.com/pytorch/pytorch/pull/153246

Pull Request resolved: https://github.com/pytorch/pytorch/pull/153277
Approved by: https://github.com/malfet
2025-05-14 16:23:50 +00:00
PyTorch UpdateBot
0ce941f994 [audio hash update] update the pinned audio hash (#153507)
This PR is auto-generated nightly by [this action](https://github.com/pytorch/pytorch/blob/main/.github/workflows/nightly.yml).
Update the pinned audio hash.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/153507
Approved by: https://github.com/pytorchbot
2025-05-14 15:16:35 +00:00
Horace He
cd119ddd7c Add matching against hypothetical (new) ghstack pull-request trailer (#153528)
I would like to change ghstack to use a new trailer
Pull Request resolved: https://github.com/pytorch/pytorch/pull/153528
Approved by: https://github.com/malfet
2025-05-14 14:07:01 +00:00
Ting Lu
7f79222992 Upgrade to NCCL 2.26.5 for CUDA 12 (#152810)
Upgrade NCCL to latest 2.26.5

Pull Request resolved: https://github.com/pytorch/pytorch/pull/152810
Approved by: https://github.com/eqy, https://github.com/albanD, https://github.com/nWEIdia, https://github.com/atalman, https://github.com/cyyever
2025-05-14 00:52:50 +00:00
Aaron Gokaslan
3555ebb63d [BE]: Update ruff to 0.11.8 (#153249)
Fixes a ton of false negatives throughout the codebase. RUFF also properly validates NOQA comments now and most of the changes are fixing typos there or removing filewide flake8 suppressions that were also silencing ruff issues.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/153249
Approved by: https://github.com/cyyever, https://github.com/albanD, https://github.com/seemethere
2025-05-12 18:30:52 +00:00
PyTorch UpdateBot
7e1790d86b [xla hash update] update the pinned xla hash (#153368)
This PR is auto-generated nightly by [this action](https://github.com/pytorch/pytorch/blob/main/.github/workflows/nightly.yml).
Update the pinned xla hash.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/153368
Approved by: https://github.com/pytorchbot
2025-05-12 17:11:23 +00:00
Thanh Ha
de5c5f4fb7 Opt-out LF runners from of inductor jobs (#153151)
Opt-out of inductor jobs for the lf experiment configuration.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/153151
Approved by: https://github.com/seemethere
2025-05-12 15:52:53 +00:00
PyTorch UpdateBot
909ec495b8 [audio hash update] update the pinned audio hash (#153301)
This PR is auto-generated nightly by [this action](https://github.com/pytorch/pytorch/blob/main/.github/workflows/nightly.yml).
Update the pinned audio hash.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/153301
Approved by: https://github.com/pytorchbot
2025-05-11 03:47:56 +00:00
Xu Han
e4f22822cb Revert "Cleanup VS 2019 refs in pytorch (#145863)" (#152613)
This reverts commit b45e6fa707.

revert PRs:
https://github.com/pytorch/pytorch/pull/145863
https://github.com/pytorch/pytorch/pull/145319

Pull Request resolved: https://github.com/pytorch/pytorch/pull/152613
Approved by: https://github.com/atalman, https://github.com/malfet
2025-05-10 19:33:26 +00:00
Nikita Shulga
4f068598c4 [BE] Delete now unused mac-mps.yml (#153263)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/153263
Approved by: https://github.com/Skylion007, https://github.com/cyyever
ghstack dependencies: #153013, #153057, #152719
2025-05-10 19:10:41 +00:00
Huy Do
fbb6412fdb Stop uploading sccache stats to benchmark database (#153285)
This is not used for anything atm and potentially bloat up the size of the database
Pull Request resolved: https://github.com/pytorch/pytorch/pull/153285
Approved by: https://github.com/clee2000, https://github.com/malfet
2025-05-10 00:17:38 +00:00
Nikita Shulga
e73a4c3643 [BE][CI] Merge regular and MPS test config shards (#152719)
Unsure why there were separate to beging with
Pull Request resolved: https://github.com/pytorch/pytorch/pull/152719
Approved by: https://github.com/seemethere, https://github.com/atalman
ghstack dependencies: #153013, #153057
2025-05-09 17:01:35 +00:00
Nikita Shulga
309ecb2277 [CI] Add opt-in h100 tests (#153170)
So far only run:
 - inductor/test_fp8.py
 - test_matmul_cuda.py
 - inductor/test_max_autotune.py
Pull Request resolved: https://github.com/pytorch/pytorch/pull/153170
Approved by: https://github.com/drisspg, https://github.com/eellison
2025-05-09 17:01:05 +00:00
Thanh Ha
50657120a0 Allow workflows to opt-out of experiments (#153085)
This change adds support to allow workflows to opt-out of experiments.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/153085
Approved by: https://github.com/ZainRizvi

Co-authored-by: Zain Rizvi <ZainRizvi@users.noreply.github.com>
2025-05-09 16:34:46 +00:00
PyTorch MergeBot
34196301d5 Revert "[CI] Add opt-in h100 tests (#153170)"
This reverts commit f87a0fe2ca.

Reverted https://github.com/pytorch/pytorch/pull/153170 on behalf of https://github.com/clee2000 due to workflow doesnt have right concurrency group? ([comment](https://github.com/pytorch/pytorch/pull/153170#issuecomment-2864951319))
2025-05-09 03:04:50 +00:00
Nikita Shulga
f87a0fe2ca [CI] Add opt-in h100 tests (#153170)
So far only run:
 - inductor/test_fp8.py
 - test_matmul_cuda.py
 - inductor/test_max_autotune.py
Pull Request resolved: https://github.com/pytorch/pytorch/pull/153170
Approved by: https://github.com/drisspg
2025-05-09 01:03:12 +00:00
Nikita Shulga
30a3c5d970 Skip lintchecks for now (#153156)
As devs has been complaining it's failing. Completely remove them from lint.yml as https://github.com/pytorch/pytorch/pull/153157 moved it to nightly

See https://github.com/pytorch/pytorch/issues/152439  as well as https://github.com/pytorch/pytorch/issues/152884 and https://github.com/pytorch/pytorch/issues/152489 for more details

Was introduced in https://github.com/pytorch/pytorch/pull/152377
Pull Request resolved: https://github.com/pytorch/pytorch/pull/153156
Approved by: https://github.com/albanD, https://github.com/ZainRizvi
2025-05-08 17:58:05 +00:00
Jithun Nair
fe8ebacee4 [ROCm] Upgrade ROCm CI to ROCm6.4 (#151368)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/151368
Approved by: https://github.com/jeffdaily, https://github.com/malfet

Co-authored-by: Jeff Daily <jeff.daily@amd.com>
2025-05-08 16:12:16 +00:00
Wang, Chuanqi
1d3e8f326a [CI] Increase shards number for XPU ci UT tests (#149113)
The XPU CI test met timeout issue, refer https://github.com/pytorch/pytorch/actions/runs/14897047392/job/41842336828 and this PR will reduce the ci time cost
Pull Request resolved: https://github.com/pytorch/pytorch/pull/149113
Approved by: https://github.com/etaf, https://github.com/EikanWang
2025-05-08 15:42:33 +00:00
Anthony Shoumikhin
8141b146ca Run URL linter on nightly only (#153157)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/153157
Approved by: https://github.com/malfet
2025-05-08 15:32:42 +00:00
Anthony Shoumikhin
1dd36ad2d4 Fix conditional git diff in _link_check.yml (#152919)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/152919
Approved by: https://github.com/huydhn
2025-05-06 07:01:45 +00:00
PyTorch UpdateBot
dd766e1dc5 [audio hash update] update the pinned audio hash (#152885)
This PR is auto-generated nightly by [this action](https://github.com/pytorch/pytorch/blob/main/.github/workflows/nightly.yml).
Update the pinned audio hash.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/152885
Approved by: https://github.com/pytorchbot
2025-05-06 05:29:25 +00:00
Anthony Shoumikhin
337895eaaf Run url and xref linters independently (#152899)
Also introduce `skip-xref-lint` label

Pull Request resolved: https://github.com/pytorch/pytorch/pull/152899
Approved by: https://github.com/huydhn
2025-05-06 05:02:32 +00:00
Anthony Shoumikhin
1798b0db25 Use three-dot diffs in URL and xref lint workflows (#152895)
Only run on the files actually modified in a PR, not every file touched on main since the branch point

Fixes #152884

Pull Request resolved: https://github.com/pytorch/pytorch/pull/152895
Approved by: https://github.com/huydhn
2025-05-06 01:33:52 +00:00
Catherine Lee
0145f9e29e [CI] docker images use tags instead of image name (#152209)
Change CI docker images to be `ci-image:<image name>-<folder sha>` instead of `<image name>:<folder sha>` so we never have to make a new ecr repo ever again

Pros:
never have to make a new ecr repo ever again
Cons:
if it aint broken, dont fix it?

Don't need to change linux-test images since they use the "full name" of the image with the docker registry and the tag

In order to prevent others needing to rebase past this PR, also push the image to the "old name".  This can be removed after this PR has been in main for a while
Pull Request resolved: https://github.com/pytorch/pytorch/pull/152209
Approved by: https://github.com/seemethere, https://github.com/atalman
2025-05-05 18:02:29 +00:00
PyTorch UpdateBot
9210a98b92 [xla hash update] update the pinned xla hash (#152809)
This PR is auto-generated nightly by [this action](https://github.com/pytorch/pytorch/blob/main/.github/workflows/nightly.yml).
Update the pinned xla hash.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/152809
Approved by: https://github.com/pytorchbot
2025-05-05 11:21:11 +00:00
PyTorch MergeBot
cc28b43950 Revert "[ROCm] Upgrade ROCm CI to ROCm6.4 (#151368)"
This reverts commit 844842dfbf.

Reverted https://github.com/pytorch/pytorch/pull/151368 on behalf of https://github.com/malfet due to This broke inductor cpp wrapper ([comment](https://github.com/pytorch/pytorch/pull/151368#issuecomment-2848519706))
2025-05-03 08:31:31 +00:00
Sam Larsen
36140e01fd Rename "startup-tracing-compile" to "compile-time" in label_to_label.yml (#152711)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/152711
Approved by: https://github.com/oulgen
2025-05-03 00:35:05 +00:00
Anthony Shoumikhin
54f29b04d6 Improve error wording in _link_check.yml (#152726)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/152726
Approved by: https://github.com/huydhn
2025-05-02 22:43:05 +00:00
Yang Wang
44f29a3669 Add parameters for monitor (#152541)
Add log interval and log-data-collect interval to all test yml

Add upload step for all test yml files

next step:
enable perf test with utilization

Pull Request resolved: https://github.com/pytorch/pytorch/pull/152541
Approved by: https://github.com/huydhn
2025-05-02 20:24:11 +00:00
Jithun Nair
844842dfbf [ROCm] Upgrade ROCm CI to ROCm6.4 (#151368)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/151368
Approved by: https://github.com/jeffdaily

Co-authored-by: Jeff Daily <jeff.daily@amd.com>
2025-05-02 17:21:18 +00:00
Nikita Shulga
3849fd13de 🐛 Add ciflow/pull🦋 (#152567)
To make it easier to workaround GitHub relibability issues, when it sometime fails to scheduled `on: pull_request` workflows

See https://github.com/pytorch/pytorch/issues/151322

But alas, it does not fixes problem at hand...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/152567
Approved by: https://github.com/clee2000, https://github.com/huydhn, https://github.com/ZainRizvi, https://github.com/Camyll, https://github.com/atalman
2025-05-01 02:00:51 +00:00
Jeff Daily
3c2bf24786 [ROCm] add almalinux images (#152492)
Fixes #ISSUE_NUMBER

Pull Request resolved: https://github.com/pytorch/pytorch/pull/152492
Approved by: https://github.com/atalman
2025-04-30 15:14:01 +00:00
Anthony Shoumikhin
accffef504 Run link checks on modified files on push too (#152464)
https://github.com/pytorch/pytorch/issues/152439
Pull Request resolved: https://github.com/pytorch/pytorch/pull/152464
Approved by: https://github.com/huydhn
2025-04-29 22:08:40 +00:00
atalman
28efeb1522 Remove unused Manylinux2014 Docker files and builds (#152428)
Related to Manylinux 2.28 migration: https://github.com/pytorch/pytorch/issues/123649
Cleanup old Docker files and `manylinuxaarch64-builder:cpu-aarch64` image which has been replaced by `manylinux2_28_aarch64-builder:cpu-aarch64`
Pull Request resolved: https://github.com/pytorch/pytorch/pull/152428
Approved by: https://github.com/Skylion007, https://github.com/malfet
2025-04-29 20:57:29 +00:00
Catherine Lee
a04f4622e1 [conda] Remove conda from lint-autoformat.yml (#152433)
Installs setuptools since I get
https://github.com/pytorch/pytorch/actions/runs/14736804186/job/41364832984#step:5:60
```
+ python3 -m tools.generate_torch_version --is_debug=false
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/ec2-user/actions-runner/_work/pytorch/pytorch/tools/generate_torch_version.py", line 9, in <module>
    from setuptools import distutils  # type: ignore[import]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'setuptools'
```
It should be a no op in the normal lint workflow since setuptools is in the docker image

Switched from using python3.10 to system python, which should be python3.9

Use venv to put deps not in the base?

Pull Request resolved: https://github.com/pytorch/pytorch/pull/152433
Approved by: https://github.com/huydhn
2025-04-29 20:14:21 +00:00
Anthony Shoumikhin
a3123dd3ab Run link linters on modified files only or on everything when scheduled (#152377)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/152377
Approved by: https://github.com/huydhn
2025-04-29 18:30:40 +00:00
Catherine Lee
de20d76622 [conda] Remove conda usage from upload test stats while running workflow (#152431)
The original uses python 3.10 and the base is 3.9 but I think that's ok
Pull Request resolved: https://github.com/pytorch/pytorch/pull/152431
Approved by: https://github.com/atalman
2025-04-29 16:16:54 +00:00
Catherine Lee
f84062f78d [conda] Remove conda usage from TD llm retriever job (#152338)
Remove conda usage from TD llm retriever job

python3 in the base is python3.9 right now.  I'm not sure what the best way to deal with a potentially different python version would be, dnf install?
Pull Request resolved: https://github.com/pytorch/pytorch/pull/152338
Approved by: https://github.com/huydhn
2025-04-29 15:17:50 +00:00
atalman
22f179d77d Use almalinux docker files for building Magma (#152358)
Resolves https://github.com/pytorch/pytorch/issues/151707 for CUDA Nvidia Magma builds.
Removes deprecated cuda 12.4 build.

Using `pytorch/manylinux2_28-builder` image for magma build creates circular dependency.

For a while for magma builds we used `conda-builder` image since it does not have circular dependency:
https://github.com/pytorch/builder/blob/release/2.4/magma/Makefile#L13
However during migration to pytorch/pytorch: https://github.com/pytorch/pytorch/pull/139888 we introduced circular dependency using Manylinux 2.28 docker image.

Hence using almalinux image which suppose to be general usage image

Please note: Magma builds using Docker build : https://github.com/pytorch/pytorch/blob/main/.ci/magma/README.md we can look into migrating them to Docker images if required as a followup BE change if needed

TODO: Make same change for rocm builds. I believe some more work for rocm is required, since maga-rocm is requires rocm dev, utils and lib to be installed : https://github.com/pytorch/pytorch/blob/main/.ci/docker/common/install_rocm.sh

Pull Request resolved: https://github.com/pytorch/pytorch/pull/152358
Approved by: https://github.com/nWEIdia, https://github.com/huydhn
2025-04-29 00:45:01 +00:00
Andrey Talman
86b0271b00 Add CUDA 12.8 almalinux image, remove CUDA 12.4 almalinux (#152362)
This is general purpose image located in: https://hub.docker.com/r/pytorch/almalinux-builder
Updating it to match our supported CUDA matrix

Adding this build to use as general purpose image and use for Magma build
Pull Request resolved: https://github.com/pytorch/pytorch/pull/152362
Approved by: https://github.com/malfet
2025-04-28 21:15:05 +00:00
Anthony Shoumikhin
fcebaedebc Add a label to skip URL lint if needed (#152340)
Some URLs may be down due to server side issues we can't control
Pull Request resolved: https://github.com/pytorch/pytorch/pull/152340
Approved by: https://github.com/huydhn, https://github.com/malfet
2025-04-28 20:29:40 +00:00
Thanh Ha
79e8dc7d53 Pin to SHA for actions outside of PyTorch (#152110)
Pin actions from repos external to the PyTorch project to their shasums for security. This is a best practice as Git tags are not immutable.

https://openssf.org/blog/2024/08/12/mitigating-attack-vectors-in-github-workflows/

Pull Request resolved: https://github.com/pytorch/pytorch/pull/152110
Approved by: https://github.com/seemethere, https://github.com/ZainRizvi
2025-04-28 17:57:32 +00:00
atalman
c869862875 Remove cuda dependencies from non cuda buids (#152333)
These dependancies added to fix poetry issue on pypi. However inclusion of these dependencies creates issue with poetry on download.pytorch.org due to poetry reading first available wheel on index for METADATA requirements. Hence all metadata requirements for CPU wheels can't list any cuda dependencies.

Injecting these dependencies via prep for pypi will need to be done via:
https://github.com/pytorch/test-infra/blob/main/release/pypi/prep_binary_for_pypi.sh

Ref: https://github.com/pytorch/pytorch/issues/152121

Pull Request resolved: https://github.com/pytorch/pytorch/pull/152333
Approved by: https://github.com/jeanschmidt, https://github.com/malfet
2025-04-28 16:46:44 +00:00
Nikita Shulga
6e5e9dc321 [benchmarking] Inc aarch64 bench shards to 15 (#152324)
As it frequently timing out with 12, but also it feels like shards are somewhat unbalanced
I.e. if one to look at https://github.com/pytorch/pytorch/actions/runs/14696840776/job/41239776679
Shard 12 takes 3.6 hours, while shard 11 is only 40 min
Pull Request resolved: https://github.com/pytorch/pytorch/pull/152324
Approved by: https://github.com/janeyx99, https://github.com/atalman
2025-04-28 16:08:39 +00:00
PyTorch UpdateBot
0b6ea0b959 [xla hash update] update the pinned xla hash (#151210)
This PR is auto-generated nightly by [this action](https://github.com/pytorch/pytorch/blob/main/.github/workflows/nightly.yml).
Update the pinned xla hash.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/151210
Approved by: https://github.com/pytorchbot
2025-04-28 11:45:09 +00:00
Anthony Shoumikhin
7cae7902a2 Add scripts to check xrefs and urls (#151844)
Traverses the docs and code to find any broken links
Pull Request resolved: https://github.com/pytorch/pytorch/pull/151844
Approved by: https://github.com/huydhn
2025-04-28 09:30:07 +00:00
Anthony Shoumikhin
e2f9759bd0 Fix broken URLs (#152237)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/152237
Approved by: https://github.com/huydhn, https://github.com/malfet
2025-04-27 09:56:42 +00:00
PyTorch UpdateBot
dda0c952e7 [audio hash update] update the pinned audio hash (#152149)
This PR is auto-generated nightly by [this action](https://github.com/pytorch/pytorch/blob/main/.github/workflows/nightly.yml).
Update the pinned audio hash.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/152149
Approved by: https://github.com/pytorchbot
2025-04-25 04:20:06 +00:00
PyTorch MergeBot
8313bc27f2 Revert "Add OIDC permissions to bazel workflow (#151456)"
This reverts commit 5fc1eb85fc.

Reverted https://github.com/pytorch/pytorch/pull/151456 on behalf of https://github.com/seemethere due to This is causing downstream failures on PRs, see examples in PR comment ([comment](https://github.com/pytorch/pytorch/pull/151456#issuecomment-2829130319))
2025-04-25 00:37:15 +00:00