Commit Graph

852 Commits

Author SHA1 Message Date
Eli Uriegas
48ea440b15 ci: Unblock syncbranches, add a58c6ae and 7106d21 to block list (#76417)
Summary:
Adds a58c6aea5a0c9f8759a4154e46f544c8b03b8db1 and 7106d216c29ca16a3504aa2bedad948ebcf4abc2 to the list of excluded
commits since this was landed through phab and cherry picked to master
directly

Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

Pull Request resolved: https://github.com/pytorch/pytorch/pull/76417

Reviewed By: janeyx99

Differential Revision: D35951416

Pulled By: seemethere

fbshipit-source-id: 30a226c381e0cebfccc82f7ccfa7ce79075220c9
(cherry picked from commit b75fbe3b9e8024734e749a42464620c1879265ad)
2022-04-26 23:54:22 +00:00
Yeounoh Chung
3c327f3f01 Reenable XLA workflow/test
XLA test was disabled [PR](https://github.com/pytorch/pytorch/pull/76237) due to failure. We have reverted the PR, to re-enable the test.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/76339
Approved by: https://github.com/seemethere
2022-04-26 23:24:51 +00:00
Michael Suo
20543221f4 [lint] url encode lint message
This is so that things like newlines will be rendered by github
correctly. Sadly problem matchers don't work with multiline messages
(see https://github.com/actions/toolkit/issues/319), so this just uses
workflow syntax commands
(https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#setting-a-warning-message)
directly to show annotations.

This has the extra bonus that we can make the title nicer as well.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/76351

Approved by: https://github.com/janeyx99
2022-04-26 15:17:06 +00:00
Michael Suo
1e1118957f [lint] correctly display annotations for all severities
GitHub's problem matchers only match against "warning" or "error" as
severities. So translate the "advice" and "disabled" severities into
warnings so they'll show up.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/76348

Approved by: https://github.com/seemethere
2022-04-25 23:20:43 +00:00
Thiago Crepaldi
90d31cb311 Emit ATen ops when symbolics raise + minor fixes
Currently `torch.onnx.export(.., operator_export_type=OperatorExportTypes.ONNX_ATEN_FALLBACK)` only issues ATen ops through explicit requests (e.g. `g.at()`) calls inside each op symbolic function. This is done based on specific conditions such as `operator_export_type==OperatorExportTypes.ONNX_ATEN_FALLBACK)` or `is_caffe2_aten_fallback()`

This PR extends the ATen fallback mechanism for scenarios when the symbolic function raises `RuntimeError` during export. The idea is that partial implementation of existing ONNX ops can fallback to ATen as a last resort. That is valuable because each operator can have many input combinations and not all are always implemented.

A minor fix was done to make sure the `overload_name` attribute is added to explicit ATen op fallback requests when a symbolic is not registered to a particular op.

ps: The behavior for builds with BUILD_CAFFE2=1 is not changed to ensure BC.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/74759
Approved by: https://github.com/garymm, https://github.com/msaroufim
2022-04-23 21:24:25 +00:00
atalman
31a6e6cabc Remove cuda 11.5 builds since we have 11.6
CUDA 11.5 was experimental, but now that we are moving to 11.6, we don't need to keep 11.5 around.
Please note following PR's where merged for domain libraries
[vision](https://github.com/pytorch/vision/pull/5862)
[audio](https://github.com/pytorch/audio/pull/2346)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/76257
Approved by: https://github.com/seemethere, https://github.com/malfet
2022-04-23 00:24:58 +00:00
Jeff Daily
44bbb247a6 [ROCm] enable fsdp tests
Pull Request resolved: https://github.com/pytorch/pytorch/pull/75632
Approved by: https://github.com/kumpera, https://github.com/malfet
2022-04-22 19:50:36 +00:00
Jeff Daily
e846ef8818 add rocm ciflow/slow workflow
Enables additional tests that historically have been missed for ROCm CI.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/72686
Approved by: https://github.com/seemethere
2022-04-22 17:41:28 +00:00
zengk95
b167897317 disable xla test job
This disables XLA test for now cause it's consistently failing on HUD and we're waiting on XLA to update.

https://hud.pytorch.org/minihud
Pull Request resolved: https://github.com/pytorch/pytorch/pull/76237
Approved by: https://github.com/seemethere
2022-04-22 15:56:28 +00:00
Nikita Shulga
25d5b63acf [GHF] Skip b3aa2de (#76231)
Summary:
Difference between were b5222584e6 and 69e048b090 are reconciled in b3aa2de5be, so the commit must be manually skipped

Pull Request resolved: https://github.com/pytorch/pytorch/pull/76231

Reviewed By: bigfootjon

Differential Revision: D35845975

Pulled By: malfet

fbshipit-source-id: 4e4a2f03a26202bffe2045ac80704b356144164e
(cherry picked from commit dd32c3e33059b28c4727ffbeb40661dd14b3c7dc)
2022-04-22 14:19:24 +00:00
Nikita Shulga
b447fa3912 [GHF] Manual fix syncbranches (#76200)
Summary:
By excluding 6d0f4a1d545a8f161df459e8d4ccafd4b9017dbe and edf909e58f06150f7be41da2f98a3b9de3167bca (which we re-relanded thrice)

Pull Request resolved: https://github.com/pytorch/pytorch/pull/76200

Reviewed By: george-qi

Differential Revision: D35828342

Pulled By: malfet

fbshipit-source-id: 27581a3cf00b9eadfff8ccc05b59243439aabb4d
(cherry picked from commit 5c6a502727c022816c682edda7b9cb4c10ed900b)
2022-04-21 17:27:05 -07:00
Sergii Dymchenko
e5282c3cb8 Again add first version of Buck build workflow
Next attempt after https://github.com/pytorch/pytorch/pull/75815 (that PR was reverted for breaking internal build because of BUCK.github file name doesn't get synced internal->github).

Pull Request resolved: https://github.com/pytorch/pytorch/pull/76156
Approved by: https://github.com/seemethere
2022-04-21 16:26:10 +00:00
Nikita Shulga
547ac879f4 [GHF] Add pagination to commits_with_authors
Which enables one to fetch authorship of PRs with 100+ commits

Add unittest based on https://github.com/pytorch/pytorch/pull/76118

Pull Request resolved: https://github.com/pytorch/pytorch/pull/76137
Approved by: https://github.com/seemethere, https://github.com/atalman
2022-04-21 16:25:28 +00:00
David Berard
cebdca4191 Add more nvfuser merge_rules.json
Pull Request resolved: https://github.com/pytorch/pytorch/pull/76053

Approved by: https://github.com/osalpekar
2022-04-20 22:25:39 +00:00
Nikita Shulga
f31d518283 [GHF] Improve failures debugability
Print exception backtrace in the run log
Print GraphQL arguments as well as query itself when it fails
Print more concise message when PR is not reviewed

Make error reports like https://github.com/pytorch/pytorch/pull/75851#issuecomment-1102868693 much easier to debug

Pull Request resolved: https://github.com/pytorch/pytorch/pull/76121
Approved by: https://github.com/janeyx99
2022-04-20 19:24:18 +00:00
Jane Xu
ae864d4fb9 Remove 11.5 periodic
Relanding #75749 without the binary changes
Pull Request resolved: https://github.com/pytorch/pytorch/pull/76114
Approved by: https://github.com/malfet
2022-04-20 17:23:36 +00:00
Catherine Lee
5477f0ae60 back to fetch depth 0
Fixes #ISSUE_NUMBER
undo #75783 b/c setting fetch depth 1 doesn't really help reduce time b/c most of the jobs need either master or viable/strict

also, more branches need viable/strict than i thought, so sharding isn't picking up test times (although default sharding seems to do pretty well) (regarding the jobs i didn't realize needed viable/strict: it looks like the linux-bionic jobs don't fail when `git rev-parse viable/strict` is run but viable/strict doesn't exist but the linux-xenial ones do)

pretty sure jobs are broken only b/c its using the master version of `checkout-pytorch/action.yml`
tested via #76077
Pull Request resolved: https://github.com/pytorch/pytorch/pull/76090
Approved by: https://github.com/seemethere
2022-04-20 15:55:12 +00:00
Edward Z. Yang
ee955b8bb9 Cannibalize noarch CI job into crossref CI job
crossref is a new strategy for performing tests when you want
to run a normal PyTorch API call, separately run some variation of
the API call (e.g., same thing but all the arguments are meta tensors)
and then cross-reference the results to see that they are consistent.
Any logic you add to CrossRefMode will get run on *every* PyTorch API
call that is called in the course of PyTorch's test suite.  This can
be a good choice for correctness testing if OpInfo testing is not
exhaustive enough.

For now, the crossref test doesn't do anything except verify that
we can validly push a mode onto the torch function mode stack for all
functions.

Signed-off-by: Edward Z. Yang <ezyangfb.com>

Pull Request resolved: https://github.com/pytorch/pytorch/pull/75988

Approved by: https://github.com/seemethere
2022-04-20 11:56:25 +00:00
Jane Xu
839109f689 [GH1] Add sparse related changes to merge rules
This PR would allow Quansight sparse experts (in addition to metamates) to approve sparse related changes. As the sparse module is relatively new and should not have many internal dependencies, we can start encouraging more GitHub 1st (GH1) landing for these.

This is DIFFERENT from the superuser rule because it allows non-metamates to be approvers.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/75872
Approved by: https://github.com/IvanYashchuk, https://github.com/osalpekar
2022-04-19 22:03:17 +00:00
Jane Xu
b1a369b423 [GH1] Add FFT related changes to merge rules
This PR would allow Quansight FFT experts (in addition to metamates) to approve sparse related changes. As the fft module is not really internally used, we can start encouraging more GitHub 1st (GH1) landing for these.

This is DIFFERENT from the superuser rule because it allows non-metamates to be approvers.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/75874
Approved by: https://github.com/osalpekar
2022-04-19 21:54:11 +00:00
Jon Janzen
0dc860dbd6 [pytorch][require export] Skip internal checks in Meta service (#75837)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/75837

We run this as an internal service to check if a PR can be merged. We don't care about internal checks because these diffs are landing internally.

bypass-github-export-checks

Reviewed By: seemethere, osalpekar

Differential Revision: D35657708

fbshipit-source-id: f52cf28a424839532b5be4cce0f7010a6816e179
(cherry picked from commit f7a8f8c4f979e77b3ce6c659e49fc213860b3351)
2022-04-19 19:37:51 +00:00
Jane Xu
a5cb0d6be4 [GH1] Add linalg related changes to merge rules
This PR would allow Quansight linear algebra experts (in addition to metamates) to approve sparse related changes. Linear algebra would be a great place to start encouraging more GitHub 1st (GH1) landing to test our external contributor GH1 experience.

This is DIFFERENT from the superuser rule because it allows non-metamates to be approvers.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/75881
Approved by: https://github.com/osalpekar
2022-04-19 17:50:41 +00:00
Nikita Shulga
af13797c8f [GHF] Exclude 5f37e5c2a39c3acb776756a17730b865f0953432 from sync
Corresponding trunk commit generate different patch-id, as history
diverged between `master` and `fbsync`

Fixes #ISSUE_NUMBER

Pull Request resolved: https://github.com/pytorch/pytorch/pull/76048
Approved by: https://github.com/seemethere
2022-04-19 16:35:57 +00:00
Thiago Crepaldi
eab3f42883 Update symbolics policy to emit aten::ATen for Caffe2 build only
Currently ONNX exporter symbolics can emit ATen operators when `operator_export_type==ONNX_ATEN_FALLBACK`. However, this is a behavior specific to Caffe2 builds, as the intend use of `ONNX_ATEN_FALLBACK` is to emit ATen operators only when there is no ONNX equivalent.

The reason Caffe2 choses to emit ATen operators when ONNX counterpart exists is for performance on their particular engine implementation, which might not be true for other implementations. e.g. ONNX Runtime can optimize the generated ONNX graph into something more efficient

This PR must be merged only after https://github.com/pytorch/pytorch/pull/73954
Pull Request resolved: https://github.com/pytorch/pytorch/pull/74680
Approved by: https://github.com/garymm, https://github.com/malfet
2022-04-19 15:57:54 +00:00
Jane Xu
86ea57805b Add back SHARD_NUMBER and TEST_CONFIG to upload test stats step
These are used in the print_test_stats file and somehow are no longer passed.

Test plan:
After CI runs, check the S3 to make sure the file names include shard number and config.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/75989
Approved by: https://github.com/clee2000
2022-04-19 00:20:47 +00:00
Jane Xu
6b72357b14 Modify GraphQL PR info query to adjust for workflow consolidation
We used to have a ton of workflow runs each with few jobs, but now we are switching it up to fewer workflow runs with many jobs each.

Thus edit the query so we can get the maximum checks for a PR, which is a preliminary thing for when we want to add mroe required status checks :)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/75820
Approved by: https://github.com/seemethere, https://github.com/osalpekar
2022-04-19 00:20:15 +00:00
Catherine Lee
cdbd39ba57 fix fetch-depth: 1
Fixes #ISSUE_NUMBER

Follow up to https://github.com/pytorch/pytorch/pull/75783

Inputs are always strings, and the string wasn't empty, so the if statement always evaluated to true, so it always checked out the extra branches. Add check that the input is literally 'true' in order to checkout the extra branches

tested via #75955
Pull Request resolved: https://github.com/pytorch/pytorch/pull/75957
Approved by: https://github.com/seemethere
2022-04-18 20:56:13 +00:00
Eli Uriegas
e70fea8e76 ci: Add credentials to upload test stats for rocm
These are the same credentials we use to upload test stats for macOS so
ideally these should "just work"

Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

Pull Request resolved: https://github.com/pytorch/pytorch/pull/75979
Approved by: https://github.com/janeyx99
2022-04-18 20:14:23 +00:00
Nikita Shulga
c4cf51de99 Revert D35679120: Add first version of Buck build workflow
Test Plan: revert-hammer

Differential Revision:
D35679120 (c132b9fd71)

Original commit changeset: 5544227c2734

Original Phabricator Diff: D35679120 (c132b9fd71)

fbshipit-source-id: c47eff7e1b5124dd7d82a6d927d1030fba22d5da
(cherry picked from commit cd52bda782d1b22a2a27a3221757f5d3a6c4a2ad)
2022-04-18 16:22:08 +00:00
Nikita Shulga
b582472ac3 [GHF] Fix sync-branches
Manually skip 8e09e20c1dafcdbdb45c2d1574da68a32e54a3a5 which introduces
unstable patch-ids

Pull Request resolved: https://github.com/pytorch/pytorch/pull/75971
Approved by: https://github.com/janeyx99
2022-04-18 16:09:36 +00:00
Sergii Dymchenko (Meta Employee)
c132b9fd71 Add first version of Buck build workflow (#75815)
Summary:
Next attempt after https://github.com/pytorch/pytorch/pull/75360 (that PR was reverted for breaking internal build because of BUCK file names conflicts).

Pull Request resolved: https://github.com/pytorch/pytorch/pull/75815
Approved by: https://github.com/linbinyu

Test Plan: contbuild & OSS CI, see de66304aa5

Reviewed By: mehtanirav

Differential Revision: D35679120

Pulled By: kit1980

fbshipit-source-id: 5544227c273418571d5c34b5f601fb9006463684
(cherry picked from commit 35b0b483d3c21d0e0ad49ddafac2f9e271af944c)
2022-04-17 18:51:23 +00:00
PyTorch MergeBot
b5a25180f1 Revert "Add first version of Buck build workflow"
This reverts commit de66304aa5.

Reverted https://github.com/pytorch/pytorch/pull/75815 on behalf of https://github.com/malfet
2022-04-17 18:48:44 +00:00
Eli Uriegas
18b9d6b20a ci: Change with-ssh to be on by default
Changes the behavior of with-ssh to no longer require a label in order to activate

Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

Pull Request resolved: https://github.com/pytorch/pytorch/pull/75922
Approved by: https://github.com/osalpekar, https://github.com/malfet
2022-04-15 22:05:49 +00:00
Michael Suo
85235c6f8e [lint] Use a problem matcher for GitHub annotations
Pull Request resolved: https://github.com/pytorch/pytorch/pull/75899

Approved by: https://github.com/seemethere, https://github.com/janeyx99
2022-04-15 20:20:40 +00:00
Michael Suo
3e0e137555 [lint] add test ownership lint to lintrunner
Pull Request resolved: https://github.com/pytorch/pytorch/pull/75898

Approved by: https://github.com/seemethere, https://github.com/janeyx99
2022-04-15 20:20:40 +00:00
Catherine Lee
991c89b2d1 set fetch-depth: 1
Fixes #ISSUE_NUMBER

tested via #75232 b/c need to change the source of the workflow
- set fetch-depth: 1
- manually checkout additional branches/history (usually either viable/strict, or master and the rest of the commit's history) when needed
- seems to reduce checkout time by about 30s for jobs that don't need additional branches/history, but minimal improvement otherwise
  - checkouts for most lint jobs now takes <15s

Rough estimates for how long different parts of checkout take on linux (windows is similar, but scaled up):
- just the commit, no history: <15s, seems to be around 6-7s
- viable/strict: 25-30s
- submodules: 80-120s
- master + commit history: 40-50s (if checked out viable/strict before this, then this time is much smaller, <10s)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/75783
Approved by: https://github.com/seemethere, https://github.com/janeyx99
2022-04-15 19:48:30 +00:00
Michael Suo
9555f3b3c1 [lint] improve retries in stale job
Add exponential backoff and some logging to figure out if it's working

Pull Request resolved: https://github.com/pytorch/pytorch/pull/75860

Approved by: https://github.com/seemethere
2022-04-15 18:02:14 +00:00
Michael Suo
356f1478d8 [lint] add actionlint to lintrunner
Pull Request resolved: https://github.com/pytorch/pytorch/pull/75857

Approved by: https://github.com/malfet
2022-04-15 04:03:54 +00:00
Michael Suo
cbbb96c271 [lint] add shellcheck to lintrunner
As title

Pull Request resolved: https://github.com/pytorch/pytorch/pull/75856

Approved by: https://github.com/malfet
2022-04-15 04:03:54 +00:00
Michael Suo
03cbdaee20 [lint] use new merge-base-with feature in lintrunner ci
Pull Request resolved: https://github.com/pytorch/pytorch/pull/75816

Approved by: https://github.com/seemethere
2022-04-15 02:50:57 +00:00
Michael Suo
e843b6667e [ci] add retries to stale action
We are seeing some http errors, a retry should help. Hook-based approach
suggested in https://github.com/octokit/octokit.js/issues/1069#retry

Example workflow: https://github.com/pytorch/pytorch/runs/6033246410?check_suite_focus=true
Pull Request resolved: https://github.com/pytorch/pytorch/pull/75855
Approved by: https://github.com/seemethere
2022-04-15 02:12:33 +00:00
Michael Suo
1c60b9aaa5 [ci] use lintrunner in CI
This changes our lint workflows to use lintrunner for the linters that
are currently supported

+ some random fixes to make things lint clean on master
+ changes to Makefile to use lintrunner

Pull Request resolved: https://github.com/pytorch/pytorch/pull/68460

Approved by: https://github.com/t10-13rocket, https://github.com/seemethere, https://github.com/janeyx99
2022-04-15 00:08:21 +00:00
PyTorch MergeBot
db6165215e Revert "[ci] use lintrunner in CI"
This reverts commit 4c3ee53522.

Reverted https://github.com/pytorch/pytorch/pull/68460 on behalf of https://github.com/malfet
2022-04-14 23:27:27 +00:00
Sergii Dymchenko
de66304aa5 Add first version of Buck build workflow
Next attempt after https://github.com/pytorch/pytorch/pull/75360 (that PR was reverted for breaking internal build because of BUCK file names conflicts).
Pull Request resolved: https://github.com/pytorch/pytorch/pull/75815
Approved by: https://github.com/linbinyu
2022-04-14 20:18:13 +00:00
PyTorch MergeBot
496d4bb7ca Revert "Add first version of Buck build workflow"
This reverts commit 045228bad1.

Reverted https://github.com/pytorch/pytorch/pull/75360 on behalf of https://github.com/kit1980
2022-04-14 18:01:33 +00:00
Michael Suo
4c3ee53522 [ci] use lintrunner in CI
This changes our lint workflows to use lintrunner for the linters that
are currently supported

+ some random fixes to make things lint clean on master
+ changes to Makefile to use lintrunner

Pull Request resolved: https://github.com/pytorch/pytorch/pull/68460

Approved by: https://github.com/t10-13rocket, https://github.com/seemethere, https://github.com/janeyx99
2022-04-14 17:43:41 +00:00
PyTorch MergeBot
1118b157bc Revert "Remove 11.5 experimental builds now that we have 11.6"
This reverts commit 5d059d20ad.

Reverted https://github.com/pytorch/pytorch/pull/75749 on behalf of https://github.com/janeyx99
2022-04-14 00:54:22 +00:00
Sergii Dymchenko
045228bad1 Add first version of Buck build workflow
Pull Request resolved: https://github.com/pytorch/pytorch/pull/75360
Approved by: https://github.com/linbinyu, https://github.com/seemethere
2022-04-13 23:50:23 +00:00
Jane Xu
5d059d20ad Remove 11.5 experimental builds now that we have 11.6
CUDA 11.5 was experimental, but now that we are hoping to jump to 11.6, we don't need to keep 11.5 around.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/75749
Approved by: https://github.com/seemethere
2022-04-13 20:28:52 +00:00
Jon Janzen
01c8ac3bd2 [pytorch][require export] Don't require the repo to be cloned
Summary: Turns out, we don't install `git` on Meta's internal deployment images. Instead of using git, we can just use the API if the repo is `None`.

Differential Revision: D35598595

Pull Request resolved: https://github.com/pytorch/pytorch/pull/75709
Approved by: https://github.com/seemethere, https://github.com/osalpekar
2022-04-13 20:19:07 +00:00