Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/67455
Migrates docker builds that don't have dependent jobs within the pytorch
repository to our new GHA docker build job
Signed-off-by: Eli Uriegas <eliuriegas@fb.com>
Test Plan: Imported from OSS
Reviewed By: malfet, janeyx99
Differential Revision: D31997671
Pulled By: seemethere
fbshipit-source-id: 9d6f58fa8ea8731cf12457fe64dc65e70f3d9f25
Summary:
linux-xenial-cuda10.2 and linux-bionic-cuda10.2 are very similar, no
need to run both configs
Moved all auxiliary builds from xenial to bionic
Pull Request resolved: https://github.com/pytorch/pytorch/pull/67344
Reviewed By: seemethere, janeyx99
Differential Revision: D31964850
Pulled By: malfet
fbshipit-source-id: d07ce266c843c7fd69b281e678c4247b0bf6da20
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/67264
Downgrades linux gpu instances from 4xlarge -> 8xlarge
We were seeing capacity issues in terms of scaling 8xlarge instances,
downgrading this to 4xlarge (which only have a single gpu) to see if
that helps resolve some of the capacity issues we were seeing
Signed-off-by: Eli Uriegas <eliuriegas@fb.com>
Test Plan: Imported from OSS
Reviewed By: janeyx99
Differential Revision: D31933488
Pulled By: seemethere
fbshipit-source-id: b41922ebb675e663cb035cd3795bc9bae94dcac7
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/67215
We were regularly seeing gaps in our docker image builds due to specific
workflows not being run when docker builds occurred on PRs, this should
remove that ambiguity and ensure that all docker builds be re-built if a
rebuild is deemed necessary
Signed-off-by: Eli Uriegas <eliuriegas@fb.com>
Test Plan: Imported from OSS
Reviewed By: albanD
Differential Revision: D31910422
Pulled By: seemethere
fbshipit-source-id: f346e64f1857e35a995c49bf30521a3acd8af0b1
Summary:
CAFFE2 has been deprecated for a while, but still included in every PyTorch build.
We should stop building it by default, although CI should still validate that caffe2 code is buildable.
Build even fewer dependencies when compiling mobile builds without Caffe2
Introduce `TEST_CAFFE2` in torch.common.utils
Skip `TestQuantizedEmbeddingOps` and `TestJit.test_old_models_bc` is code is compiled without Caffe2
Should be landed after https://github.com/pytorch/builder/pull/864
Pull Request resolved: https://github.com/pytorch/pytorch/pull/66658
Reviewed By: driazati, seemethere, janeyx99
Differential Revision: D31669156
Pulled By: malfet
fbshipit-source-id: 1cc45e2d402daf913a4685eb9f841cc3863e458d
Summary:
`linux-xenial-py3-clang5-mobile-build`, `linux-xenial-py3-clang5-mobile-custom-build-dynamic`, `linux-xenial-py3-clang5-mobile-custom-build-dynamic` and `linux-xenial-py3-clang5-mobile-code-analysis` are just the flavors of regular linux build job with no tests.
`linux-xenial-py3-clang5-mobile-code-analysis` is the master only job
`code-analysis` job is dispatch to `.jenkins/pytorch/build-mobile-code-analysis.sh` in
583217fe37/.jenkins/pytorch/build.sh (L23-L25)
and all `mobile-build` jobs are dispatched to `.jenkins/pytorch/build-mobile.sh` in
583217fe37/.jenkins/pytorch/build.sh (L19-L21)
Rename `is_libtorch` `CIWorkflow` property into `build_generates_artifacts` and change defaults from False to True
Both libtorch and mobile build jobs do not generate build artifacts
Pull Request resolved: https://github.com/pytorch/pytorch/pull/66673
Reviewed By: janeyx99
Differential Revision: D31674434
Pulled By: malfet
fbshipit-source-id: 24d05d55366202cd4d9c25ecab429cb8f670ded0
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/66260
Every workflow has ciflow enabled so this is not needed anymore
Signed-off-by: Eli Uriegas <eliuriegas@fb.com>
Test Plan: Imported from OSS
Reviewed By: dagitses, janeyx99
Differential Revision: D31493340
Pulled By: seemethere
fbshipit-source-id: 8718fe5d22f4be6e0900962576782a9f23162a39
Summary:
Noticed that `periodic-pytorch-linux-xenial-cuda10.2-cudnn7-py3-gcc7-slow-gradcheck` job has a `ciflow/default`, but does not have a `ciflow/scheduled` label
Added asserts to enforce that jobs with non-trival is_scheduled property do not have default and do have scheduled labesl
Rename `periodic-pytorch-linux-xenial-cuda10.2-cudnn7-py3-gcc7-slow-gradcheck` to `periodic-linux-xenial-cuda10.2-py3-gcc7-slow-gradcheck`
Fixes #{issue number}
Pull Request resolved: https://github.com/pytorch/pytorch/pull/66300
Reviewed By: seemethere
Differential Revision: D31493323
Pulled By: malfet
fbshipit-source-id: 194c1d7a4e659847d94a547b87a0d7d08e66406d
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/65730
This should close out the door on migrating all scheduled workflows we have for CircleCI
Signed-off-by: Eli Uriegas <eliuriegas@fb.com>
cc ezyang seemethere malfet pytorch/pytorch-dev-infra
Test Plan: Imported from OSS
Reviewed By: albanD
Differential Revision: D31225188
Pulled By: seemethere
fbshipit-source-id: 4c49e88ec017edc30e07325dbc613ff54dd164d8
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/65731
It originally had purpose but after ciflow was introduced every PR had
on_pull_request set so it's not really as useful as it once was
Also removes the equally as confusing only_build_on_pull_request
variable as well
This change should produce no functional changes in our generated workflows
Signed-off-by: Eli Uriegas <eliuriegas@fb.com>
cc ezyang seemethere malfet pytorch/pytorch-dev-infra
Test Plan: Imported from OSS
Reviewed By: janeyx99
Differential Revision: D31225398
Pulled By: seemethere
fbshipit-source-id: 7bd8e8175794ab7d09b0632321bf52538435e858
Summary:
CIFLow workflows should always run on push event
On pull-request workflow should run if label conditions are met or if
no `ciflow/` labels are associated with it, workflow is enabled by
default
Pull Request resolved: https://github.com/pytorch/pytorch/pull/65733
Reviewed By: zhouzhuojie
Differential Revision: D31251278
Pulled By: malfet
fbshipit-source-id: 31ce745cb224df7c6fec1682ec4180513e3dadf3
Summary:
Part of migrating from Circle.
Once we get a successful force_on_cpu test, we can move it to trunk only.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/65094
Reviewed By: seemethere
Differential Revision: D31086289
Pulled By: janeyx99
fbshipit-source-id: e1d135cc844d51f0b243b40efb49edca277d9de8
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/65099
Utilizes ciflow to enable only specific workflows for
pytorch/pytorch-canary to reduce noise on that specific repository
Signed-off-by: Eli Uriegas <eliuriegas@fb.com>
Test Plan: Imported from OSS
Reviewed By: jbschlosser
Differential Revision: D30973691
Pulled By: seemethere
fbshipit-source-id: 371765535b42a00bd72c2551c4faebf733d759f0
Summary:
As we default to linux CUDA 11.3 on PRs, we should do the same thing with Windows (instead of having 10.2 be the default). This means that 10.2 will now be master only, and 11.3 windows smoke tests will run on every PR.
This also copies over the "run smoke tests only" config--removing that will be in a separate PR once there's more certain decision making.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/65090
Reviewed By: seemethere
Differential Revision: D30968382
Pulled By: janeyx99
fbshipit-source-id: c73f9a2cc800b678909365c4d80627d29fc09f94
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/64958
This is a re-do of #64846 which was missing a path prefix for windows test reports
Test Plan: Imported from OSS
Reviewed By: seemethere
Differential Revision: D30915253
Pulled By: driazati
fbshipit-source-id: d14d0a64d2f8aabc335db9c4d0d2b63512887c66
Summary:
Previously we just weren't uploading Windows test report XML files to S3, only to GitHub actions. This was different than Linux where we use both (though maybe we can kill the GHA upload in a follow up PR since I don't think it's very useful anymore). This factors it all out into a macro so they both do the same thing. This also fixes the naming of uploaded files to include info about the job name (the full config, so they can be matched to the job visually or by the included job id).
See https://hud.pytorch.org/pr/64846 for results
Pull Request resolved: https://github.com/pytorch/pytorch/pull/64846
Reviewed By: seemethere
Differential Revision: D30878101
Pulled By: driazati
fbshipit-source-id: 0730f17fa3f46a32c131f52669084c3103b0e616
Summary:
## Context
We take the first step at tackling the GPU-bazel support by adding bazel external workspaces `local_config_cuda` and `cuda`, where the first one has some hardcoded values and lists of files, and the second one provides a nicer, high-level wrapper that maps into the already expected by pytorch bazel targets that are guarded with `if_cuda` macro.
The prefix `local_config_` signifies the fact that we are breaking the bazel hermeticity philosophy by explicitly relaying on the CUDA installation that is present on the machine.
## Testing
Notice an important scenario that is unlocked by this change: compilation of cpp code that depends on cuda libraries (i.e. cuda.h and so on).
Before:
```
sergei.vorobev@cs-sv7xn77uoy-gpu-1628706590:~/src/pytorch4$ bazelisk build --define=cuda=true //:c10
ERROR: /home/sergei.vorobev/src/pytorch4/tools/config/BUILD:12:1: no such package 'tools/toolchain': BUILD file not found in any of the following directories. Add a BUILD file to a directory to mark it as a package.
- /home/sergei.vorobev/src/pytorch4/tools/toolchain and referenced by '//tools/config:cuda_enabled_and_capable'
ERROR: While resolving configuration keys for //:c10: Analysis failed
ERROR: Analysis of target '//:c10' failed; build aborted: Analysis failed
INFO: Elapsed time: 0.259s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (2 packages loaded, 2 targets configured)
```
After:
```
sergei.vorobev@cs-sv7xn77uoy-gpu-1628706590:~/src/pytorch4$ bazelisk build --define=cuda=true //:c10
INFO: Analyzed target //:c10 (6 packages loaded, 246 targets configured).
INFO: Found 1 target...
Target //:c10 up-to-date:
bazel-bin/libc10.lo
bazel-bin/libc10.so
INFO: Elapsed time: 0.617s, Critical Path: 0.04s
INFO: 0 processes.
INFO: Build completed successfully, 1 total action
```
The `//:c10` target is a good testing one for this, because it has such cases where the [glob is different](075024b9a3/BUILD.bazel (L76-L81)), based on do we compile for CUDA or not.
## What is out of scope of this PR
This PR is a first in a series of providing the comprehensive GPU bazel build support. Namely, we don't tackle the [cu_library](11a40ad915/tools/rules/cu.bzl (L2)) implementation here. This would be a separate large chunk of work.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/63604
Reviewed By: soulitzer
Differential Revision: D30442083
Pulled By: malfet
fbshipit-source-id: b2a8e4f7e5a25a69b960a82d9e36ba568eb64595
Summary:
Moving distributed to its own job.
- [x] ensure there should be a distributed test job for every default test job matrix (on GHA)
- [x] ensure that circleci jobs works for distributed as well
- [x] waiting for test distributed to have its own run_test.py launch options, see https://github.com/pytorch/pytorch/issues/63147
Pull Request resolved: https://github.com/pytorch/pytorch/pull/62896
Reviewed By: seemethere
Differential Revision: D30230856
Pulled By: walterddr
fbshipit-source-id: 0cad620f6cd9e56c727c105458d76539a5ae976f
Summary:
This removes the `render_test_results` job we had before which had been causing some confusion among devs when it failed and isn't really necessary now that we can actually render test results on the PR HUD.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/63877
Reviewed By: walterddr, janeyx99
Differential Revision: D30546705
Pulled By: driazati
fbshipit-source-id: 55fdafdb6f80924d941ffc15ee10787cb54f34a1
Summary:
the `ciflow/all` is automatically added but need to be added before we call `gen_root_job_condition`.
- fix the order of adding `ciflow/all`
- refactor all the string into global constants
Pull Request resolved: https://github.com/pytorch/pytorch/pull/63954
Reviewed By: malfet
Differential Revision: D30545596
Pulled By: zhouzhuojie
fbshipit-source-id: 83ab668f0234488afb855a72e3ebd4503f7f1a78
Summary:
Unlike circle which all workflow goes in one file, GHA legacy generated files will stay silently in once's PR. e.g. when we change build_environment name and that's not ideal.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/63148
Reviewed By: bdhirsh
Differential Revision: D30283382
Pulled By: walterddr
fbshipit-source-id: ffdd5bf9561dd38499052855a12ee5cf838a20b0
Summary:
Fixes https://github.com/pytorch/pytorch/issues/62357
there's no need to specify cudnn version since they are recommended from cuda version already.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/63030
Reviewed By: zhouzhuojie, driazati
Differential Revision: D30226354
Pulled By: walterddr
fbshipit-source-id: 7e2dc577810e0ce80ee27569c25a814566250ab1
