OSSForks/pytorch
1
0
Fork 0
mirror of https://github.com/zebrajr/pytorch.git synced 2025-12-07 12:21:27 +01:00
Code Issues Actions 38 Packages Projects Releases Wiki Activity
48ea440b15
pytorch/.github/scripts
History
Eli Uriegas 48ea440b15 ci: Unblock syncbranches, add a58c6ae and 7106d21 to block list (#76417) 
Summary:
Adds a58c6aea5a0c9f8759a4154e46f544c8b03b8db1 and 7106d216c29ca16a3504aa2bedad948ebcf4abc2 to the list of excluded
commits since this was landed through phab and cherry picked to master
directly

Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

Pull Request resolved: https://github.com/pytorch/pytorch/pull/76417

Reviewed By: janeyx99

Differential Revision: D35951416

Pulled By: seemethere

fbshipit-source-id: 30a226c381e0cebfccc82f7ccfa7ce79075220c9
(cherry picked from commit b75fbe3b9e8024734e749a42464620c1879265ad)
2022-04-26 23:54:22 +00:00
..
build_publish_nightly_docker.sh ci: Fix cudatoolkit issue, make docker builds testable 2022-03-09 19:49:05 +00:00
convert_lintrunner_annotations_to_github.py [ci] use lintrunner in CI 2022-04-15 00:08:21 +00:00
ensure_actions_will_cancel.py Workflow consolidation for GitHub actions 2022-03-21 14:15:03 +00:00
export_pytorch_labels.py Add lint to ensure all test files have headers with ownership info (#66826) 2021-11-03 18:21:49 -07:00
generate_binary_build_matrix.py Remove cuda 11.5 builds since we have 11.6 2022-04-23 00:24:58 +00:00
generate_ci_workflows.py remove from generate_ci_workflows.py, add tags back in .github/templates 2022-04-08 15:48:18 +00:00
generate_pytorch_version.py .github: Fix --no-build-suffix (#62739) 2021-08-04 14:19:38 -07:00
get_workflow_job_id.py [ci] Fix bug in get_workflow_job_id.py 2022-04-08 21:09:38 +00:00
gitutils.py ci: Unblock syncbranches, add a58c6ae and 7106d21 to block list (#76417) 2022-04-26 23:54:22 +00:00
gql_mocks.json [GHF] Add pagination to commits_with_authors 2022-04-21 16:25:28 +00:00
install_nvidia_utils_linux.sh Cuda 11.6 Disable failing tests (#75420) 2022-04-07 22:43:15 +00:00
kill_active_ssh_sessions.ps1 .github: Enable with-ssh for Windows (#63440) 2021-08-24 14:14:27 -07:00
lint_native_functions.py Disable warn_unused_ignores again (#60480) 2021-06-22 12:42:37 -07:00
parse_ref.py Use tools/print_test_stats.py in GHA (#57647) 2021-05-10 12:29:40 -07:00
process_commit.py Fix alignment, make sure release labels are included (#73739) 2022-03-03 16:30:55 +00:00
README.md [BE] add readme for .github, located in .github/scripts 2022-04-01 19:53:44 +00:00
report_git_status.sh [lint] Move shellcheck to its own step (#58623) 2021-05-21 18:23:40 -07:00
run_torchbench.py Remove the dependency of pytorch nightly. (#71323) 2022-01-17 09:52:36 -08:00
syncbranches.py [GHA] Small updates to syncbranches.yml 2022-04-04 19:20:37 +00:00
test_gitutils.py Add gitutils tests (#71580) 2022-01-20 14:33:41 -08:00
test_trymerge.py [GHF] Add pagination to commits_with_authors 2022-04-21 16:25:28 +00:00
trymerge.py [GHF] Add pagination to commits_with_authors 2022-04-21 16:25:28 +00:00
wait_for_ssh_to_drain.ps1 .github: Enable with-ssh for Windows (#63440) 2021-08-24 14:14:27 -07:00
wait_for_ssh_to_drain.sh .github: Enable SSH to linux runners (#62280) 2021-08-03 15:15:39 -07:00

README.md

pytorch/.github

NOTE: This README contains information for the .github directory but cannot be located there because it will overwrite the repo README.

This directory contains workflows and scripts to support our CI infrastructure that runs on Github Actions.

Workflows

  • Pull CI (pull.yml) is run on PRs and on master.
  • Trunk CI (trunk.yml) is run on trunk to validate incoming commits. Trunk jobs are usually more expensive to run so we do not run them on PRs unless specified.
  • Scheduled CI (periodic.yml) is a subset of trunk CI that is run every few hours on master.
  • Binary CI is run to package binaries for distribution for all platforms.

Templates

Templates written in Jinja are located in the .github/templates directory and used to generate workflow files for binary jobs found in the .github/workflows/ directory. These are also a couple of utility templates used to discern common utilities that can be used amongst different templates.

(Re)Generating workflow files

You will need jinja2 in order to regenerate the workflow files which can be installed using:

pip install -r .github/requirements.txt

Workflows can be generated / regenerated using the following command:

.github/regenerate.sh

Adding a new generated binary workflow

New generated binary workflows can be added in the .github/scripts/generate_ci_workflows.py script. You can reference examples from that script in order to add the workflow to the stream that is relevant to what you particularly care about.

Different parameters can be used to acheive different goals, i.e. running jobs on a cron, running only on trunk, etc.

ciflow (trunk)

The label ciflow/trunk can be used to run trunk only workflows. This is especially useful if trying to re-land a PR that was reverted for failing a non-default workflow.

Infra

Currently most of our self hosted runners are hosted on AWS, for a comprehensive list of available runner types you can reference .github/scale-config.yml.

Exceptions to AWS for self hosted:

  • ROCM runners

Adding new runner types

New runner types can be added by committing changes to .github/scale-config.yml. Example: https://github.com/pytorch/pytorch/pull/70474

NOTE: New runner types can only be used once the changes to .github/scale-config.yml have made their way into the default branch