OSSForks/pytorch
1
0
Fork 0
mirror of https://github.com/zebrajr/pytorch.git synced 2025-12-06 00:20:18 +01:00
Code Issues Actions 38 Packages Projects Releases Wiki Activity
b2eb0e8c6a
pytorch/.github/scripts
History
Stonepia 30d7e7a1cd [XPU] Fix patch for old llvm package error for triton xpu (#134204) 
Fixes #134199

The PR #133694 does a workaround to replace the str `"https://tritonlang.blob.core.windows.net/llvm-builds/"` with  `"https://oaitriton.blob.core.windows.net/public/llvm-builds/"` in `triton/python/setup.py`. However, in [newer version of Triton](06e6799f4e), it has already been changed to `"https://oaitriton.blob.core....` and don't need to be replaced.  But formerly, this will throw a runtime error.

This PR makes the `check_and_replace` logic won't fail in such a scenario. Both the old link and the newer link could work.

Also note that the `.ci/docker/common/install_triton.sh` does not need the fix, because its `sed` command won't be in effect if there is no such pattern.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/134204
Approved by: https://github.com/chuanqi129, https://github.com/EikanWang, https://github.com/atalman
2024-08-22 23:18:44 +00:00
..
amd [ROCm] Update nightly triton-rocm pin to release branch (#129361) 2024-07-02 11:49:52 +00:00
s390x-ci Add copy of scripts for setting up s390x workers (#120417) 2024-02-23 17:01:44 +00:00
build_triton_wheel.py [XPU] Fix patch for old llvm package error for triton xpu (#134204) 2024-08-22 23:18:44 +00:00
check_labels.py [BE][Easy][2/19] enforce style for empty lines in import segments in .ci/ and .github/ (#129753) 2024-07-16 09:40:00 +00:00
cherry_pick.py Cherry-Picking don't resolve conflicts (#134047) 2024-08-20 21:48:05 +00:00
close_nonexistent_disable_issues.py [BE][Easy][2/19] enforce style for empty lines in import segments in .ci/ and .github/ (#129753) 2024-07-16 09:40:00 +00:00
collect_ciflow_labels.py [BE][Easy][2/19] enforce style for empty lines in import segments in .ci/ and .github/ (#129753) 2024-07-16 09:40:00 +00:00
comment_on_pr.py [EZ][GHF] Rephrase cancelled message (#124947) 2024-04-25 22:24:08 +00:00
convert_lintrunner_annotations_to_github.py [BE][Easy][2/19] enforce style for empty lines in import segments in .ci/ and .github/ (#129753) 2024-07-16 09:40:00 +00:00
delete_old_branches.py [BE][Easy][2/19] enforce style for empty lines in import segments in .ci/ and .github/ (#129753) 2024-07-16 09:40:00 +00:00
docathon-label-sync.py Add a GH action to autolabel docathon PRs (#127569) 2024-05-31 17:57:07 +00:00
drci_mocks.json.gz Skip signals from older runs of the same workflows (#129291) 2024-06-26 03:49:09 +00:00
ensure_actions_will_cancel.py [BE][Easy][2/19] enforce style for empty lines in import segments in .ci/ and .github/ (#129753) 2024-07-16 09:40:00 +00:00
export_pytorch_labels.py [BE][Easy][2/19] enforce style for empty lines in import segments in .ci/ and .github/ (#129753) 2024-07-16 09:40:00 +00:00
file_io_utils.py [BE] Enable ruff's UP rules and autoformat tools and scripts (#105428) 2023-07-19 01:24:44 +00:00
filter_test_configs.py [CI] add new test config label ci-test-showlocals to control test log verbosity (#131981) 2024-07-29 18:53:14 +00:00
generate_binary_build_matrix.py Enable CUDA 12.4.1 (#132202) 2024-08-20 17:52:50 +00:00
generate_ci_workflows.py Move python 3.8 to 3.9 for linux-binary-manywheel workflow (#133621) 2024-08-16 13:49:26 +00:00
generate_docker_release_matrix.py [BE][Easy][2/19] enforce style for empty lines in import segments in .ci/ and .github/ (#129753) 2024-07-16 09:40:00 +00:00
generate_pytorch_version.py [BE][Easy][2/19] enforce style for empty lines in import segments in .ci/ and .github/ (#129753) 2024-07-16 09:40:00 +00:00
get_aws_session_tokens.py [CI] Script to fetch creds from current AWS session (#121426) 2024-03-07 19:25:54 +00:00
get_workflow_job_id.py [BE][Easy][2/19] enforce style for empty lines in import segments in .ci/ and .github/ (#129753) 2024-07-16 09:40:00 +00:00
github_utils.py [BE][Easy][2/19] enforce style for empty lines in import segments in .ci/ and .github/ (#129753) 2024-07-16 09:40:00 +00:00
gitutils.py [BE][Easy] enable ruff rule PIE790: unnecessary pass statement (#133200) 2024-08-15 15:50:19 +00:00
gql_mocks.json.gz Skip signals from older runs of the same workflows (#129291) 2024-06-26 03:49:09 +00:00
kill_active_ssh_sessions.ps1 .github: Enable with-ssh for Windows (#63440) 2021-08-24 14:14:27 -07:00
label_utils.py [BE][Easy][2/19] enforce style for empty lines in import segments in .ci/ and .github/ (#129753) 2024-07-16 09:40:00 +00:00
lint_native_functions.py Revert "[BE][Easy] use pathlib.Path instead of dirname / ".." / pardir (#129374)" 2024-06-29 00:47:15 +00:00
lintrunner.sh Flip default value for mypy disallow_untyped_defs [final] (#127836) 2024-06-12 15:28:42 +00:00
parse_ref.py [BE] Apply ufmt to run_test and GitHub Python util scripts (#97588) 2023-03-26 04:52:55 +00:00
pr-sanity-check.sh Make sure we get full file path for filtering in pr-sanity-check (#100978) 2023-05-09 18:42:42 +00:00
pytest_cache.py [BE][Easy][2/19] enforce style for empty lines in import segments in .ci/ and .github/ (#129753) 2024-07-16 09:40:00 +00:00
pytest_caching_utils.py [BE][Easy][2/19] enforce style for empty lines in import segments in .ci/ and .github/ (#129753) 2024-07-16 09:40:00 +00:00
README.md Rename default branch to main (#99210) 2023-04-16 18:48:14 -07:00
report_git_status.sh
rockset_mocks.json.gz [BE] Clean up trymerge code handling broken trunk failures (#111520) 2023-10-19 02:30:56 +00:00
runner_determinator.py Add get_optin_feature() to allow opt-in to amz2023 (#131792) 2024-08-06 17:54:20 +00:00
stop_runner_service.sh Stop runner service when its GPU crashes (#97585) 2023-03-29 21:17:13 +00:00
sync_distributed_folder_prototype.sh Switch sync_distributed_folder to use non-reverse order (#131683) 2024-07-25 20:44:23 +00:00
tag_docker_images_for_release.py [BE][Easy] apply autofix for ruff rules unnecessary-collection-call (C408): list() / tuple() / dict() (#130199) 2024-07-11 17:30:28 +00:00
td_llm_indexer.sh [TD] Pin numpy to 1.26.0 in indexer (#130442) 2024-07-10 20:52:24 +00:00
test_check_labels.py [BE] Enable ruff's UP rules and autoformat tools and scripts (#105428) 2023-07-19 01:24:44 +00:00
test_filter_test_configs.py [CI] add new test config label ci-test-showlocals to control test log verbosity (#131981) 2024-07-29 18:53:14 +00:00
test_gitutils.py Revert "[BE][Easy] use pathlib.Path instead of dirname / ".." / pardir (#129374)" 2024-06-29 00:47:15 +00:00
test_label_utils.py [BE] Apply ufmt to run_test and GitHub Python util scripts (#97588) 2023-03-26 04:52:55 +00:00
test_pytest_caching_utils.py Preserve PyTest Cache across job runs (#100522) 2023-05-10 18:37:28 +00:00
test_trymerge.py [BE][Easy][2/19] enforce style for empty lines in import segments in .ci/ and .github/ (#129753) 2024-07-16 09:40:00 +00:00
test_tryrebase.py Bot message changes for -f and rebase (#106150) 2023-07-28 16:13:51 +00:00
trymerge_explainer.py Bot message changes for -f and rebase (#106150) 2023-07-28 16:13:51 +00:00
trymerge.py [GHF] Fix co-authors attribution (#133372) 2024-08-14 00:48:24 +00:00
tryrebase.py [BE][Easy][2/19] enforce style for empty lines in import segments in .ci/ and .github/ (#129753) 2024-07-16 09:40:00 +00:00
wait_for_ssh_to_drain.ps1 .github: Enable with-ssh for Windows (#63440) 2021-08-24 14:14:27 -07:00

README.md

pytorch/.github

NOTE: This README contains information for the .github directory but cannot be located there because it will overwrite the repo README.

This directory contains workflows and scripts to support our CI infrastructure that runs on GitHub Actions.

Workflows

  • Pull CI (pull.yml) is run on PRs and on main.
  • Trunk CI (trunk.yml) is run on trunk to validate incoming commits. Trunk jobs are usually more expensive to run so we do not run them on PRs unless specified.
  • Scheduled CI (periodic.yml) is a subset of trunk CI that is run every few hours on main.
  • Binary CI is run to package binaries for distribution for all platforms.

Templates

Templates written in Jinja are located in the .github/templates directory and used to generate workflow files for binary jobs found in the .github/workflows/ directory. These are also a couple of utility templates used to discern common utilities that can be used amongst different templates.

(Re)Generating workflow files

You will need jinja2 in order to regenerate the workflow files which can be installed using:

pip install -r .github/requirements/regenerate-requirements.txt

Workflows can be generated / regenerated using the following command:

.github/regenerate.sh

Adding a new generated binary workflow

New generated binary workflows can be added in the .github/scripts/generate_ci_workflows.py script. You can reference examples from that script in order to add the workflow to the stream that is relevant to what you particularly care about.

Different parameters can be used to achieve different goals, i.e. running jobs on a cron, running only on trunk, etc.

ciflow (trunk)

The label ciflow/trunk can be used to run trunk only workflows. This is especially useful if trying to re-land a PR that was reverted for failing a non-default workflow.

Infra

Currently most of our self hosted runners are hosted on AWS, for a comprehensive list of available runner types you can reference .github/scale-config.yml.

Exceptions to AWS for self hosted:

  • ROCM runners

Adding new runner types

New runner types can be added by committing changes to .github/scale-config.yml. Example: https://github.com/pytorch/pytorch/pull/70474

NOTE: New runner types can only be used once the changes to .github/scale-config.yml have made their way into the default branch

Testing pytorch/builder changes

In order to test changes to the builder scripts:

  1. Specify your builder PR's branch and repo as builder_repo and builder_branch in .github/templates/common.yml.j2.
  2. Regenerate workflow files with .github/regenerate.sh (see above).
  3. Submit fake PR to PyTorch. If changing binaries build, add an appropriate label like ciflow/binaries to trigger the builds.