OSSForks/pytorch
1
0
Fork 0
mirror of https://github.com/zebrajr/pytorch.git synced 2025-12-06 12:20:52 +01:00
Code Issues Actions 38 Packages Projects Releases Wiki Activity
81,624 Commits 8,304 Branches 150 Tags 1.5 GiB
c17ba69ba5
Commit Graph

14 Commits

Author SHA1 Message Date
Xuehai Pan
747b38c131 [BE][Easy][2/19] enforce style for empty lines in import segments in .ci/ and .github/ (#129753) 
See https://github.com/pytorch/pytorch/pull/129751#issue-2380881501. Most changes are auto-generated by linter.

You can review these PRs via:

```bash
git diff --ignore-all-space --ignore-blank-lines HEAD~1
```

Pull Request resolved: https://github.com/pytorch/pytorch/pull/129753
Approved by: https://github.com/malfet
ghstack dependencies: #129752
 2024-07-16 09:40:00 +00:00
Aaron Gokaslan
2f3b0befed [BE]: Apply ruff FURB 118. (#124743) 
Replaces various lambdas with operator.itemgetter which is more efficient (as it's a builtin function). Particularly useful for when lambdas are used as 'key' functions.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/124743
Approved by: https://github.com/albanD, https://github.com/malfet
 2024-04-26 14:34:52 +00:00
Catherine Lee
6e73ae2022 [ci][ez] Add job_id to emit_metrics (#113099) 
As in title.

Also print the job id in the step since I'm struggling to find it
Pull Request resolved: https://github.com/pytorch/pytorch/pull/113099
Approved by: https://github.com/seemethere
 2023-11-08 10:32:41 +00:00
Huy Do
f6f81a5969 Update get-workflow-job-id to also return job name (#112103) 
Then we can use this job name in `filter-test-configs` if it's available.  This addresses the issue in which `filter-test-configs` on GitHub runners (MacOS x86) couldn't find the runner log to get the job name.  This is expected because GitHub runners are isolated, so a job should not be able to access runner logs, which could contains information from other jobs.

This allows all missing features depending on running `filter-test-configs` on GitHub runners:
* Rerun disabled tests and memory leak check. For example, this would help avoid closing https://github.com/pytorch/pytorch/issues/110980#issuecomment-1779806466 early with the disabled test running properly on MacOS x86
* MacOS x86 jobs can now be disabled or marked as unstable

I keep the current logic to parse the log as a fallback because it's working fine on self-hosted runners.  That also handles the case where `get-workflow-job-id` fails.  Also I move the rest of `get-workflow-job-id` up before the test step like https://github.com/pytorch/pytorch/pull/111483

### Testing

Spot checks some jobs to confirm they have the correct names:

* MacOS M1 test job https://github.com/pytorch/pytorch/actions/runs/6648305319/job/18065275722?pr=112103#step:10:8
* MacOS x86 build job https://github.com/pytorch/pytorch/actions/runs/6648306305/job/18065138137?pr=112103#step:9:14
* Linux test job has https://github.com/pytorch/pytorch/actions/runs/6648300991/job/18065354503?pr=112103#step:13:7
* Windows test job https://github.com/pytorch/pytorch/actions/runs/6648305319/job/18065599500?pr=112103#step:12:7
* MacOS x86 test job https://github.com/pytorch/pytorch/actions/runs/6648306305/job/18066312801#step:10:8
Pull Request resolved: https://github.com/pytorch/pytorch/pull/112103
Approved by: https://github.com/clee2000
 2023-10-26 16:42:46 +00:00
Huy Do
4c0dce50fd [BE] Apply ufmt to run_test and GitHub Python util scripts (#97588) 
This has been bugging me for a while as I'm working on these Python scripts and they are not tracked by ufmt linter.  So I add these script into that linter.

```
[[linter]]
code = 'UFMT'
include_patterns = [
    '.github/**/*.py',
    'test/run_test.py',
```

This change should just work and not break anything as ufmt (black + usort) linter is very safe to use for standalone util scripts.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/97588
Approved by: https://github.com/kit1980
 2023-03-26 04:52:55 +00:00
Nikita Shulga
31d466f925 [BE][ez] Move hardcoded constants to function args (#93874) 
Also use tail-recursion instead of for loop to dismantle pyramid of doom

Pull Request resolved: https://github.com/pytorch/pytorch/pull/93874
Approved by: https://github.com/clee2000
 2023-02-02 00:47:18 +00:00
Catherine Lee
1dcd2609b5 Add retries for get_workflow_job_id and try catch in upload_test_stats (#93401) 
upload_test_stats keeps failing b/c it can't handle when the id is workflow-<workflow_id> so add a try catch for this.

Add retries to get_workflow_job_id to try and reduce the number of times the id can't be found

Failure to upload test stats and inability to get the job id cause our sharding infra and slow test infra (probably also flaky test detection) to be less effective.  This does not completely resolve the issue since we do rely on the job id

Failure to get the workflow job id happens tragically often, hopefully retries will help
Pull Request resolved: https://github.com/pytorch/pytorch/pull/93401
Approved by: https://github.com/huydhn
 2023-02-01 18:33:32 +00:00
Nikita Shulga
32937f39f4 Don't raise error if job_id can't be fetched (#92192) 
But always return `workflowi_d`, which is not unique across reruns but it's better than failing the entire run just because API call failed. Test it locally by feeding the program an incorrect input and observe the failure.
Fixes https://github.com/pytorch/pytorch/issues/91332

Pull Request resolved: https://github.com/pytorch/pytorch/pull/92192
Approved by: https://github.com/kit1980
 2023-01-17 00:09:05 +00:00
Nikita Shulga
2293a6b95e [BE] Refactor get_workflow_job_id (#92191) 
A noop change that refactors existing codebase and prints a bit more
verbose error message when request fails.

Get rid of `requests` as it inevitable results in flakiness

TODO: Remove in a few days after PR is landed
4af5939d7a/.github/actions/get-workflow-job-id/action.yml (L29)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/92191
Approved by: https://github.com/kit1980
 2023-01-15 23:02:29 +00:00
Eli Uriegas
f471770fd4 Add bad status management for get_workflow_job_id (#91145) 
To help resolve issues like:
```
++ python3 .github/scripts/get_workflow_job_id.py 3736406815 i-08b8fd3e605729ed9
+ GHA_WORKFLOW_JOB_ID=
Warning: Attempt 2 failed. Reason: Child_process exited with error code 1
```

This should only happen when github actions is experiencing degraded service

Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

Pull Request resolved: https://github.com/pytorch/pytorch/pull/91145
Approved by: https://github.com/malfet
 2022-12-22 23:33:43 +00:00
Huy Do
3953764620 Fix get_workflow_job_id to dynamically get the repo name (#83036) 
This helps testing on pytorch-canary, which has a different repo name. The different name fails the get_workflow_job_id step there because the canary PR refers to run ID and canary runner name, for example:

https://github.com/pytorch/pytorch-canary/runs/7735191886?check_suite_focus=true
Pull Request resolved: https://github.com/pytorch/pytorch/pull/83036
Approved by: https://github.com/zengk95
 2022-08-09 21:30:25 +00:00
Michael Suo
1fc2f4cc31 [ci] Fix bug in get_workflow_job_id.py 
See comment in code for description

Pull Request resolved: https://github.com/pytorch/pytorch/pull/75531

Approved by: https://github.com/janeyx99, https://github.com/seemethere
 2022-04-08 21:09:38 +00:00
Michael Suo
9b563d6468 initial test stats work 
Pull Request resolved: https://github.com/pytorch/pytorch/pull/74406

Approved by: https://github.com/atalman, https://github.com/janeyx99
 2022-04-06 20:40:04 +00:00
Michael Suo
0111065c8f [ci] add a script to get the workflow job id 
This is inexplicably not available in the runtime environment of a job,
so we need to retrieve it using the GH API.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/74118
Approved by: https://github.com/janeyx99
 2022-03-14 21:30:16 +00:00