Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/55328
**Summary**
The `clang-format` reference hashes committed in #54737 have newlines at
the end but the locally computed ones do not. This commit removes these
newlines so that the `clang-format` binary verification step doesn't
fail.
**Test Plan**
`./tools/clang_format_all.py`, ran successfully.
**Fixes**
This commit fixes#54790.
Test Plan: Imported from OSS
Reviewed By: nikithamalgifb
Differential Revision: D27577398
Pulled By: SplitInfinity
fbshipit-source-id: e30bee58c2eb5ea96ed0a503480dea4f67b86aca
Summary:
*Context:* https://github.com/pytorch/pytorch/issues/53406 added a lint for trailing whitespace at the ends of lines. However, in order to pass FB-internal lints, that PR also had to normalize the trailing newlines in four of the files it touched. This PR adds an OSS lint to normalize trailing newlines.
The changes to the following files (made in 54847d0adb9be71be4979cead3d9d4c02160e4cd) are the only manually-written parts of this PR:
- `.github/workflows/lint.yml`
- `mypy-strict.ini`
- `tools/README.md`
- `tools/test/test_trailing_newlines.py`
- `tools/trailing_newlines.py`
I would have liked to make this just a shell one-liner like the other three similar lints, but nothing I could find quite fit the bill. Specifically, all the answers I tried from the following Stack Overflow questions were far too slow (at least a minute and a half to run on this entire repository):
- [How to detect file ends in newline?](https://stackoverflow.com/q/38746)
- [How do I find files that do not end with a newline/linefeed?](https://stackoverflow.com/q/4631068)
- [How to list all files in the Git index without newline at end of file](https://stackoverflow.com/q/27624800)
- [Linux - check if there is an empty line at the end of a file [duplicate]](https://stackoverflow.com/q/34943632)
- [git ensure newline at end of each file](https://stackoverflow.com/q/57770972)
To avoid giving false positives during the few days after this PR is merged, we should probably only merge it after https://github.com/pytorch/pytorch/issues/54967.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/54737
Test Plan:
Running the shell script from the "Ensure correct trailing newlines" step in the `quick-checks` job of `.github/workflows/lint.yml` should print no output and exit in a fraction of a second with a status of 0. That was not the case prior to this PR, as shown by this failing GHA workflow run on an earlier draft of this PR:
- https://github.com/pytorch/pytorch/runs/2197446987?check_suite_focus=true
In contrast, this run (after correcting the trailing newlines in this PR) succeeded:
- https://github.com/pytorch/pytorch/pull/54737/checks?check_run_id=2197553241
To unit-test `tools/trailing_newlines.py` itself (this is run as part of our "Test tools" GitHub Actions workflow):
```
python tools/test/test_trailing_newlines.py
```
Reviewed By: malfet
Differential Revision: D27409736
Pulled By: samestep
fbshipit-source-id: 46f565227046b39f68349bbd5633105b2d2e9b19
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/50520
**Summary**
The new version of `clang-format` for linux64 that was uploaded to S3
earlier this week was dynamically linked to fbcode's custom platform.
A new binary has been uploaded that statically links against `libgcc`
and `libstdc++`, which seems to have fixed this issue. Ideally, all
libraries would be statically linked.
**Test Plan**
`clang-format` workflow passes on this PR and output shows that it
successfully downloaded, verified and ran.
```
Created directory /home/runner/work/pytorch/pytorch/.clang-format-bin for clang-format binary
Downloading clang-format to /home/runner/work/pytorch/pytorch/.clang-format-bin
Reference Hash: 9073602de1c4e1748f2feea5a0782417b20e3043
Actual Hash: 9073602de1c4e1748f2feea5a0782417b20e3043
Using clang-format located at /home/runner/work/pytorch/pytorch/.clang-format-bin/clang-format
no modified files to format
```
Test Plan: Imported from OSS
Reviewed By: pbelevich
Differential Revision: D25908868
Pulled By: SplitInfinity
fbshipit-source-id: 5667fc5546e5ed0bbf9f36570935d245eb26629b
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/50399
**Summary**
This commit updates the expected hashes of the `clang-format` binaries
downloaded from S3. These binaries themselves have been updated due to
having been updated inside fbcode.
**Test Plan**
Uploaded new binaries to S3, deleted `.clang-format-bin` and ran
`clang_format_all.py`.
Test Plan: Imported from OSS
Reviewed By: seemethere
Differential Revision: D25875184
Pulled By: SplitInfinity
fbshipit-source-id: da483735de1b5f1dab7b070f91848ec5741f00b1
Summary:
This commit adds a reference hash for the linux64 clang-format binary and in
doing so, enables this script to be used on Linux machines.
Test Plan:
Ran the script.
```
meghanl@devvm1517:caffe2 (ff25240c|remote/master)$ export http_proxy=fwdproxy:8080
meghanl@devvm1517:caffe2 (ff25240c|remote/master)$ export https_proxy=fwdproxy:8080
meghanl@devvm1517:caffe2 (ff25240c|remote/master)$ python3 ./tools/clang_format_new.py --diff
Downloading clang-format to /data/users/meghanl/fbsource/fbcode/caffe2/.clang-format-bin
0% |################################################################| 100%
Using clang-format located at /data/users/meghanl/fbsource/fbcode/caffe2/.clang-format-bin/clang-format
meghanl@devvm1517:caffe2 (ff25240c|remote/master)$ echo $?
1
```
A non-zero return code indicates that `clang-format` will make changes.
Reviewed By: suo
Differential Revision: D20434291
fbshipit-source-id: fa13766e9d94720d4b0d8a540d2f1507e788f7a5
Summary:
**Summary**
This commit adds `tools/clang_format_new.py`, which downloads a platform-appropriate
clang-format binary to a `.gitignored` location, verifies the binary by comparing its
SHA1 hash to a reference hash (also included in this commit), and runs it on all files
matched a specific regex in a list of whitelisted subdirectories of pytorch.
This script will eventually replace `tools/clang_format.py`.
**Testing**
Ran the script.
*No Args*
```
pytorch > ./tools/clang_format.py
Downloading clang-format to /Users/<user>/Desktop/pytorch/.clang-format-bin
0% |################################################################| 100%
Using clang-format located at /Users/<user>/Desktop/pytorch/.clang-format-bin/clang-format
> echo $?
0
> git status
<bunch of files>
```
`--diff` *mode*
```
> ./tools/clang_format.py --diff
Using clang-format located at /Users/<user>/Desktop/pytorch/.clang-format-bin/clang-format
Some files are not formatted correctly
> echo $?
1
<format files using the script>
> ./tools/clang_format.py --diff
Using clang-format located at /Users/<user>/Desktop/pytorch/.clang-format-bin/clang-format
All files are formatted correctly
> echo $?
0
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/34566
Differential Revision: D20431290
Pulled By: SplitInfinity
fbshipit-source-id: 3966f769cfb923e58ead9376d85e97127415bdc6
