Elton Leander Pinto 711ded688d Add a script to codemod max_tokens_total pragmas to C/C++ files (#61369) ...

Summary:
This PR adds a new script: `max_tokens_pragmas.py`

This is a utility script that can add/remove `max_tokens_total` pragmas from the codebase.

- [x] Implement script and test manually
- [x] Write test script

Examples:
First, change directories
```bash
cd tools/linter/clang_tidy
```

Then run the following:
```bash
cat << EOF > test/test1.cpp
// File without any prior pragmas

int main() {
    for (int i = 0; i < 10; i++);
    return 0;
}
EOF

cat << EOF > test/test2.cpp
// File with prior pragmas

#pragma clang max_tokens_total 1

int main() {
    for (int i = 0; i < 10; i++);
    return 0;
}
EOF

cat << EOF > test/test3.cpp
// File with multiple prior pragmas

#pragma clang max_tokens_total 1

// Different pragma; script should ignore this
#pragma clang max_tokens_here 20

int main() {
    for (int i = 0; i < 10; i++);
    return 0;
}

#pragma clang max_tokens_total 1
EOF

# Add pragmas to some files
python3 max_tokens_pragma.py --num-max-tokens 42 test/*.cpp
grep "#pragma clang max_tokens_total 42" test/*.cpp

# Remove pragmas from files
python3 max_tokens_pragma.py --strip test/*.cpp
grep "#pragma clang max_tokens_total 42" test/*.cpp # should fail

# Ignore files
python3 max_tokens_pragma.py --num-max-tokens 42 test/*.cpp --ignores test/test2.cpp
grep "#pragma clang max_tokens_total 42" test/*.cpp # should not list `test/test2.cpp`
```

Pull Request resolved: https://github.com/pytorch/pytorch/pull/61369

Test Plan: `tools/linter/clang_tidy/test/test_max_tokens_pragma.py`

Reviewed By: malfet

Differential Revision: D29604291

Pulled By: 1ntEgr8

fbshipit-source-id: 3efe52573583769041a07e6776161d4d5bbf16a7

2021-07-09 13:30:52 -07:00

0 lines

Python

Raw Blame History