pytorch/tools/test/test_extract_scripts.py
Sam Estep c5e80d30bf Harden "Add annotations" workflow (#56071)
Summary:
Resolves https://github.com/pytorch/pytorch/issues/55810 by closing some possible security holes due to using [GitHub Actions `${{ <expressions> }}`](https://docs.github.com/en/actions/reference/context-and-expression-syntax-for-github-actions#about-contexts-and-expressions) in `.github/workflows/add_annotations.yml` and also patching a few other possible scenarios that could cause the workflow to fail by a PR passing a malformed artifact.

- [x] flag and remove GitHub Actions expressions in JS scripts
- [x] don't fail the workflow if the artifact doesn't look as expected
- [x] write unit tests for `tools/extract_scripts.py`

Pull Request resolved: https://github.com/pytorch/pytorch/pull/56071

Test Plan:
I tested the end-to-end "Lint" and "Add annotations" system in a separate sandbox repo, including the following cases:

- well-formed artifact
- missing artifact
- artifact containing a file named `linter-output.zip` (name clash)
- artifact whose `commit-sha.txt` doesn't contain a 40-digit hex string
- artifact whose `commit-sha.txt` contains a 40-digit hex string that isn't a valid Git hash for the current repo
  - in this last case, the workflow does fail, but handling that is the responsibility of [pytorch/add-annotations-github-action](https://github.com/pytorch/add-annotations-github-action), not pytorch/pytorch

To run the new unit tests added in this PR:
```
python tools/test/test_extract_scripts.py
```

Reviewed By: seemethere

Differential Revision: D27807074

Pulled By: samestep

fbshipit-source-id: e2d3cc5437fe80ff03d46237ebba289901bc567c
2021-04-16 07:46:20 -07:00

86 lines
2.1 KiB
Python

import unittest
from tools import extract_scripts
requirements_sh = '''
#!/usr/bin/env bash
set -eo pipefail
pip install -r requirements.txt
'''.strip()
hello_sh = '''
#!/usr/bin/env sh
set -e
echo hello world
'''.strip()
class TestExtractScripts(unittest.TestCase):
def test_extract_none(self) -> None:
self.assertEqual(
extract_scripts.extract({
'name': 'Checkout PyTorch',
'uses': 'actions/checkout@v2',
}),
None,
)
def test_extract_run_default_bash(self) -> None:
self.assertEqual(
extract_scripts.extract({
'name': 'Install requirements',
'run': 'pip install -r requirements.txt',
}),
{
'extension': '.sh',
'script': requirements_sh,
},
)
def test_extract_run_sh(self) -> None:
self.assertEqual(
extract_scripts.extract({
'name': 'Hello world',
'run': 'echo hello world',
'shell': 'sh',
}),
{
'extension': '.sh',
'script': hello_sh,
},
)
def test_extract_run_py(self) -> None:
self.assertEqual(
extract_scripts.extract({
'name': 'Hello world',
'run': 'print("Hello!")',
'shell': 'python',
}),
{
'extension': '.py',
'script': 'print("Hello!")',
},
)
def test_extract_github_script(self) -> None:
self.assertEqual(
# https://github.com/actions/github-script/tree/v3.1.1#reading-step-results
extract_scripts.extract({
'uses': 'actions/github-script@v3',
'id': 'set-result',
'with': {
'script': 'return "Hello!"',
'result-encoding': 'string',
},
}),
{
'extension': '.js',
'script': 'return "Hello!"',
},
)
if __name__ == '__main__':
unittest.main()