Commit Graph

5 Commits

Author SHA1 Message Date
Sam Estep
737d920b21 Strictly type everything in .github and tools (#59117)
Summary:
This PR greatly simplifies `mypy-strict.ini` by strictly typing everything in `.github` and `tools`, rather than picking and choosing only specific files in those two dirs. It also removes `warn_unused_ignores` from `mypy-strict.ini`, for reasons described in https://github.com/pytorch/pytorch/pull/56402#issuecomment-822743795: basically, that setting makes life more difficult depending on what libraries you have installed locally vs in CI (e.g. `ruamel`).

Pull Request resolved: https://github.com/pytorch/pytorch/pull/59117

Test Plan:
```
flake8
mypy --config mypy-strict.ini
```

Reviewed By: malfet

Differential Revision: D28765386

Pulled By: samestep

fbshipit-source-id: 3e744e301c7a464f8a2a2428fcdbad534e231f2e
2021-06-07 14:49:36 -07:00
Chen Lai
0c3db1cb33 [Pytorch] Build lite interpreter as default for Android
Summary:
Build lite interpreter as default for android, should wait until https://github.com/pytorch/pytorch/pull/56002 lands
Mainly two changes:
1. Use lite interpreter as default for Android
2. Switch the lite interpreter build test to full jit build test

Test Plan: Imported from OSS

Differential Revision: D27695530

Reviewed By: IvanKobzarev

Pulled By: cccclai

fbshipit-source-id: e1b2c70fee6590accc22c7404b9dd52c7d7c36e2
2021-05-17 14:12:48 -07:00
Sam Estep
4753100a3b Un-ignore F403 in .flake8 (#55838)
Summary:
Generally wildcard imports are bad for the reasons described here: https://www.flake8rules.com/rules/F403.html

This PR replaces wildcard imports with an explicit list of imported items where possible, and adds a `# noqa: F403` comment in the other cases (mostly re-exports in `__init__.py` files).

This is a prerequisite for https://github.com/pytorch/pytorch/issues/55816, because currently [`tools/codegen/dest/register_dispatch_key.py` simply fails if you sort its imports](https://github.com/pytorch/pytorch/actions/runs/742505908).

Pull Request resolved: https://github.com/pytorch/pytorch/pull/55838

Test Plan: CI. You can also run `flake8` locally.

Reviewed By: jbschlosser

Differential Revision: D27724232

Pulled By: samestep

fbshipit-source-id: 269fb09cb4168f8a51fd65bfaacc6cda7fb87c34
2021-04-13 09:24:07 -07:00
Chen Lai
b7b481bd07 [PyTorch] Enable template build at aten operator level (#53801)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/53801

## Summary

Enable partial explicit Aten level sources list for lite interpreter. More aten level source list will be added.

1. Use `gen_selected_mobile_ops_header.py ` to generate `selected_mobile_ops.h`. Currently, it only includes selected operators, and dtypes is all.
2. Add a custom target includes only `seleteted_mobile_ops.h`, and add it to `torch_cpu` dependency, when `BUILD_LITE_INTERPRETER` is enabled.

As a note, the current input yaml file is slightly different than the one use in internal. Will align these two yaml as next step.

**Android**
x86:
`SELECTED_OP_LIST=/Users/chenlai/Documents/pytorch/experiemnt/deeplabv3_scripted.yaml BUILD_LITE_INTERPRETER=1 ./scripts/build_pytorch_android.sh x86`

libpytorch_jni_lite.so -- 3.4 MB

armeabi-v7a
`SELECTED_OP_LIST=/Users/chenlai/Documents/pytorch/experiemnt/deeplabv3_scripted.yaml BUILD_LITE_INTERPRETER=1 ./scripts/build_pytorch_android.sh armeabi-v7a`
libpytorch_jni_lite.so -- 2.5 MB

**iOS:**
```
(base) chenlai@chenlai-mp install % du -sh *
 15M	include
 57M	lib
2.8M	share
```

```
(base) chenlai@chenlai-mp lib % ls -lh
total 117296
-rw-r--r--  1 chenlai  staff   3.2M Mar 15 22:03 libXNNPACK.a
-rw-r--r--  1 chenlai  staff   913K Mar 15 22:03 libc10.a
-rw-r--r--  1 chenlai  staff   4.6K Mar 15 22:03 libclog.a
-rw-r--r--  1 chenlai  staff    42K Mar 15 22:03 libcpuinfo.a
-rw-r--r--  1 chenlai  staff   1.5M Mar 15 22:03 libeigen_blas.a
-rw-r--r--  1 chenlai  staff    44K Mar 15 22:03 libpthreadpool.a
-rw-r--r--  1 chenlai  staff   166K Mar 15 22:03 libpytorch_qnnpack.a
-rw-r--r--  1 chenlai  staff   384B Mar 15 22:03 libtorch.a
-rw-r--r--  1 chenlai  staff    51M Mar 15 22:03 libtorch_cpu.a
```

### **Master (Baseline):**

**Android**
x86:
`SELECTED_OP_LIST=/Users/chenlai/Documents/pytorch/experiemnt/deeplabv3_scripted.yaml BUILD_LITE_INTERPRETER=1 ./scripts/build_pytorch_android.sh x86`

libpytorch_jni_lite.so -- 3.8 MB

armeabi-v7a
`SELECTED_OP_LIST=/Users/chenlai/Documents/pytorch/experiemnt/deeplabv3_scripted.yaml BUILD_LITE_INTERPRETER=1 ./scripts/build_pytorch_android.sh armeabi-v7a`
libpytorch_jni_lite.so -- 2.8 MB

**iOS:**
```
(base) chenlai@chenlai-mp install % du -sh *
 15M	include
 58M	lib
2.8M	share
```

```
(base) chenlai@chenlai-mp lib % ls -lh
total 119600
-rw-r--r--  1 chenlai  staff   3.2M Mar  4 23:16 libXNNPACK.a
-rw-r--r--  1 chenlai  staff   910K Mar  4 23:16 libc10.a
-rw-r--r--  1 chenlai  staff   4.6K Mar  4 23:16 libclog.a
-rw-r--r--  1 chenlai  staff    42K Mar  4 23:16 libcpuinfo.a
-rw-r--r--  1 chenlai  staff   1.5M Mar  4 23:16 libeigen_blas.a
-rw-r--r--  1 chenlai  staff    44K Mar  4 23:16 libpthreadpool.a
-rw-r--r--  1 chenlai  staff   166K Mar  4 23:16 libpytorch_qnnpack.a
-rw-r--r--  1 chenlai  staff   384B Mar  4 23:16 libtorch.a
-rw-r--r--  1 chenlai  staff    52M Mar  4 23:16 libtorch_cpu.a
```

Test Plan: Imported from OSS

Reviewed By: dhruvbird

Differential Revision: D27074814

Pulled By: cccclai

fbshipit-source-id: 762b5ad5b87b6a262444392fd089249c4837ba18
2021-03-25 23:57:48 -07:00
Chen Lai
1772e26f63 [PyTorch] Move selected_mobile_ops.h codegen function to tools (#53786)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/53786

To generate `selected_mobile_ops.h` in OSS, move the header file codegen functions to `tools/lite_interpreter/gen_selected_mobile_ops_header.py` file, so OSS can reuse these functions.
ghstack-source-id: 123754437

Test Plan:
```
buck test //xplat/caffe2:supported_mobile_models_test
```

```
buck run //xplat/caffe2:gen_oplist -- --model_file_list_path @/data/users/chenlai/data/pytorch/oplist_folder/file_list_path.macro  --allow_include_all_overloads --output_dir /home/chenlai/local/data/pytorch/oplist_folder
```

`file_list_path.macro` content is:
```
chenlai@devvm2090:~/fbsource(45a9b7888)$ cat /data/users/chenlai/data/pytorch/oplist_folder/file_list_path.macro
/data/users/chenlai/fbsource/buck-out/gen/aab7ed39/xplat/caffe2/supported_mobile_models_test_op_list/model_operators.yaml
```

In output folder `/home/chenlai/local/data/pytorch/oplist_folder`, these files are generated:
```
selected_mobile_ops.h  selected_operators.yaml  SupportedMobileModelsRegistration.cpp
```

the generated files are the same as before.

{P282056731}

{P282055046}

Reviewed By: dhruvbird, iseeyuan

Differential Revision: D26907868

fbshipit-source-id: 9ba786f9c5674a72cad237ae7baadbe4642c51d5
2021-03-12 00:13:03 -08:00