Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/61592
Add activation handles for each layer (stored in a list), so they can each be removed.
We don't remove them in the `convert` in eager mode because we aren't modifying output/input layer dimensions. We will need this in Fx mode though.
ghstack-source-id: 133497376
Test Plan:
Added some tests to make sure `model(x)` runs without error.
`buck test mode/dev-nosan //caffe2/test:ao --
TestBasePruner`
https://pxl.cl/1LBf4
Reviewed By: z-a-f
Differential Revision: D29682789
fbshipit-source-id: 9185702736e5f7f4320754ffef441610738ac154
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/61425
Adding handle for activation reconstruction and bias forward hooks so they can be removed later
ghstack-source-id: 133244536
Test Plan:
This change should not affect behavior yet, but to double check:
`buck test mode/dev-nosan //caffe2/test:ao -- TestBasePruner`
https://pxl.cl/1LpM9
Reviewed By: z-a-f
Differential Revision: D29619720
fbshipit-source-id: c7428d2d0325cd11ce7919e0b67321e8cc196041
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/59898
In `weight_norm_sparsifier`, the name of the argument `sparsity_pattern` is not intuitive for an argument describing the shape of the sparse block. It has been changed to `sparse_block_shape`.
Test Plan:
`buck test mode/dev-nosan //caffe2/test:ao -- TestWeightNormSparsifier`
https://pxl.cl/1LhRM
Reviewed By: z-a-f
Differential Revision: D29077045
fbshipit-source-id: 0cf9c5387d41ca8e839ee050d71f4fe477374143
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/59771
Implements a specific sparsity scheduler, that uses a user-provided lambda's to change the levels.
Test Plan:
```
python test/test_ao_sparsity.py
```
Imported from OSS
Differential Revision:
D29070604
D29070604
Reviewed By: raghuramank100
Pulled By: z-a-f
fbshipit-source-id: c7ccbe63fe4cd6a0c3563541b7fcf93a99d0e62f
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/59770
Implements the base scheduler class for changing the sparsity levels in the sparsifier.
Test Plan:
```
python test/test_ao_sparsity.py
```
Imported from OSS
Differential Revision:
D29070603
D29070603
Reviewed By: raghuramank100
Pulled By: z-a-f
fbshipit-source-id: 0b160e4eb0a2a303d2d19e6a3beb4784002b2cb7
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/58955
Implements the weight norm sparsifier.
This type of sparsifier computes the norm of the weights, sorts them, and zeroes-out the target fraction of them.
The main imeplemented method is `update_mask`, which holds the main logic of changing the masks.
Test Plan:
```
python test/test_ao_sparsity.py
```
Imported from OSS
Differential Revision:
D28970960
D28970960
Reviewed By: raghuramank100
Pulled By: z-a-f
fbshipit-source-id: 8f2a4360ad877f430cdc1065c6777106938b58d5
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/58707
Minor refactor that changes the format of the import.
This is done to avoid accidental circular dependencies.
Test Plan:
```
python test/test_ao_sparsity.py
```
Imported from OSS
Differential Revision:
D28970961
D28970961
Reviewed By: raghuramank100
Pulled By: z-a-f
fbshipit-source-id: c312742f5e218c435a1a643532f5842116bfcfff
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/58704
Implements the base sparsifier class based on the #59835 RFC documents.
This PR implements the base class for the sparsification. Specifically, the prepare method is implemented.
Test Plan:
```
python test/test_ao_sparsity.py
```
Imported from OSS
Differential Revision:
D28970958
D28970958
Reviewed By: raghuramank100
Pulled By: z-a-f
fbshipit-source-id: 0ef98a445c0a0aca22ce5708e34a9f94606d0e2b
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/60395
Experimental folder so other developers know this is work in progress
Test Plan:
`buck test mode/dev-nosan //caffe2/test:ao -- TestBasePruner`
https://pxl.cl/1KGJD
Reviewed By: z-a-f
Differential Revision: D29272319
fbshipit-source-id: 93eeeceba0376753efc9a5bb69a155278ceb2fca
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/60278
Implemented `PruningParametrization`, which removes pruned rows, and `BasePruner`, which is the base class for structured pruning.
Test Plan:
`buck test mode/dev-nosan //caffe2/test:ao -- TestBasePruner`
https://pxl.cl/1KC2n
Reviewed By: z-a-f
Differential Revision: D29208349
fbshipit-source-id: f34e8e258bf13fa80292c2bd64d56f5ad1e72b6a
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/56617
This migrates the sparsity to the open source
Test Plan: `buck test mode/opt //caffe2/test:ao`
Reviewed By: raghuramank100
Differential Revision: D27812207
fbshipit-source-id: cc87d9d2b486269901a4ad9b483615741a1cd712
