OSSForks/pytorch
1
0
Fork 0
mirror of https://github.com/zebrajr/pytorch.git synced 2025-12-08 07:39:33 +01:00
Code Issues Actions 38 Packages Projects Releases Wiki Activity
665da61d2b
pytorch/test/cpp_api_parity
History
Xiaomeng Yang 4ae832e106 Optimize SiLU (Swish) op in PyTorch (#42976) 
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/42976

Optimize SiLU (Swish) op in PyTorch.

Some benchmark result

input = torch.rand(1024, 32768, dtype=torch.float, device="cpu")
forward: 221ms -> 133ms
backward: 600ms -> 170ms

input = torch.rand(1024, 32768, dtype=torch.double, device="cpu")
forward: 479ms -> 297ms
backward: 1438ms -> 387ms

input = torch.rand(8192, 32768, dtype=torch.float, device="cuda")
forward: 24.34ms -> 9.83ms
backward: 97.05ms -> 29.03ms

input = torch.rand(4096, 32768, dtype=torch.double, device="cuda")
forward: 44.24ms -> 30.15ms
backward: 126.21ms -> 49.68ms

Test Plan: buck test mode/dev-nosan //caffe2/test:nn -- "SiLU"

Reviewed By: houseroad

Differential Revision: D23093593

fbshipit-source-id: 1ba7b95d5926c4527216ed211a5ff1cefa3d3bfd
2020-08-16 13:21:57 -07:00
..
__init__.py remediation of S205607 2020-07-17 17:19:47 -07:00
functional_impl_check.py Updates assertEqual to require atol and rtol, removes positional atol (#38872) 2020-05-27 06:31:07 -07:00
module_impl_check.py Updates assertEqual to require atol and rtol, removes positional atol (#38872) 2020-05-27 06:31:07 -07:00
parity_table_parser.py Refactor C++ API parity test mechanism and turn it on in CI again (#35190) 2020-04-03 11:20:36 -07:00
parity-tracker.md Optimize SiLU (Swish) op in PyTorch (#42976) 2020-08-16 13:21:57 -07:00
sample_functional.py Refactor C++ API parity test mechanism and turn it on in CI again (#35190) 2020-04-03 11:20:36 -07:00
sample_module.py Refactor C++ API parity test mechanism and turn it on in CI again (#35190) 2020-04-03 11:20:36 -07:00
utils.py Enabled debug symbol in test_cpp_api_parity tests by default. (#36209) 2020-04-08 09:34:32 -07:00