mirror of https://github.com/zebrajr/pytorch.git synced 2025-12-07 12:21:27 +01:00

History

Raghavan Raman 8af648354f [nnc] Benchmarks for concat (#52592 ) Summary: This PR adds a c++ benchmark for "concat" with 3 different versions - 1) aten::cat, 2) NNC implementation with if-then-else, 3) NNC implementation using multiple loops. It also adds a python benchmark for "concat" which can now be invoked with and without CPU fusion. Here are the results of these benchmarks on a `Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz` machine with `OMP_NUM_THREADS=1` ``` -------------------------------------------------------------------------------------------------------------------------- Benchmark Time CPU Iterations UserCounters... -------------------------------------------------------------------------------------------------------------------------- Concat2D2 (`678fe9f077`)Input/ATen/1/160/1/14/1 1211 ns 1211 ns 567896 GB/s=1.14953G/s Concat2D2 (`678fe9f077`)Input/ATen/1/580/1/174/1 1296 ns 1296 ns 537060 GB/s=4.65362G/s Concat2D2 (`678fe9f077`)Input/ATen/20/160/20/14/1 1823 ns 1823 ns 382052 GB/s=15.2677G/s Concat2D2 (`678fe9f077`)Input/ATen/20/580/20/174/1 3347 ns 3347 ns 210036 GB/s=36.0432G/s Concat2D2 (`678fe9f077`)Input/ATen/8/512/8/512/1 2093 ns 2093 ns 324760 GB/s=31.3061G/s Concat2D2 (`678fe9f077`)Input/NNC/1/160/1/14/1 694 ns 694 ns 1002902 GB/s=2.00692G/s Concat2D2 (`678fe9f077`)Input/NNC/1/580/1/174/1 852 ns 852 ns 803002 GB/s=7.08127G/s Concat2D2 (`678fe9f077`)Input/NNC/20/160/20/14/1 1639 ns 1639 ns 419683 GB/s=16.9828G/s Concat2D2 (`678fe9f077`)Input/NNC/20/580/20/174/1 5956 ns 5956 ns 117833 GB/s=20.2548G/s Concat2D2 (`678fe9f077`)Input/NNC/8/512/8/512/1 3136 ns 3136 ns 224122 GB/s=20.8958G/s Concat2D2 (`678fe9f077`)Input/NNCLoop/1/160/1/14/1 581 ns 581 ns 1209873 GB/s=2.39737G/s Concat2D2 (`678fe9f077`)Input/NNCLoop/1/580/1/174/1 614 ns 614 ns 1132332 GB/s=9.82955G/s Concat2D2 (`678fe9f077`)Input/NNCLoop/20/160/20/14/1 1091 ns 1091 ns 622952 GB/s=25.5247G/s Concat2D2 (`678fe9f077`)Input/NNCLoop/20/580/20/174/1 2399 ns 2399 ns 288376 GB/s=50.289G/s Concat2D2 (`678fe9f077`)Input/NNCLoop/8/512/8/512/1 1500 ns 1500 ns 478360 GB/s=43.6968G/s Concat2D3 (`e23ddf06e9`)Input/ATen/8/512/8/512/8/512/1 2584 ns 2584 ns 266394 GB/s=38.0397G/s Concat2D3 (`e23ddf06e9`)Input/NNC/8/512/8/512/8/512/1 5056 ns 5056 ns 139768 GB/s=19.4416G/s Concat2D3 (`e23ddf06e9`)Input/NNCLoop/8/512/8/512/8/512/1 1917 ns 1917 ns 369626 GB/s=51.2758G/s Concat2D7 (`b5edf329f8`)Input/ATen/8/128/8/256/8/384/8/512/8/512/8/512/8/512/1 3888 ns 3888 ns 178124 GB/s=46.3571G/s Concat2D7 (`b5edf329f8`)Input/NNC/8/128/8/256/8/384/8/512/8/512/8/512/8/512/1 24639 ns 24638 ns 28336 GB/s=7.31481G/s Concat2D7 (`b5edf329f8`)Input/NNCLoop/8/128/8/256/8/384/8/512/8/512/8/512/8/512/1 3093 ns 3093 ns 226326 GB/s=58.265G/s ``` Pull Request resolved: https://github.com/pytorch/pytorch/pull/52592 Reviewed By: bertmaher Differential Revision: D26596701 Pulled By: navahgar fbshipit-source-id: 650fa88febf4423ea49f5a1d3d734edc2294d257		2021-02-24 06:09:32 -08:00
..
cpp/tensorexpr	[nnc] Benchmarks for concat (#52592 )	2021-02-24 06:09:32 -08:00
distributed	Fix benchmarks/distributed/ddp/benchmark.py (#51095 )	2021-01-29 11:10:13 -08:00
fastrnns	Back out "Revert D25717510: Clean up some type annotations in benchmarks/fastrnns" (#50556 )	2021-01-14 15:15:03 -08:00
framework_overhead_benchmark	Remove py2 compatible future imports (#44735 )	2020-09-16 12:55:57 -07:00
functional_autograd_benchmark	Enable distribution validation if __debug__ (#48743 )	2021-01-05 13:59:10 -08:00
operator_benchmark	Fix torch.nn.functional.interpolate microbenchmark for non-4D inputs	2021-02-18 02:07:54 -08:00
overrides_benchmark	Add __torch_function__ for methods (#37091 )	2020-08-05 20:44:13 -07:00
profiler_benchmark	Use libkineto in profiler (#46470 )	2020-11-25 04:32:16 -08:00
record_function_benchmark	Fix D23995953 import.	2020-09-29 19:30:23 -07:00
serialization	[JIT] Make new zip serialization for torch save/load significantly (~70%) faster (#38379 )	2020-05-29 01:56:18 -07:00
sparse	Sparse-sparse matrix multiplication (CPU/CUDA) (#39526 )	2020-12-21 11:53:55 -08:00
static_runtime	[static runtime] Fix up deprecated exact equality in tests (#52617 )	2021-02-22 17:50:14 -08:00
tensorexpr	[nnc] Benchmarks for concat (#52592 )	2021-02-24 06:09:32 -08:00
compare-fastrnn-results.py	Benchmarks: add scripts for FastRNNs results comparison. (#44134 )	2020-09-03 13:44:42 -07:00
compare.sh	Benchmarks: add scripts for FastRNNs results comparison. (#44134 )	2020-09-03 13:44:42 -07:00
README.md	Fix spelling errors	2020-01-28 04:46:15 -08:00
upload_scribe.py	Benchmarks: make fuser and executor configurable from command line. (#44291 )	2020-09-09 11:59:35 -07:00

README.md

PyTorch Benchmarks

NOTE: This folder is currently work in progress.

This folder contains scripts that produce reproducible timings of various PyTorch features.

It also provides mechanisms to compare PyTorch with other frameworks.

Setup environment

Make sure you're on a machine with CUDA, torchvision, and pytorch installed. Install in the following order:

# Install torchvision. It comes with the pytorch stable release binary
conda install pytorch torchvision -c pytorch

# Install the latest pytorch master from source.
# It should supersede the installation from the release binary.
cd $PYTORCH_HOME
python setup.py build develop

# Check the pytorch installation version
python -c "import torch; print(torch.__version__)"

Benchmark List

Please refer to each subfolder to discover each benchmark suite

Fast RNNs benchmarks